# Monday, April 23, 2012
IKDASM - IL Disassembler Example for IKVM.Reflection

A while ago I wrote an IL disassembler to test IKVM.Reflection. Both as a correctness test and also to test if the API surface exposes enough of the underlying information.

I thought it would make a good IKVM.Reflection example (although the code won't win any awards, it's a bit of a hack). One nice feature is that it tries really hard to emit the same output file as the .NET ildasm to make comparison easier. There's even a command line option to match a specific ildasm version (2.0, 4.0 or 4.5) and its quirks.

The binaries are available in ikdasm-v0.1-binaries.zip and the Visual Studio 2010 solution in ikdasm-v0.1.zip.

Note that in its current form the ildasm compatibility mode only works on Windows, because it needs to P/Invoke _gcvt in msvcrt.dll to make sure the floating point numbers match the ildasm output.

Monday, April 23, 2012 4:54:40 PM (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Thursday, April 19, 2012
New Development Snapshot

At Lang.NEXT I met someone who was interested in using IKVM.Reflection and after he started porting his code, he ran into some missing functionality in IKVM.Reflection, so I've made some improvements there.

I've also fixed the remaining known issues with access stubs and "unloadable" (missing types) custom modifiers.

Changes:

  • Fix for recently introduced bug (with access stub rewrite). Bug #3512589.
  • Changed build process to fall back to the NAnt task, if we can't find the resource compiler.
  • Made WinForms message loop thread creation lazy to hopefully allow more applications to run without message loop thread. This is a (partial) workaround for bug #3515033.
  • Changed ikvmc to read input files after processing all the options (to make -nowarn: and -warnaserror: options that follow the file names work for warnings produced during input file reading).
  • Added support for type 2 access stubs for constructors.
  • Bug fix. When an unloadable type is used in a method signature that overrides a method (or implements an interface method), the custom modifier must be the same as the base class or an override stub must be generated.
  • Added partial implementation of ThreadMXBean.
  • IKVM.Reflection: Bug fix. When writing an assembly that has a .netmodule, the TypeDefId field in the ExportedType in the manifest module should contain a TypeDef token instead of an index.
  • IKVM.Reflection: Bug fix. When exporting a nested type (via AssemblyBuilder.__AddTypeForwarder()), we should also set the namespace (in practice it is unlikely for a nested type to have a namespace, but is is possible).
  • IKVM.Reflection: Corrected a couple of method parameter names in Assembly.
  • IKVM.Reflection: Added Assembly.GetType(string,bool,bool) method.
  • IKVM.Reflection: Added support for case-insensitive type and member lookup.
  • IKVM.Reflection: Implemented case insensitive lookup in Type.GetInterface().
  • IKVM.Reflection: Moved GetEvents(), GetFields(), GetConstructors(), GetNestedTypes() and GetProperties() to a common implementation that fixes a number of bugs.
  • IKVM.Reflection: Fixed GetMethods() to properly filter out base class methods that have been overridden.
  • IKVM.Reflection: Moved member lookup by name to a common implementation that fixes a number of bugs and adds IgnoreCase support.
  • IKVM.Reflection: Added MemberInfo.ReflectedType.
  • IKVM.Reflection: Added Binder support for method and property lookup.
  • IKVM.Reflection: Bug fix. ParameterBuilder.Position should return the 1-based position passed in to DefineParameter, not the 0-based ParameterInfo.Position.
  • IKVM.Reflection: Changed Type.__ContainsMissingType to return true for generic type parameters that have constraints that return true for __ContainsMissingType.
  • IKVM.Reflection: Bug fix. It should be possible to import a function pointer type into a ModuleBuilder.
  • IKVM.Reflection: Implemented Assembly.ToString().
  • IKVM.Reflection: Added [Flags] attribute to ResourceLocation enum.
  • IKVM.Reflection: Added support for reading/querying manifest resources that are forwarded to another assembly.
  • IKVM.Reflection: Bug fix. Module.GetManifestResourceStream() should return null (instead of throwing FileNotFoundException) for non-existing resource names.

Binaries available here: ikvmbin-7.1.4491.zip

Thursday, April 19, 2012 9:26:43 AM (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Thursday, April 12, 2012
MS12-025

This patch Tuesday Microsoft released MS12-025 that fixes approximately a zillion vulnerabilities in System.Drawing.dll.

Here's what they fixed (multiple instances of each issue):

  • Added security demands to unsafe methods.
  • Wrap handles in SafeHandle instead of using IntPtr.
  • Use checked to guard against integer overflow when calculating how many bytes to AllocHGlobal.

It's a little embarassing to have so many vulnerabilities in this old code. Luckily, silently running .NET code in the browser is becoming a thing of the past.

Thursday, April 12, 2012 12:33:30 PM (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Friday, April 6, 2012
Lang.NEXT

I'm back home after a really great Lang.NEXT conference.

The video of my talk is available on Channel 9. The slides are here.

Friday, April 6, 2012 10:24:37 PM (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Monday, March 26, 2012
New Development Snapshot

Enough changes to warrant a new development snapshot.

Changes:

  • Fixes and improvements to Windows version of JFileChooser.
  • Bug fix. When writing a direct ByteBuffer to a non-blocking socket and the write fails because there is no kernel buffer available, we should not advance the ByteBuffer position.
  • Bug fix. Non-public property getter/setter methods should be ignored when we create properties to hide properties inherited from shadow types. This fixes a build break with .NET 4.5 beta which introduces a protected setter for Exception.HResult.
  • Bug fix. Custom attribute properties that don't have a public getter and setter should not be exposed as annotation properties.
  • Added ikvmc -win32manifest: option.
  • Added ikvmc -filealign: option.
  • Added ikvmc -highentropyva option (to enable high entropy ASLR in 64 bit processes on Windows 8).
  • Added support for custom paper format to Win32 print service.
  • Changed ikvmstub to create a missing assembly when a dependency is not found and only complain about it when it is actually needed.
  • Added explicit -help and -? options to ikvmc.
  • Added ikvmc -nologo option.
  • Changed ikvmc to print copyright when compiling (unless -nolog is specified).
  • Cleaned up to ikvmc help message.
  • Lots of ikvmc error handling clean up. All errors now have an IKVMCnnnn error code.
  • Added support to ikvmc to automatically set the full source path in the debugging info if the source file lives next to the .class file.
  • Improved BufferedImage.setRGB().
  • IKVM.Reflection: Added AssemblyBuilder.__DefineManifestResource() API to add a Win32 manifest resource.
  • IKVM.Reflection: Various win32 resource related methods on AssemblyBuilder now throw ArgumentException if a conflicting resource has already been defined.
  • IKVM.Reflection: Marked ModuleBuilder.__SetStackReserve() obsolete and made ModuleBuilder.__StackReserve property writeable to be consistent with __ImageBase property.
  • IKVM.Reflection: New API. Made ModuleBuilder.__FileAlignment writeable.
  • IKVM.Reflection: New API. Added Module.__DllCharacteristics and ModuleBuilder.__DllCharacteristics properties to get and set image DLL characteristics flags.

Binaries available here: ikvmbin-7.1.4468.zip

Monday, March 26, 2012 8:53:15 AM (W. Europe Daylight Time, UTC+02:00)  #    Comments [10]
# Friday, March 16, 2012
IKVM.Reflection: Inspecting an Assembly Without Loading Dependencies

One of the advantages of IKVM.Reflection over System.Reflection is that it is much easier to load an assembly without having to load its dependencies. This functionality is used by Mono's C# compiler to load referenced assemblies without having to (potentially) load the transitive closure of dependencies of those assemblies.

Here's a simple example that loads an assembly and prints its types and members:

using System;
using IKVM.Reflection;

class Program {
  const BindingFlags AllDeclared =
    BindingFlags.Public |
    BindingFlags.NonPublic |
    BindingFlags.Instance |
    BindingFlags.Static |
    BindingFlags.DeclaredOnly;

  static void Main(string[] args) {
    var universe = new Universe();
    universe.AssemblyResolve += AssemblyResolve;
    var assembly = universe.LoadFile(args[0]);
    foreach (var type in assembly.GetTypes()) {
      Console.WriteLine(type.FullName);
      WriteMembers(type.GetFields(AllDeclared));
      WriteMembers(type.GetProperties(AllDeclared));
      WriteMembers(type.GetEvents(AllDeclared));
      WriteMembers(type.GetConstructors(AllDeclared));
      WriteMembers(type.GetMethods(AllDeclared));
    }
  }

  static void WriteMembers(MemberInfo[] members) {
    foreach (var member in members)
      Console.WriteLine(" {0}", member);
  }

  static Assembly AssemblyResolve(object sender, IKVM.Reflection.ResolveEventArgs args) {
    return ((Universe)sender).CreateMissingAssembly(args.Name);
  }
}

When you don't handle the Universe.AssemblyResolve event (or non of the handlers return an assembly), IKVM.Reflection will fall back to using System.Reflection.Assembly.ReflectionOnlyLoad() and then use the Location property of the returned assembly and load the assembly from there.

If you do handle the Universe.AssemblyResolve event, you can either find the assembly somewhere and load it, or use Universe.CreateMissingAssembly() to create a placeholder Assembly that will automatically resolve all types and members that it is expected to contain (based on references from loaded assemblies that are inspected).

When you're reflecting over the members you can encounter types and members from the missing assemblies and you should be prepared to handle these because many operations are not allowed on them (since there is no type or member definition, a lot of information simply isn't available). You can detect these missing types or members by looking at the MemberInfo.__IsMissing property.

Friday, March 16, 2012 1:26:07 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Thursday, March 15, 2012
Lesser Known CLR Custom Attributes -- UnsafeValueType

In a comment to the previous post about CLR Custom Attributes I listed some other custom attributes that the CLR recognizes (by name). Some of them I previously thought were compiler only custom attributes, so I decided to investigate them.

System.Runtime.CompilerServices.UnsafeValueTypeAttribute

The documentation for this attribute, somewhat uncharacteristically, actually explains what it does, but I decided to try it out.

Here's an example that demonstrates what it does:

using System;
using System.Runtime.CompilerServices;

//[UnsafeValueType]
struct Foo {
  public int field;
}

class Program {
  [MethodImpl(MethodImplOptions.NoOptimization)]
  static void Main() {
    int i = 1234;
    Foo foo = new Foo();
    Corrupt(ref foo, ref i);
    Console.WriteLine(i);
  }

  [MethodImpl(MethodImplOptions.NoInlining)]
  unsafe static void Corrupt(ref Foo foo, ref int unused) {
    fixed (int* p = &foo.field) {
      *(long*)p = 4567L << 32;
    }
  }
}

When you run this it prints out 4567 and terminates successfully. However, when you uncomment the //[UnsafeValueType] line and then run it again, you'll see that it prints out 1234 and crashes and if you attach a debugger you see that it crashes with error code STATUS_STACK_BUFFER_OVERRUN because the CLR inserted a canary on the stack after the unsafe value type.

As the documentation indicates, both the C++ and C# compiler use this attribute. The C++ compiler uses it to implement /GS for managed code and the C# compiler automatically applies it to the value types that it creates to represent fixed size buffers.

Thursday, March 15, 2012 9:42:27 AM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Monday, March 12, 2012
New Development Snapshot

It's been a while since I released a development snapshot. Volker fixed several AWT issues and I've been doing mostly IKVM.Reflection work.

Changes:

  • Prevent IKVMC0109 warning when a stub is encountered after the corresponding type has already been loaded.
  • Bug fix. The $Method inner class for delegates should also be loadable for generic delegates. Thanks to Michael Bayne for reporting this.
  • Add dummy splashscreen native library to VFS. Fix for #3480917.
  • Fix a bug with a ToolkitImage as frame icon.
  • Fix the native file dialog, it was created in the wrong thread and the method blockWindows(List) was not implemented
  • Synchronized the BufferedImage bitmap to fix "System.InvalidOperationException: The object is currently in use elsewhere." if the BufferedImage is used from multiple threads (as is valid in Java).
  • Workaround a problem in sun.util.locale.LocaleObjectCache that assumes that a SoftReference is always immediately enqueued when get() returns null. Now we actively enqueue the reference in get() when the reference was cleared by the GC (instead of waiting for the QueueWatcher to eventually enqueue the reference from the finalizer).
  • Merged security changes to AtomicReferenceArray.
  • Intrinsified the unsafe.objectFieldOffset(XXX.class.getDeclaredField("xxx")) pattern to avoid expensive reflection field lookup in static initializers of common OpenJDK classes.
  • Override Toolkit.areExtraMouseButtonsEnabled() to avoid infinite recursion.
  • Fix a deadlock for TooltipImages which complete with a ImageObserver.FRAMEBITS instead with ImageObserver.ALLBITS. The problem occur with JDownloader.
  • Replace all RasterOp with the versions from GNU Classpath because the original use native code that was not ported and didn't work.
  • Fixed a NullReferenceException in ConvertRegion.
  • Added support for window with transparent background.
  • Added support for undecorated Frame and Dialog.
  • Fixed IPv6 address bug. When the scope ID is zero we should pass -1 to the Inet6Address constructor (this will cause its scope_id to remain 0 and scope_id_set to remain false).
  • Added Unsafe methods used by Akka.
  • Avoid linking class constant pool entries that aren't used (or are only used by name). This avoids spurious IKVMC0100 warnings and unnecessary class load attempts.
  • IKVM.Reflection: Performance tweak to AssemblyName.FullName. Inspired by Marek Safar.
  • IKVM.Reflection: TypeNameParser.Escape() performance tweak by Marek Safar.
  • IKVM.Reflection: Added UniverseOptions.DisableFusion to explicitly disable Fusion (without having to resort to setting the IKVM_DISABLE_FUSION environment variable).
  • IKVM.Reflection: Optimized assembly lookup. Thanks to Marek Safar for the pointer.
  • IKVM.Reflection: Cache the assembly FullName in Assembly. This brings the Assembly.FullName property performance more inline with System.Reflection which also caches the FullName (computing the FullName is expensive).
  • IKVM.Reflection: Use binary search for sorted metadata table lookups.
  • IKVM.Reflection: Added new API MethodInfo.__GetMethodImpls() to efficiently get the MethodImpls for a specific method.
  • IKVM.Reflection: Bug fix. TypeDefImpl.__GetMethodImplMap() should populate typeArgs before using it.
  • IKVM.Reflection: Added RawModule.ModuleVersionId property.
  • IKVM.Reflection: Support building an "mscorlib" assembly that is not named mscorlib.
  • IKVM.Reflection: Added Type.__CreateMissingProperty() API to enable symbolic properties in custom attributes.
  • IKVM.Reflection: Added Universe.MissingTypeIsValueType event to enable missing types (i.e. symbolic types) to be used in signatures.

Binaries available here: ikvmbin-7.1.4454.zip

Monday, March 12, 2012 10:50:41 AM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Thursday, March 8, 2012
CLR Supported Custom Attributes

After working with the CLR for more than a decade, once in a while I still run into surprising behavior.

It turns out that the (non pseudo-) custom attributes that CLR recognizes are only matched by name, not assembly.

So you can do this for example:

using System;
using System.Threading;

namespace System {
  class ThreadStaticAttribute : global::System.Attribute { }
}

class Program {
  [System.ThreadStaticAttribute]
  static int foo;

  public static void Main() {
    WriteFoo();
    foo = 42;
    WriteFoo();
    new Thread(WriteFoo).Start();
  }

  static void WriteFoo() {
    Console.WriteLine(foo);
  }
}

On the CLR the foo static variable is a thread local, but on Mono it isn't.

Thursday, March 8, 2012 2:35:51 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [5]
# Sunday, March 4, 2012
Lang.NEXT

I'm looking forward to speaking at Lang.NEXT. To be held at the Microsoft Campus on April 2 - 4.

If you're in the neighborhood and are interested in programming language design and implementation, this is your opportunity to find out if there really is such a thing as a free lunch.

Sunday, March 4, 2012 11:15:02 AM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]