# Wednesday, 23 March 2011
IKVM.Reflection Update

Missing Members

To support mcs I previously added some limited support for missing members. This has now been extended substantially to allow full fidelity reading/writing of assemblies. Normally IKVM.Reflection behaves like System.Reflection in how it tries to resolve member and assembly references, but now it is possible to customize this to enable reading a module without resolving any dependencies.

Roundtripping

To test this version of IKVM.Reflection, I've roundtripped 1758 (non-C++/CLI) modules from the CLR GAC plus the Mono 2.10 directories on my system. Roundtripping here means reading in the module and writing it back out. The original module and resulting module are both disassembled with ildasm and the resulting IL files compared (after going through two dozen regex replacements to filter out some trivial differences).

The code I used for the roundtripping is an (d)evolution of the linker prototype code. It is available here. Note that it is very far from production quality, but does make for a good case study in how to use the advanced IKVM.Reflection functionality.

Unmanaged Exports

A cool CLR feature is the ability to export static managed methods as unmanaged DLL exports. The only languages that support this, AFAIK, are C++/CLI and ILASM, but now you easily do it with IKVM.Reflection as well. Below is a Hello World example. Note that while this example exports a global method, you can export any static method. Note also that the string passed in must be ASCII, may be null and that the ordinal must be a 16 bit value that is unique (in the scope of the module).

using IKVM.Reflection;
using IKVM.Reflection.Emit;

class Demo
{
  static void Main()
  {
    var universe = new Universe();
    var name = new AssemblyName("ExportDemo");
    var ab = universe.DefineDynamicAssembly(name, AssemblyBuilderAccess.Save);
    var modb = ab.DefineDynamicModule("ExportDemo", "ExportDemo.dll");
    var mb = modb.DefineGlobalMethod("HelloWorld", MethodAttributes.Static, null, null);
    var ilgen = mb.GetILGenerator();
    ilgen.EmitWriteLine("Hello World");
    ilgen.Emit(OpCodes.Ret);
    mb.__AddUnmanagedExport("HelloWorld", 1);
    modb.CreateGlobalFunctions();
    ab.Save("ExportDemo.dll", PortableExecutableKinds.Required32Bit, ImageFileMachine.I386);
  }
}

The resulting ExportDemo.dll can now be called from native code in the usual ways, for example:

    typedef void (__stdcall * PROC)();
    HMODULE hmod = LoadLibrary(L"ExportDemo.dll");
    PROC proc = (PROC)GetProcAddress(hmod, "HelloWorld");
    proc();

Creating unmanaged exports is supported for I386 and AMD64. Itanium (IA64) support has not been implemented.

What's Missing

It should now be possible to write an ILDASM clone using IKVM.Reflection, however to be able to write ILASM there are still a couple of things missing:

  • Function pointers (used by C++/CLI).
  • API to create missing type.
  • Preserving interleaved modopt/modreq ordering.
  • Various C++/CLI quirks (e.g. custom modifiers on local variable signatures).
  • Ability to set file alignment.

Changes:

  • Added support for using missing members in emitted assembly.
  • If you build IKVM.Reflection with the STABLE_SORT symbol defined, all metadata sorting will be done in a stable way, thus retaining the order in which the items are defined.
  • When target is .NET 1.x adjust the type name writing algorithm to match the quirks of the Microsoft .NET 1.x metadata writer.
  • Fixed Type.StructLayoutAttribute to be (bug) compatible with .NET.
  • Don't map unspecified pinvoke calling convention to winapi.
  • Don't needlessly read the contents of a resource module.
  • Automatically emit .NET 1.x compatible declarative security when appropriate.
  • Added support for having multiple mscorlib versions in a universe.
  • Added Assembly.__AssemblyFlags property.
  • Added CustomAttributeBuilder.__FromBlob().
  • Added CustomAttributeBuilder.__MakeTypedArgument() to make it possible to box an enum or create an array of an enum type in a location typed as object.
  • Added ILGenerator.__MaxStackSize property.
  • Added EventInfo.__GetMethods().
  • Added FieldBuilder.__SetReadOnlyDataAndRVA().
  • Added FieldInfo.__FieldRVA property.
  • Added FieldInfo.__GetFieldOnTypeDefinition().
  • Added ManifestResourceInfo.__ResourceAttributes property.
  • Added MethodBase.__GetMethodOnTypeDefinition().
  • Added MethodBuilder.__AddUnmanagedExport().
  • Added MethodBuilder.__SetCallingConvention().
  • Added Module.__ImageBase property.
  • Added Module.__GetCustomAttributesFor().
  • Added Module.__GetExportedTypes().
  • Added Module.__GetReferencedModules().
  • Added Module.__GetReferencedTypes().
  • Added Module.__GetSectionInfo().
  • Added Module.__ModuleHash property (for resource modules).
  • Added Module.__ReadDataFromRVA().
  • Added Module.__ResolveReferencedAssemblies().
  • Added Module.__StackReserve property.
  • Added ModuleBuilder.__AddModule().
  • Added ModuleBuilder.__AddModuleReference().
  • Added ModuleBuilder.__AddAssemblyReference(AssemblyName, Assembly) overload.
  • Added ModuleBuilder.__AddUnmanagedExportStub().
  • Added ModuleBuilder.__AddVTableFixups().
  • Added ModuleBuilder.__SetCustomAttributeFor().
  • Added ModuleBuilder.__SetModuleVersionId().
  • Added ModuleBuilder.__SetStackReserve().
  • Added PropertyInfo.__CallingConvention property.
  • Added Type.__GetLayout().
  • Added Type.__CreateMissingMethod(), Type.__CreateMissingField() and MissingGenericMethodBuilder.
  • Added Type.__GetArraySizes() and Type.__GetArrayLowerBounds().
  • Added Type.__MakeArrayType() overload that takes sizes and lower bounds.
  • Added TypeBuilder.__SetLayout().

Binary available here: IKVM.Reflection-0.47.4098.zip

Wednesday, 23 March 2011 08:04:24 (W. Europe Standard Time, UTC+01:00)  #    Comments [6]
# Thursday, 17 March 2011
IKVM.NET 0.46 Release Candidate 1

A new release candidate is available.

Changes since previous release candidate:

  • Changed version to 0.46.0.1.
  • Zip inflater fix (Classpath bug #48131).
  • Support loading resources from assemblies added to the boot classloader (with ikvm.runtime.Startup.addBootClassPathAssemby()).
  • IKVM.Reflection: ILGenerator.BeginExceptFilterBlock() should behave like BeginCatchBlock(), not BeginFinallyBlock().
  • IKVM.Reflection: Type.FullName for a generic type should separate type parameters by commas.
  • IKVM.Reflection: Type.FullName for a generic type should only escape the ']' characters in the assembly names of type parameters.
  • IKVM.Reflection: <Module> should not extend object.
  • IKVM.Reflection: String literals should be read in "as-is", not interpreted as UTF-16.
  • IKVM.Reflection: MethodBody.MaxStack returned incorrect value for tiny header methods.
  • IKVM.Reflection: Getting Module methods/fields should not try to look at the base class of the <Module> type.
  • IKVM.Reflection: Align initialized data arrays on 8 byte boundary.
  • IKVM.Reflection: Align managed resources on 8 byte boundary.

Binaries available here: ikvmbin-0.46.0.1.zip

Sources: ikvmsrc-0.46.0.1.zip, openjdk6-b22-stripped.zip

Thursday, 17 March 2011 06:49:16 (W. Europe Standard Time, UTC+01:00)  #    Comments [4]
# Wednesday, 16 March 2011
.NET/C# Generics History

Yesterday Don Syme posted a short history of .NET/C# generics.

Wow. Just wow.

I knew MSR was instrumental in getting generics into the CLR, but not that it was this critical.

Thank you Don Syme, Andrew Kennedy, et al.

Wednesday, 16 March 2011 06:41:40 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Thursday, 03 March 2011
IKVM.NET 0.46 Release Candidate 0

Finally, after way too long, the first 0.46 release candidate is available.

What's New (relative to IKVM.NET 0.44):

  • Integrated OpenJDK 6 b22.
  • Added -Xreference: option to ikvm.exe and ikvm.runtime.Startup.addBootClassPathAssemby() API.
  • The binaries zip now includes lib/ikvm-api.jar to avoid having to run ikvmstub to get access to the IKVM Java APIs.
  • Ported parts of fdlibm to C# to use for StrictMath methods tan, cbrt, floor, pow, hypot, expm1 and log1p.
  • Add support for serializing .NET exceptions in Java. We don't serialize the actual object, but a placeholder instead, to avoid having to implement full .NET serialization interop.
  • Added IL optimization step to code generator.
  • Added SynchronizationContext for AWT event thread.
  • Many IKVM.Reflection improvements and bug fixes.
  • Many Swing/AWT improvements.
  • Implemented IPv6 support (.NET only) for java.net package APIs.
  • The sun.boot.class.path system property now points to VFS to allow javac to work (although a bit slow, because the boot class files are dynamically generated on demand).
  • Resource and generated stub classes are now projected into the virtual IKVM home directory, to make code that assumes that resources live in jars happy.
  • Improvements to @ikvm.lang.Internal handling.
  • Many minor improvements.

Fixes since previous development snapshot:

  • IKVM.Reflection: Added limited support for #- metadata stream ("uncompressed" table heap). ParamPtr table is not yet implemented.
  • IKVM.Reflection: Fixed bug in CustomAttributeData.__ToBuilder(). Array values should be unwrapped.
  • IKVM.Reflection: Added __GetBlob() API extension to allow custom attribute arguments to be parsed/used even when some of the argument types are missing.
  • IKVM.Reflection: Fixed stack height computation bug. For a newobj the parameters should be popped before the new object is pushed. Previously we could set the maximum stack height one slot too high.
  • IKVM.Reflection: Added ConstructorBuilder.__SetSignature() API extension to allow constructor to be defined before the parameter types are available and to allow return type custom modifiers.

Binaries available here: ikvmbin-0.46.0.0.zip

Sources: ikvmsrc-0.46.0.0.zip, openjdk6-b22-stripped.zip

Thursday, 03 March 2011 14:33:03 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Wednesday, 02 March 2011
New Development Snapshot

OpenJDK 6 b22 has been integrated.

Changes:

  • Integrated OpenJDK 6 b22.
  • Fixed runtime to not convert assembly class loader construction exceptions into critical failure, because critical failure is of dubious value and exception might be handleable by client code.
  • Fixed ProtectionDomain. Use Assembly.EscapedCodeBase to construct CodeSource url, instead of unescaped CodeBase.
  • Allow @ikvm.lang.Internal methods to implement an interface method.
  • Fixed ikvmc to ignore @ikvm.lang.Internal annotation on interface methods and emit a warning.
  • IKVM.Reflection: Fixed exception table sorting bug.
  • IKVM.Reflection: Allow mcs to avoid mdb/pdb writer dependencies.
  • IKVM.Reflection: Added Mono Ref.Emit compatibility feature to allow modules currently in use to be overwritten.

Binaries available here: ikvmbin-0.45.4078.zip

OpenJDK 6 b22 stripped sources: openjdk6-b22-stripped.zip

Wednesday, 02 March 2011 09:25:17 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Friday, 25 February 2011
CLR Type Names

Type names look like a simple concept. Every type has a unique name within the assembly that defines it.

It turns out that there is a slight complication. Even though the CLI specification and the reflection API suggest that a type name is simply a string, in reality it is a pair of strings: { namespace, name }

Here's some code that uses IKVM.Reflection to generate an interesting assembly:

using IKVM.Reflection;
using IKVM.Reflection.Emit;

class Program {
  static void Main() {
    var universe = new Universe();
    var ab = universe.DefineDynamicAssembly(new AssemblyName("Test"), AssemblyBuilderAccess.Save);
    var modb = ab.DefineDynamicModule("Test", "Test.dll");
    modb.__DefineType("A.B", "C").CreateType();
    modb.__DefineType("A", "B.C").CreateType();
    ab.Save("Test.dll");
  }
}

This creates a valid (and verifiable) assembly containing two different types, both named A.B.C.

If you disassemble this assembly with ildasm and the resulting IL is reassembled with ilasm you won't end up with the same assembly. I don't know if there are any obfuscators that use this trick, but maybe they should.

Reflection APIs assume that the last dot in the type name separates the namespace from the name, so doing Type.GetType("A.B.C") will return the first type {"A.B", "C"}. You can get the second type by enumerating all types in the assembly.

Note that static binding just works, because in that case the {namespace, name} pair is specified explicitly.

Friday, 25 February 2011 09:09:01 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Monday, 21 February 2011
Exception Performance Part 3

This is the final part of three part series on exception performance that started in 2008. Previous parts are:
Exception Performance Part 1
Exception Performance Part 2

Let's introduce a slight variation of ExceptionPerf1 where we throw 10000 exceptions instead of 100000 and throw the exception from a method, instead of directly in the loop.

using System;
using System.Diagnostics;

class ExceptionPerf5 {
  static void Main() {
    var sw = Stopwatch.StartNew();
    for (int i = 0; i < 10000; i++) {
      try {
        Foo();
      } catch { }
    }
    sw.Stop();
    Console.WriteLine(sw.ElapsedMilliseconds);
    Console.ReadLine();
  }

  private static void Foo() {
    throw new Exception();
  }
}

When this is compiled with the standard Debug configuration in Visual Studio 2010 it yields the following performance numbers (times in milliseconds) when run either with Ctrl-F5 (i.e. no debugger attached) or F5 (debugger attached):

  HotSpot 1.6 .NET 2.0 .NET 4.0
  x86 x86 x64 x86 x64
Ctrl-F5 25 286 413 292 305
F5   16950 16134 45223 114790

For comparison, the table also includes the time it takes to run the equivalent code in HotSpot 1.6 Client VM on x86.

As we saw in the previous two articles on exception performance, .NET is significantly slower in handling exceptions, so that is not surprising. However, what is surprising is how much the overhead is of simply having the debugger attached.

Another depressing thing to note is that things have gotten much worse with .NET 4.0.

Unfortunately, many developers have the habit of always running their code in the debugger, so when they first try code IKVM.NET compiled code from within Visual Studio they often get a very bad impression of the performance, simply because the debugger sucks.

I considered filing a connect bug for this, but you know they'll just close it as By Design. I guess the CLR is only a Common Language Runtime, if you language doesn't use exceptions for control flow.

Monday, 21 February 2011 09:46:12 (W. Europe Standard Time, UTC+01:00)  #    Comments [5]
# Thursday, 17 February 2011
Mono 2.10 Released

Mono 2.10 was released this week. It includes a version of the Mono C# compiler that uses IKVM.Reflection as its back end.

Last year in the two days before FOSDEM I hacked mcs to use IKVM.Reflection and while at FOSDEM I showed this hack to Miguel and was met with his usual enthusiasm and he told me that he was already planning on talking to me about using IKVM.Reflection for mcs.

In May, Kornél Pál did a, much more complete than my hack, prototype port of mcs to IKVM.Reflection and that resulted in a number of IKVM.Reflection bug fixes and enhancements.

Last November, Marek Safar starting work on integrating IKVM.Reflection support for real and made sure that everything was production ready. This resulted in some mcs restructuring and yet more IKVM.Reflection fixes, features and also performance improvements.

The result of all this is that now IKVM.Reflection is a great library to use if you have a codebase that has both a dynamic and a static compilation mode (as both IKVM.NET and mcs have), because you can easily share the bulk of your code between System.Reflection and IKVM.Reflection without having to suffer from the (significant) limitations of System.Reflection for the static scenario.

Thursday, 17 February 2011 06:59:38 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Tuesday, 15 February 2011
New Development Snapshot

I decided to delay the 0.46 release to wait for OpenJDK 6 build 22, which will be a security update and should be available soon. In the mean time, here is a new development snapshot.

Changes:

  • Fixed some compilation warnings.
  • Fix for the timezone used to convert .NET DateTime to Java timezone in the JDBC-ODBC bridge.
  • Fix for bug # 3129981.
  • Fixed .NET 4.0 build verification issue.
  • IKVM.Reflection: Enabled framework unification support.
  • IKVM.Reflection: Avoid re-firing the assembly resolve event for missing assemblies.
  • IKVM.Reflection: Several performance tweaks by Marek Safar.
  • IKVM.Reflection: Added feature to (optionally) allow resolving of missing methods.
  • IKVM.Reflection: Added [Constructor|Method]Builder.__ReleaseILGenerator() API to eagerly bake the method body and release the ILGenerator.

Binaries available here: ikvmbin-0.45.4062.zip

Tuesday, 15 February 2011 07:54:58 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Tuesday, 25 January 2011
New Development Snapshot

A new snapshot with OpenJDK 6 b21 integrated. If you want to build from cvs, from now on you'll need the OpenJDK 6 b21 sources (available in the zip linked below).

Changes:

  • Integrated OpenJDK 6 b21.
  • IKVM.Reflection: Mark modules as executable when running on Mono.

Binaries available here: ikvmbin-0.45.4042.zip

OpenJDK 6 b21 stripped sources: openjdk6-b21-stripped.zip

Tuesday, 25 January 2011 06:07:07 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]