# Tuesday, 19 June 2012
Ten Years of IKVM.NET

Ten years ago today I started blogging about IKVM.NET. It's been an amazing journey and when I started it I never would have guessed I'd still be on it ten years later.

Some of the highlights include making many new friends, meeting industry luminaries, speaking at conferences, working with the GNU Classpath community (and the fun FOSDEM weekends), working with the Mono community and, of course, the open sourcing of OpenJDK.

The end of the journey is not yet in sight as I still have enormous amounts of fun and both Java and .NET keep adding new features that keep things interesting.

Some statistics:

  • 540 blog entries
  • 739 blog comments
  • 22 releases
  • Over 200000 downloads
  • 3637 messages to the ikvm-developers mailing list
  • 4577 messages to the ikvm-commit mailing list
  • 10 security vulnerabilities reported
  • 9 CLR JIT bugs encountered.
  • 2 name changes (from I<<K.VM.NET to IK.VM.NET to IKVM.NET)
Tuesday, 19 June 2012 08:49:34 (W. Europe Daylight Time, UTC+02:00)  #    Comments [10]
# Wednesday, 13 June 2012
IKVM.NET 7.1 Release Candidate 1

The second release candidate is available. Unlike the previous rc, this one is buildable on Linux again (tested with Mono 2.10.5). It also fixes a regression in final field handling.

Changes (relative to rc 0):

  • Updated version to 7.1.4532.1
  • Fixed Linux build issue due to assembly.class filename case error in tools.rsp
  • Updated copyright years in LICENSE
  • Merged in OpenJDK 7u4 changes in THIRD_PARTY_README.
  • Bug fix. AssemblyClassLoader.InternalsVisibleToImpl() would crash with NRE if it got called on a single assembly class loader, because it should call GetLoader(Assembly) to get the AssemblyLoader instead of GetLoaderForExportedAssembly().
  • Bug fix. When resolving properties corresponding to fields with type 2 access stubs, unloadable types with the same name should compare as equal.
  • Bug fix. When a final field is wrapped in a property, any assemblies that are concurrently compiled with the declaring assembly will access the backing field directly and hence the declaring assembly will need an InternalsVisibleToAttribute to allow them access. This fix makes sure that this attribute is applied when the field is accessed from another (concurrently compiled) assembly.
  • IKVM.Reflection: Added workaround for Mono 2.10 bug in AssemblyName (public key token for ECMA public key is not created correctly).
  • IKVM.Reflection: Added workaround for Mono to StrongNameKeyPair.
  • IKVM.Reflection: Disallow key container constructor of StrongNameKeyPair when running on Mono on Windows.
  • IKVM.Reflection: Bug fix. Type.GetInterfaces() should work for unbaked types.

When the final release is done, it will include the full release notes.

Binaries available here: ikvmbin-7.1.4532.1.zip

Sources: ikvmsrc-7.1.4532.1.zip, openjdk-7u4-stripped.zip

Wednesday, 13 June 2012 08:35:13 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Wednesday, 30 May 2012
IKVM.NET 7.1 Release Candidate 0

The first release candidate is available. No changes (except the version number and strong naming) relative to the last development snapshot.

What's New (relative to IKVM.NET 7.0):

  • Integrated OpenJDK 7u4 b22.
  • Much improved ikvmc error and warning handling.
  • Added ikvmc options: -win32manifest, -filealign, -highentropyva.
  • Removed ikvmc -platform:Itanium option.
  • Added ikvm.lang.DllExport annotation to export static methods as unmanaged entry points.
  • Added ikvm.runtime.Util.getClassFromTypeHandle() overload for classes that represent arrays of remapped .NET types and .NET primitives.
  • Added extension methods for (almost) all instance methods in Object, String and Throwable to ikvm.extensions.ExtensionMethods.
  • Added support for delegates with ByRef parameters.
  • Added support to ikvmc to automatically set the full source path in the debugging info if the source file lives next to the .class file.
  • When adding certificates to virtual cacerts file make sure that the aliases that are generated from the certificate subject are unique.
  • Many (minor) bug fixes.
  • Many IKVM.Reflection fixes and improvements.

When the final release is done, it will include the full release notes.

Binaries available here: ikvmbin-7.1.4532.0.zip

Sources: ikvmsrc-7.1.4532.0.zip, openjdk-7u4-stripped.zip

Wednesday, 30 May 2012 09:45:05 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Thursday, 24 May 2012
New Development Snapshot

OpenJDK 7u4 has been integrated. Next stop RC.

Changes:

  • Integrated OpenJDK 7u4 b22.
  • Fixed threading bug in sun.font.PhysicalStrike (bug #3524226).
  • Fixed regression introduced in 7.0 that caused the manifest to be ignored when creating the java.lang.Package objects.
  • Updated boot class package property that still referred to Sun Microsystems instead of Oracle Corporation.

Binaries available here: ikvmbin-7.1.4527.zip

OpenJDK 7u4 b22 stripped sources: openjdk-7u4-stripped.zip

Thursday, 24 May 2012 11:04:15 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Friday, 11 May 2012
Using MethodRental.SwapMethodBody to do Method Level JIT Compilation

IKVM.NET has always had a class granularity JIT. Whenever a type is first "used" the CLR fires the AppDomain.TypeResolve event and at that point the IKVM.NET runtime compiles the Java bytecode to CIL for all of the methods in the class.

I don't recall my exact thought process, but I assume that when I started on IKVM.NET I looked at MethodRental.SwapMethodBody and was scared away by the lack of documentation and the fact that it requires full trust and manually constructing a method body blob.

Later on, I focussed more on static compilation and didn't care too deeply about dynamic performance. So I never revisited this decision. However, recently I have been thinking about dynamic performance, triggered by potential invokedynamic optimizations.

To get reacquainted with MethodRental.SwapMethodBody I wrote a small program that dynamically creates the following class:

class Frob {
public Frob(int i) { Console.WriteLine(i); } public static void M(int i) { Console.WriteLine(); Console.WriteLine(new Frob(i)); } }

The code is available here: MethodRentalDemo.cs

When the constructor and the M method are first created, the method body is defined, using MethodBuilder.CreateMethodBody, as a simple trampoline that calls Program.JIT to just-in-time generate the actual CIL for the method.

Here's the managed JIT trampoline CIL code for Frob.M:

ldtoken    method void Frob::M(int32))
call       void Program::JIT(valuetype System.RuntimeMethodHandle)
jmp        void Frob::M(int32)

The jmp instruction is interesting, it transfers control to a method that takes the same arguments as the current method and passes the current argument values. Here it is used to jump to a new version the same method, where the method body has been replaced with the actual CIL code.

The native code that is generated for the trampoline is:

  x86 x64
 
push   ebp 
mov    ebp,esp 
sub    esp,8 
xor    eax,eax 
mov    dword ptr [ebp-8],eax 
mov    dword ptr [ebp-4],ecx 
lea    ecx,[ebp-8] 
mov    edx,6231A0h 
call   680065F0 
lea    eax,[ebp-8] 
push   dword ptr [eax] 
call   FFE39B70 
mov    ecx,dword ptr [ebp-4] 
mov    esp,ebp 
pop    ebp 
jmp    dword ptr ds:[006231A8h] 
 
push   rbx 
sub    rsp,20h 
mov    ebx,ecx 
lea    rcx,[00257330h] 
call   FFFFFFFFF35036D0 
mov    rcx,rax 
call   00000000001C84C0 
mov    ecx,ebx 
lea    rax,[00247D80h] 
add    rsp,20h 
pop    rbx 
jmp    rax
add    rsp,20h 
pop    rbx 
ret

(Note that the x64 JIT generates three unreachable instructions at the end. Shown in gray.)

When this code is invoked, it loads the method handle and calls the Program.JIT method. After that returns, it invokes the new method body that the JIT method installed using MethodRental.SwapMethodBody.

When run on the CLR* this all appears to work as you'd hope, but unfortunately that's no guarantee that it will work in the real world. Googling (and experience) suggests that there aren't many** users of MethodRental.SwapMethodBody, so it is quite possible that there are some showstoppers lurking somewhere.

* It does not work on Mono at the moment.
** I found one reference to it in a paper on RubySharp from 2004.

Friday, 11 May 2012 12:24:32 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Friday, 04 May 2012
New Development Snapshot

This should hopefully be the last snapshot based on OpenJDK 7 FCS as 7u4 has shipped and the bundle should be available soon. After that I'll start work on integrating 7u4 and working towards the IKVM 7.1 release.

Changes:

  • Bug fix. When adding certificates to virtual cacerts file make sure that the aliases that are generated from the certificate subject are unique.
  • Bug fix. If a class file UTF-8 string ends with an incomplete multi byte char, we should throw the appropriate exception.
  • IKVM.Reflection: Added ModuleBuilder.__GetAssemblyToken() API.
  • IKVM.Reflection: Removed Mono CryptoConvert code and use .NET 2.0 API instead.
  • IKVM.Reflection: Rewrote StrongNameKeyPair to remove dependency on System.Reflection.StrongNameKeyPair.
  • IKVM.Reflection: Improved assembly title and description.
  • IKVM.Reflection: Metadata is now sorted with a stable sort algorithm.

Binaries available here: ikvmbin-7.1.4507.zip

Friday, 04 May 2012 08:58:49 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Monday, 23 April 2012
IKDASM - IL Disassembler Example for IKVM.Reflection

A while ago I wrote an IL disassembler to test IKVM.Reflection. Both as a correctness test and also to test if the API surface exposes enough of the underlying information.

I thought it would make a good IKVM.Reflection example (although the code won't win any awards, it's a bit of a hack). One nice feature is that it tries really hard to emit the same output file as the .NET ildasm to make comparison easier. There's even a command line option to match a specific ildasm version (2.0, 4.0 or 4.5) and its quirks.

The binaries are available in ikdasm-v0.1-binaries.zip and the Visual Studio 2010 solution in ikdasm-v0.1.zip.

Note that in its current form the ildasm compatibility mode only works on Windows, because it needs to P/Invoke _gcvt in msvcrt.dll to make sure the floating point numbers match the ildasm output.

Monday, 23 April 2012 16:54:40 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Thursday, 19 April 2012
New Development Snapshot

At Lang.NEXT I met someone who was interested in using IKVM.Reflection and after he started porting his code, he ran into some missing functionality in IKVM.Reflection, so I've made some improvements there.

I've also fixed the remaining known issues with access stubs and "unloadable" (missing types) custom modifiers.

Changes:

  • Fix for recently introduced bug (with access stub rewrite). Bug #3512589.
  • Changed build process to fall back to the NAnt task, if we can't find the resource compiler.
  • Made WinForms message loop thread creation lazy to hopefully allow more applications to run without message loop thread. This is a (partial) workaround for bug #3515033.
  • Changed ikvmc to read input files after processing all the options (to make -nowarn: and -warnaserror: options that follow the file names work for warnings produced during input file reading).
  • Added support for type 2 access stubs for constructors.
  • Bug fix. When an unloadable type is used in a method signature that overrides a method (or implements an interface method), the custom modifier must be the same as the base class or an override stub must be generated.
  • Added partial implementation of ThreadMXBean.
  • IKVM.Reflection: Bug fix. When writing an assembly that has a .netmodule, the TypeDefId field in the ExportedType in the manifest module should contain a TypeDef token instead of an index.
  • IKVM.Reflection: Bug fix. When exporting a nested type (via AssemblyBuilder.__AddTypeForwarder()), we should also set the namespace (in practice it is unlikely for a nested type to have a namespace, but is is possible).
  • IKVM.Reflection: Corrected a couple of method parameter names in Assembly.
  • IKVM.Reflection: Added Assembly.GetType(string,bool,bool) method.
  • IKVM.Reflection: Added support for case-insensitive type and member lookup.
  • IKVM.Reflection: Implemented case insensitive lookup in Type.GetInterface().
  • IKVM.Reflection: Moved GetEvents(), GetFields(), GetConstructors(), GetNestedTypes() and GetProperties() to a common implementation that fixes a number of bugs.
  • IKVM.Reflection: Fixed GetMethods() to properly filter out base class methods that have been overridden.
  • IKVM.Reflection: Moved member lookup by name to a common implementation that fixes a number of bugs and adds IgnoreCase support.
  • IKVM.Reflection: Added MemberInfo.ReflectedType.
  • IKVM.Reflection: Added Binder support for method and property lookup.
  • IKVM.Reflection: Bug fix. ParameterBuilder.Position should return the 1-based position passed in to DefineParameter, not the 0-based ParameterInfo.Position.
  • IKVM.Reflection: Changed Type.__ContainsMissingType to return true for generic type parameters that have constraints that return true for __ContainsMissingType.
  • IKVM.Reflection: Bug fix. It should be possible to import a function pointer type into a ModuleBuilder.
  • IKVM.Reflection: Implemented Assembly.ToString().
  • IKVM.Reflection: Added [Flags] attribute to ResourceLocation enum.
  • IKVM.Reflection: Added support for reading/querying manifest resources that are forwarded to another assembly.
  • IKVM.Reflection: Bug fix. Module.GetManifestResourceStream() should return null (instead of throwing FileNotFoundException) for non-existing resource names.

Binaries available here: ikvmbin-7.1.4491.zip

Thursday, 19 April 2012 09:26:43 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Thursday, 12 April 2012
MS12-025

This patch Tuesday Microsoft released MS12-025 that fixes approximately a zillion vulnerabilities in System.Drawing.dll.

Here's what they fixed (multiple instances of each issue):

  • Added security demands to unsafe methods.
  • Wrap handles in SafeHandle instead of using IntPtr.
  • Use checked to guard against integer overflow when calculating how many bytes to AllocHGlobal.

It's a little embarassing to have so many vulnerabilities in this old code. Luckily, silently running .NET code in the browser is becoming a thing of the past.

Thursday, 12 April 2012 12:33:30 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Friday, 06 April 2012
Lang.NEXT

I'm back home after a really great Lang.NEXT conference.

The video of my talk is available on Channel 9. The slides are here.

Friday, 06 April 2012 22:24:37 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]