# Monday, 16 July 2012
IKVM.NET 7.1 Released

I've released IKVM.NET 7.1 to SourceForge. The binaries are identical to the ones in release candidate 2.

Release Notes

This document lists the improvements, known issues and incompatibilities.

What's New (relative to IKVM.NET 7.0):

  • Integrated OpenJDK 7 u4 b22.
  • Dropped experimental partial trust support.
  • Much improved ikvmc error and warning handling.
  • Added ikvmc options: -win32manifest, -filealign, -highentropyva.
  • Removed ikvmc -platform:Itanium option.
  • Added ikvm.lang.DllExport annotation to export static methods as unmanaged entry points.
  • Added ikvm.runtime.Util.getClassFromTypeHandle() overload for classes that represent arrays of remapped .NET types and .NET primitives.
  • Added extension methods for (almost) all instance methods in Object, String and Throwable to ikvm.extensions.ExtensionMethods.
  • Added support for delegates with ByRef parameters.
  • Added support to ikvmc to automatically set the full source path in the debugging info if the source file lives next to the .class file.
  • When adding certificates to virtual cacerts file make sure that the aliases that are generated from the certificate subject are unique.
  • Many (minor) bug fixes.
  • Many IKVM.Reflection fixes and improvements.

Runtime

  • Code unloading (aka class GC) is not supported.
  • In Java static initializers can deadlock, on .NET some threads can see uninitialized state in cases where deadlock would occur on the JVM.
  • JNI
     
    • Only supported in the default AppDomain.
    • Only the JNICALL calling convention is supported! (On Windows, HotSpot appears to also support the cdecl calling convention).
    • Cannot call string contructors on already existing string instances
    • A few limitations in Invocation API support
       
      • The Invocation API is only supported when running on .NET.
      • JNI_CreateJavaVM: init options "-verbose[:class|:gc|:jni]", "vfprintf", "exit" and "abort" are not implemented. The JDK 1.1 version of JavaVMInitArgs isn't supported.
      • JNI_GetDefaultJavaVMInitArgs not implemented
      • JNI_GetCreatedJavaVMs only returns the JavaVM if the VM was started through JNI or a JNI call that retrieves the JavaVM has already occurred.
      • DestroyJVM is only partially implemented (it waits until there are no more non-daemon Java threads and then returns JNI_ERR).
      • DetachCurrentThread doesn't release monitors held by the thread.
    • Native libraries are never unloaded (because code unloading is not supported).
  • The JVM allows any reference type to be passed where an interface reference is expected (and to store any reference type in an interface reference type field), on IKVM this results in an IncompatibleClassChangeError.
  • monitorenter / monitorexit cannot be used on unitialized this reference.
  • Floating point is not fully spec compliant.
  • A method returning a boolean that returns an integer other than 0 or 1 behaves differently (this also applies to byte/char/short and for method parameters).
  • Synchronized blocks are not async exception safe.
  • Ghost arrays don't throw ArrayStoreException when you store an object that doesn't implement the ghost interface.
  • Class loading is more eager than on the reference VM.
  • Interface implementation methods are never really final (interface can be reimplemented by .NET subclasses).
  • JSR-133 finalization spec change is not fully implemented. The JSR-133 changes dictate that an object should not be finalized unless the Object constructor has run successfully, but this isn't implemented.
  • Strict class-file checking is not implemented.
  • If a class with a finalizer and static initializer allocates instances of itself in the static initializer and the static initializer subsequently fails, the .NET runtime may abort the application when trying to finalize the objects.

Static Compiler (ikvmc)

  • Some subtle differences with ikvmc compiled code for public members inherited from non-public base classes (so called "access stubs"). Because the access stub lives in a derived class, when accessing a member in a base class, the derived cctor will be run whereas java (and ikvm) only runs the base cctor.
  • Try blocks around base class ctor invocation result in unverifiable code (no known compilers produce this type of code).
  • Try/catch blocks before base class ctor invocation result in unverifiable code (this actually happens with the Eclipse compiler when you pass a class literal to the base class ctor and compile with -target 1.4).
  • Only code compiled together during a single compilation fully obeys the JLS binary compatibility rules.

Class Library

Most class library code is based on OpenJDK 7u4 build 22. Below is a list of divergences and IKVM.NET specific implementation notes.

com.sun.security.auth.module        Not supported.
java.applet Not implemented.
java.awt Partial System.Windows.Forms based back-end. Not supported.
java.io.Console Not implemented.
java.lang.instrument Not implemented.
java.lang.management Limited implementation.
java.net SCTP and SDP not implemented.
java.net.ProxySelector Getting the default system proxy for a URL is not implemented.
java.nio.file Most optional features (e.g. ACLs) are not implemented.
java.text.Bidi Not supported.
java.util.zip Partially based on GNU Classpath implementation.
javax.crypto ECC is not implemented.
javax.imageio.plugins.jpeg Partial implementation. JPEGs can be read and written and there is limited metadata support.
javax.management Limited implementation.
javax.print There is a Win32 specific printing implementation. Not supported.
javax.script ECMAScript implementation is not included.
javax.smartcardio Not implemented.
javax.sound Not implemented.
javax.swing Not supported.
javax.tools Not supported.
org.ietfs.jgss Not implemented.
sun.jdbc.odbc Implementation based on .NET ODBC managed provider.
sun.net.www.content.audio Audio content handlers not implemented.
sun.net.www.content.image Not supported.

The entire public API is available, so "Not implemented." for javax.smartcardio, for example, means that the API is there but there is no back-end to provide the actual smartcard communication support. "Not supported." means that the code is there and probably works at least somewhat, but that I'm less likely to fix bugs reported in these areas, but patches are welcome, of course.

Specific API notes:

  • java.lang.Thread.stop(Throwable t) doesn't support throwing arbitrary exceptions on other threads (only java.lang.ThreadDeath).
  • java.lang.Thread.holdsLock(Object o) causes a spurious notify on the object (this is allowed by the J2SE 5.0 spec).
  • java.lang.String.intern() strings are never garbage collected.
  • Weak/soft references and reference queues are inefficient and do not fully implement the required semantics.
  • java.lang.ref.SoftReference: Soft references are not guaranteed to be cleared before an OutOfMemoryError is thrown.
  • Threads started outside of Java aren't "visible" (e.g. in ThreadGroup.enumerate()) until they first call Thread.currentThread().
  • java.lang.Thread.getState() returns WAITING or TIMED_WAITING instead of BLOCKING when we're inside Object.wait() and blocking to re-acquire the monitor.
  • java.nio.channel.FileChannel.lock() shared locks are only supported on Windows NT derived operating systems.
  • java.lang.SecurityManager: Deprecated methods not implemented: classDepth(String), inClass(String), classLoaderDepth(), currentLoadedClass(), currentClassLoader(), inClassLoader()
  • java.util.zip.ZipFile(String, CharSet) constructor added in Java 7 is missing.
  • The class org.omg.PortableInterceptor.UNKNOWN is included, but is not part of Java 7.

Supported Platforms

This release has been tested on the following CLI implementations / platforms:

CLI Implementation       Architecture      Operating System
.NET 2.0 SP2 x86 Windows 7
.NET 2.0 SP2 x64 Windows 7
.NET 4.0 x86 Windows 7
.NET 4.0 x64 Windows 7
Mono 2.10.5 x86 Windows 7
Mono 2.10.5 x64 Ubuntu 11.10


Partial Trust

Experimental support for running in partial trust has been withdrawn. Due to the deprecated nature of ASP.NET partial trust and the fact that supporting partial trust in .NET 4.0 will be impossible to combine with serialization interop, I've decided to stop supporting partial trust scenarios.

Monday, 16 July 2012 08:40:01 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Friday, 29 June 2012
IKVM.NET 7.1 Release Candidate 2

I found and fixed a code generation bug, so a new release candidate. The bug has been around since the very early days, but a change in 2010 made it more likely to surface. It showed up while debugging the Derby problem reported by Dash in the comments here.

Changes (relative to rc 1):

  • Updated version to 7.1.4532.2
  • Fixed a code analysis bug that under specific circumstances caused incorrect local variable types or null reference loads from local variables.

When the final release is done, it will include the full release notes.

Binaries available here: ikvmbin-7.1.4532.2.zip

Sources: ikvmsrc-7.1.4532.2.zip, openjdk-7u4-stripped.zip

Friday, 29 June 2012 11:21:38 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Tuesday, 26 June 2012
Experimental WinRT Support in IKVM.Reflection

I've added support for generating Windows Runtime assemblies with IKVM.Reflection. This is still experimental because it is mostly based on reverse engineering as there currently is virtually no documentation on this.

The code for generating a trivial component is available here. It doesn't require .NET 4.5 or any Windows Runtime metadata assemblies.

The IKVM.Reflection code is available in cvs.

BTW, I did this mostly because I was curious about what's involved. Don't read anything into this wrt IKVM.NET itself. It's unlikely to ever support running in WinRT.

Tuesday, 26 June 2012 18:05:11 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Monday, 25 June 2012
Why ASP.NET Medium Trust Isn't

On October 24 of last year I reported an ASP.NET Medium Trust vulnerability. This eventually resulted in KB 2698981 where Microsoft essentially deprecated ASP.NET Partial Trust.

The problem I reported was that it is possible to abuse Thread.Abort() to create an inconsistent TypedReference that violates type safety.

TypedReference is an interesting type and I've been on the lookout for a way to abuse it for a long time. It's purpose is to allow type safe references to be used in a generic way. To implement this a TypedReference contains both a pointer and a type and all operations it allows make sure that type safety isn't violated. It's a primitive type, so the runtime knows about it and treats it specially. It can be used from partially trusted code and because it can contain a reference to a location on the stack, the runtime enforces that TypedReference values can only be used from a single thread (by disallowing boxing or storing it in arrays or fields).

However, by having one thread repeatedly overwriting a TypedReference location on the stack with two different values and a second thread aborting the first thread at the right moment, you can end up with a TypedReference that combines the pointer from one value and the type from another value and thus violating type safety.

The source of the PoC is available here.

Monday, 25 June 2012 10:26:33 (W. Europe Daylight Time, UTC+02:00)  #    Comments [3]
# Wednesday, 20 June 2012
MS12-038 and IKDASM

Past patch Tuesday Microsoft released MS12-038 that updated System.Windows.Forms.dll. It fixes a vulnerability in clipboard handling.

To compare the unpatched and patched versions of the assembly, I added an option to ikdasm to supress some of the irrelevant differences. For example, it replaces all uses of the '<PrivateImplementationDetails>{nnnnnnnn-nnnn-nnnn-nnnn-nnnnnnnnnnnn}' class (that the C# compiler generates) where nnnnnnnn-nnnn-nnnn-nnnn-nnnnnnnnnnnn is the module version GUID with the literal string (i.e. it replaces the GUID digits with n-characters). Another change is that some metadata items are sorted by name. Specifically, property and event accessors and custom attributes tend to be ordered differently between different builds.

It's likely that I'll add more in the future.

The option is named -diffmode. Usage is straight forward:

    ikdasm -diffmode -out:System.Windows.Forms.il System.Windows.Forms.dll

Updated ikdasm sources are available here: ikdasm-v0.2.zip

Wednesday, 20 June 2012 15:07:38 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Tuesday, 19 June 2012
Ten Years of IKVM.NET

Ten years ago today I started blogging about IKVM.NET. It's been an amazing journey and when I started it I never would have guessed I'd still be on it ten years later.

Some of the highlights include making many new friends, meeting industry luminaries, speaking at conferences, working with the GNU Classpath community (and the fun FOSDEM weekends), working with the Mono community and, of course, the open sourcing of OpenJDK.

The end of the journey is not yet in sight as I still have enormous amounts of fun and both Java and .NET keep adding new features that keep things interesting.

Some statistics:

  • 540 blog entries
  • 739 blog comments
  • 22 releases
  • Over 200000 downloads
  • 3637 messages to the ikvm-developers mailing list
  • 4577 messages to the ikvm-commit mailing list
  • 10 security vulnerabilities reported
  • 9 CLR JIT bugs encountered.
  • 2 name changes (from I<<K.VM.NET to IK.VM.NET to IKVM.NET)
Tuesday, 19 June 2012 08:49:34 (W. Europe Daylight Time, UTC+02:00)  #    Comments [10]
# Wednesday, 13 June 2012
IKVM.NET 7.1 Release Candidate 1

The second release candidate is available. Unlike the previous rc, this one is buildable on Linux again (tested with Mono 2.10.5). It also fixes a regression in final field handling.

Changes (relative to rc 0):

  • Updated version to 7.1.4532.1
  • Fixed Linux build issue due to assembly.class filename case error in tools.rsp
  • Updated copyright years in LICENSE
  • Merged in OpenJDK 7u4 changes in THIRD_PARTY_README.
  • Bug fix. AssemblyClassLoader.InternalsVisibleToImpl() would crash with NRE if it got called on a single assembly class loader, because it should call GetLoader(Assembly) to get the AssemblyLoader instead of GetLoaderForExportedAssembly().
  • Bug fix. When resolving properties corresponding to fields with type 2 access stubs, unloadable types with the same name should compare as equal.
  • Bug fix. When a final field is wrapped in a property, any assemblies that are concurrently compiled with the declaring assembly will access the backing field directly and hence the declaring assembly will need an InternalsVisibleToAttribute to allow them access. This fix makes sure that this attribute is applied when the field is accessed from another (concurrently compiled) assembly.
  • IKVM.Reflection: Added workaround for Mono 2.10 bug in AssemblyName (public key token for ECMA public key is not created correctly).
  • IKVM.Reflection: Added workaround for Mono to StrongNameKeyPair.
  • IKVM.Reflection: Disallow key container constructor of StrongNameKeyPair when running on Mono on Windows.
  • IKVM.Reflection: Bug fix. Type.GetInterfaces() should work for unbaked types.

When the final release is done, it will include the full release notes.

Binaries available here: ikvmbin-7.1.4532.1.zip

Sources: ikvmsrc-7.1.4532.1.zip, openjdk-7u4-stripped.zip

Wednesday, 13 June 2012 08:35:13 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Wednesday, 30 May 2012
IKVM.NET 7.1 Release Candidate 0

The first release candidate is available. No changes (except the version number and strong naming) relative to the last development snapshot.

What's New (relative to IKVM.NET 7.0):

  • Integrated OpenJDK 7u4 b22.
  • Much improved ikvmc error and warning handling.
  • Added ikvmc options: -win32manifest, -filealign, -highentropyva.
  • Removed ikvmc -platform:Itanium option.
  • Added ikvm.lang.DllExport annotation to export static methods as unmanaged entry points.
  • Added ikvm.runtime.Util.getClassFromTypeHandle() overload for classes that represent arrays of remapped .NET types and .NET primitives.
  • Added extension methods for (almost) all instance methods in Object, String and Throwable to ikvm.extensions.ExtensionMethods.
  • Added support for delegates with ByRef parameters.
  • Added support to ikvmc to automatically set the full source path in the debugging info if the source file lives next to the .class file.
  • When adding certificates to virtual cacerts file make sure that the aliases that are generated from the certificate subject are unique.
  • Many (minor) bug fixes.
  • Many IKVM.Reflection fixes and improvements.

When the final release is done, it will include the full release notes.

Binaries available here: ikvmbin-7.1.4532.0.zip

Sources: ikvmsrc-7.1.4532.0.zip, openjdk-7u4-stripped.zip

Wednesday, 30 May 2012 09:45:05 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Thursday, 24 May 2012
New Development Snapshot

OpenJDK 7u4 has been integrated. Next stop RC.

Changes:

  • Integrated OpenJDK 7u4 b22.
  • Fixed threading bug in sun.font.PhysicalStrike (bug #3524226).
  • Fixed regression introduced in 7.0 that caused the manifest to be ignored when creating the java.lang.Package objects.
  • Updated boot class package property that still referred to Sun Microsystems instead of Oracle Corporation.

Binaries available here: ikvmbin-7.1.4527.zip

OpenJDK 7u4 b22 stripped sources: openjdk-7u4-stripped.zip

Thursday, 24 May 2012 11:04:15 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Friday, 11 May 2012
Using MethodRental.SwapMethodBody to do Method Level JIT Compilation

IKVM.NET has always had a class granularity JIT. Whenever a type is first "used" the CLR fires the AppDomain.TypeResolve event and at that point the IKVM.NET runtime compiles the Java bytecode to CIL for all of the methods in the class.

I don't recall my exact thought process, but I assume that when I started on IKVM.NET I looked at MethodRental.SwapMethodBody and was scared away by the lack of documentation and the fact that it requires full trust and manually constructing a method body blob.

Later on, I focussed more on static compilation and didn't care too deeply about dynamic performance. So I never revisited this decision. However, recently I have been thinking about dynamic performance, triggered by potential invokedynamic optimizations.

To get reacquainted with MethodRental.SwapMethodBody I wrote a small program that dynamically creates the following class:

class Frob {
public Frob(int i) { Console.WriteLine(i); } public static void M(int i) { Console.WriteLine(); Console.WriteLine(new Frob(i)); } }

The code is available here: MethodRentalDemo.cs

When the constructor and the M method are first created, the method body is defined, using MethodBuilder.CreateMethodBody, as a simple trampoline that calls Program.JIT to just-in-time generate the actual CIL for the method.

Here's the managed JIT trampoline CIL code for Frob.M:

ldtoken    method void Frob::M(int32))
call       void Program::JIT(valuetype System.RuntimeMethodHandle)
jmp        void Frob::M(int32)

The jmp instruction is interesting, it transfers control to a method that takes the same arguments as the current method and passes the current argument values. Here it is used to jump to a new version the same method, where the method body has been replaced with the actual CIL code.

The native code that is generated for the trampoline is:

  x86 x64
 
push   ebp 
mov    ebp,esp 
sub    esp,8 
xor    eax,eax 
mov    dword ptr [ebp-8],eax 
mov    dword ptr [ebp-4],ecx 
lea    ecx,[ebp-8] 
mov    edx,6231A0h 
call   680065F0 
lea    eax,[ebp-8] 
push   dword ptr [eax] 
call   FFE39B70 
mov    ecx,dword ptr [ebp-4] 
mov    esp,ebp 
pop    ebp 
jmp    dword ptr ds:[006231A8h] 
 
push   rbx 
sub    rsp,20h 
mov    ebx,ecx 
lea    rcx,[00257330h] 
call   FFFFFFFFF35036D0 
mov    rcx,rax 
call   00000000001C84C0 
mov    ecx,ebx 
lea    rax,[00247D80h] 
add    rsp,20h 
pop    rbx 
jmp    rax
add    rsp,20h 
pop    rbx 
ret

(Note that the x64 JIT generates three unreachable instructions at the end. Shown in gray.)

When this code is invoked, it loads the method handle and calls the Program.JIT method. After that returns, it invokes the new method body that the JIT method installed using MethodRental.SwapMethodBody.

When run on the CLR* this all appears to work as you'd hope, but unfortunately that's no guarantee that it will work in the real world. Googling (and experience) suggests that there aren't many** users of MethodRental.SwapMethodBody, so it is quite possible that there are some showstoppers lurking somewhere.

* It does not work on Mono at the moment.
** I found one reference to it in a paper on RubySharp from 2004.

Friday, 11 May 2012 12:24:32 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]