# Wednesday, 22 April 2009
IKVM 0.40 Release Candidate 1

Thanks to Nat Luengnaruemitchai the MDB symbol writer has been fixed and finished and a couple of bugs were fixed.

Changes:

  • Changed version to 0.40.0.1.
  • Updated copyright notices in OpenJDK assemblies.
  • Fixed bug in workaround for #486307 that could cause "duplicate MemberRef" warnings when running PEVerify on an ikvmc generated assembly.
  • Fixed bug #2777128.
  • Fixed bug #2777171.

Binaries available here: ikvmbin-0.40.0.1.zip

Sources: ikvm-0.40.0.1.zip, classpath-0.95-stripped.zip, openjdk6-b12-stripped-IKVM-0.40.zip

Wednesday, 22 April 2009 07:03:48 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Tuesday, 21 April 2009
New Development Snapshot

A couple of fixes and some more OpenJDK AWT/Swing merges.

Changes:

  • Fixed build regression introduced in previous snapshot caused by different build directory structure in OpenJDK 6 b16.
  • Handle Graphics2D.setPaint(null) correctly.
  • Integrated OpenJDK java/awt/image and java/awt/image/renderable packages.
  • Fixed duplicate MemberRefs generated by IKVM.Reflection.Emit caused by Mono workaround generic types not being canonicalized.
  • Integrated OpenJDK sun/swing and sun/awt packages.

Binaries available here: ikvmbin-0.41.3398.zip

Tuesday, 21 April 2009 08:03:55 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Thursday, 16 April 2009
New Development Snapshot

Even though 0.40 is still baking, I'm already working on the next version.

Changes:

  • Integrated OpenJDK 6 b16.
  • Changed decoded bytecode stream to use indexes in branches instead of PC offsets.
  • Fixed .MDB debug symbol generation (thanks to Nat Luengnaruemitchai).
  • Several AWT fixes contributed by Judit Vasko-Szedlar (Chemaxon).
  • Moved jsr/ret bytecode processing into a seperate (inlining) pass to reduce complexity in the verifier and compiler.
  • Fixed a bug in the compiler that caused exception handler to be dropped after an unreachable exception block was encountered.
  • Updated copyright notices.

Binaries available here: ikvmbin-0.41.3393.zip

OpenJDK 6 b16 sources + build artifacts: openjdk6-b16-stripped.zip

Thursday, 16 April 2009 12:26:38 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Monday, 06 April 2009
IKVM 0.40 Release Candidate 0

The eagerly anticipated (by some) first release candidate of 0.40 is ready!

Changes since previous snapshot:

  • Changed version to 0.40.0.0.
  • Added back RMI stubs that got left out in the module split.
  • Added cmm profiles and dummy color management implementation.
  • Fixed regression in ObjectInputStream.latestUserDefinedLoader().

Binaries available here: ikvmbin-0.40.0.0.zip

Sources: ikvm-0.40.0.0.zip, classpath-0.95-stripped.zip, openjdk6-b12-stripped-IKVM-0.40.zip

Monday, 06 April 2009 11:05:34 (W. Europe Daylight Time, UTC+02:00)  #    Comments [4]
# Friday, 27 March 2009
New Development Snapshot -- Help Needed

As 0.40 is getting nearer, I need some help. This snapshot is pretty close to what the first 0.40 release candidate is going to be. However, there is still something missing. As part of this release cycle I wrote a custom implementation of Reflection.Emit named IKVM.Reflection.Emit. This is the part that is responsible for writing managed PE binaries used by ikvmc. It is also responsible for writing the debugging files (.pdb on .NET and .mdb on Mono). I've written a working .pdb writer, but only a basic (and untested) implementation of the .mdb writer and this where I need help. I need someone to test, debug, finish this. So if you care about being able to debug your Java code in MonoDevelop, please help out! To get an idea of what's involved here is the current source.

Changes:

  • Workaround for Mono bug 486307.
  • Added "Windows 7" detection for os.name system property.
  • Improved IKVM.Reflection.Emit symbol writer "plug in" interface.
  • Wrote a basic (untested) symbol writer for Mono.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3373.zip
 

Friday, 27 March 2009 06:25:54 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Tuesday, 17 March 2009
New Development Snapshot

One major change: I rewrote exception handling to store exception state in java.lang.Throwable (for non-.NET exceptions) instead of using a weak map. I never got around to doing that earlier, because the overhead of exceptions on the CLR is such that the weak map didn't contribute significantly, but on Mono the situation was very different, as it turns out the weap map was very slow there and exception handling is much faster. Anyway, the result is that on Mono a particular microbenchmark went from 40 seconds to less than a second. On the CLR the performance is also a little bit better, but still horrible.

If you use exceptions for control flow, make sure to override fillInStackTrace (simply "return this") as this saves a lot of time in collecting the stack trace. On previous versions of IKVM this sometimes didn't help, but now it always does.

Changes:

  • Rewrote exception handling.
  • Moved PDB generating code (used by IKVM.Reflection.Emit.dll) into a separate assembly (IKVM.PdbWriter.dll) to work around missing ISymWrapper.dll on Mono.
  • Made java.lang.Object and java.lang.StackTraceElement serializable (in the .NET sense).
  • Added workaround for Mono bug (Type.IsGenericTypeDefinition throws for non-MonoType types).
  • Fixed ikvm.extensions.ExtensionMethods.printStackTrace() to unmap the exception after printing the stack trace.
  • Volker implemented a couple more AWT graphics functions.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3363.zip
 

Tuesday, 17 March 2009 07:10:27 (W. Europe Standard Time, UTC+01:00)  #    Comments [3]
# Thursday, 12 March 2009
New Development Snapshot

Various fixes and improvements.

Changes:

  • Made Mono compilation workarounds conditional on __MonoCS__.
  • Fixed Class.desiredAssertionStatus() instrinsic to use the RemoveAsserts flag from the right class loader.
  • Fixed several generics related bugs in IKVM.Reflection.Emit.
  • Implemented ikvm.internal.ClassLiteral<T> to allow for more efficient class literals.
  • Switched java.awt.print package from GNU Classpath to OpenJDK.
  • Switched java.awt.geom package from GNU Classpath to OpenJDK.
  • Switched java.awt.color package from GNU Classpath to OpenJDK.
  • Switched java.awt.event package from GNU Classpath to OpenJDK.
  • Switched java.awt.datatransfer package from GNU Classpath to OpenJDK.
  • Switched java.awt.dnd package from GNU Classpath to OpenJDK.
  • Added hack to skip running the static initializer when computing the serialVersionUID while running ikvmstub.
  • Added support for defining delegates in Java.
  • Fixed regression introduced in 0.38 that caused LinkageError to be thrown instead of ClassCircularyError.
  • Updated mscorlib.jar (to get the new default constructor in MulticastDelegate).
  • Added utility class ikvm.runtime.Delegates to make it easier to create Runnables and PrivilegeActions from .NET languages.
  • Fixed JNI constructor invocation on dynamically loaded class.
  • Added hack to java.io.FileDescriptor to support JRuby's tty detection.
  • Implemented java.lang.ClassLoader$NativeLibrary.finalize(). Note that under normal circumstances this is never called, because IKVM doesn't support class unloading and by default finalization doesn't run on exit.
  • Fixed IKVM.Reflection.Emit bug that caused NullReferenceException in Save when using the ikvmc -target:module option.

The last IKVM.Reflection.Emit fix isn't in cvs yet, because SourceForge cvs is down.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3358.zip

Thursday, 12 March 2009 06:41:37 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Wednesday, 11 March 2009
Defining Delegates in Java

I finally implemented support for defining delegates in Java. The functionality will be available in the next snapshot.

Here's how to do it:

import cli.System.AsyncCallback;
import cli.System.IAsyncResult;
import cli.System.MulticastDelegate;

public final class RunnableDelegate extends MulticastDelegate
{
    public RunnableDelegate(Method m) { }
    public native void Invoke();
    public native IAsyncResult BeginInvoke(AsyncCallback callback, Object obj);
    public native void EndInvoke(IAsyncResult res);
    public interface Method
    {
        void Invoke();
    }
}

This is the recommended pattern. There is a little flexibility, but there are a number of strict rules, if you violate any of these rules, the class will fail to load (or compile with ikvmc) with a java.lang.VerifyError.

Delegate rules:

  • Class must extend cli.System.MulticastDelegate.
  • Class must be final.
  • Class must have a single method named Invoke that is public, native, non-final and non-static.
  • Class must have a single public constructor taking an inner interface named Method and have an empty method body (actually it must call the base class default constructor using the canonical bytecode sequence).
  • Class may have BeginInvoke/EndInvoke methods, but if it has either one it must have both. The BeginInvoke and EndInvoke methods must have a singature that is implied by the signature of Invoke.
  • Class must have a public inner interface named Method with a single method named Invoke with a signature identical to the Invoke method in the class.
  • Class may not have any fields.
  • Class may not have any other native methods.
  • Class must be static if it is an inner class. (This isn't a real rule, it follows from the other rules.)

The delegate is usable from Java like other .NET delegates, but obviously it doesn't add much value there. The real value comes from being able to better interact with other .NET languages. For example, one cool trick is this:

public final class RunnableDelegate
  
 extends MulticastDelegate
    implements Runnable
{
    public RunnableDelegate(Method m) { }
    public native void Invoke();
    public interface Method
    {
        void Invoke();
    }
    public void run()
    {
        Invoke();
    }
}

Now you have a delegate that also implements java.lang.Runnable. Unfortunately, C# doesn't understand that delegates can implement interfaces, so we need a helper method:

package ikvm.runtime;

public final class Delegates
{
    public static Runnable toRunnable(RunnableDelegate delegate)
    {
        return delegate;
    }
}

Now you can do this in C#:

java.lang.Thread thread = new java.lang.Thread(
    ikvm.runtime.Delegates.toRunnable(delegate {
        Console.WriteLine("Hello World");
    }));
thread.start();

I've currently defined delegates for java.lang.Runnable and java.security.PrivilegedAction as inner classes of ikvm.runtime.Delegates. There are some other candidate interfaces in IKVM.OpenJDK.Core.dll, but I'll only add them if there is demand, so let me know.

Wednesday, 11 March 2009 06:44:02 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Tuesday, 10 March 2009
100% Pure Java

In the past weekend I tried running JRuby 1.2.0RC2's interactive command line jirb on IKVM.NET. It somehow reminded me of an experience I had a little over a decade ago getting an application certified for as 100% Pure Java.

At the time I didn't really see the point of the 100% Pure Java certification, but Sun convinced us (mostly by offering to pay the certification costs), so we went ahead and eventually got certified. As far as I remember we didn't have any major issues, mostly some hardcoded paths that included backslashes and some dependencies on a case insenstive file system (to make it easier to find these issues without having to buy Solaris, I actually patched the C runtime that came with the Windows JDK to make all file access case sensitive). The other problem was that in those days Swing was exceptionally unstable on Solaris, but they eventually conceded that this wasn't our fault ;-)

JRuby

As some may remember, I have been running JRuby tests for a while. This is an older version of JRuby (I don't even know which version) and not used interactively. In any case, running the most recent JRuby revealed a number of issues:

  1. IKVM.NET bug in JNI that caused constructing an object of a dynamically loaded class via JNI to fail.
  2. IKVM.NET regression introduced a couple of days ago.
  3. An infinite recursion bug in the CLR x64 JIT.
  4. JRuby Issue.
  5. JNA Issue (JNA is a library used by JRuby that provides P/Invoke like functionality).

JRuby Issue

The first issue was that when I started jirb, it would not come up with the irb(main):001:0> prompt but simply read lines, echo them back and execute them. After debugging this I found that JRuby uses reflection to access the private handle field in java.io.FileDescriptor (when on Windows). IKVM's FileDescriptor didn't have a handle field, because it uses .NET's System.IO.Stream instead of the native Win32 API.

I solved this by adding a handle property to FileDescriptor (not the actual code):

@ikvm.lang.Property(get = "get_handle")
private long handle;

private long get_handle() {
  if (stream instanceof cli.System.IO.FileStream) {
    cli.System.IO.FileStream fs = (cli.System.IO.FileStream)stream;
    return fs.get_Handle().ToInt64();
  } else if (this == in) {
    return GetStdHandle(-10).ToInt64();
  } else if (this == out) {
    return GetStdHandle(-11).ToInt64();
  } else if (this == err) {
    return GetStdHandle(-12).ToInt64();
  }
  return -1;
}

@DllImportAttribute.Annotation("kernel32")
private static native cli.System.IntPtr GetStdHandle(int nStdHandle);

Now when JRuby uses reflection to get the handle field, it ends up calling the get_handle method and it will get the right handle back. The nice thing about implementing this as a property it that it means the cost is virtually zero when you're not accessing it and it also delays the full trust requiring native method invocation until it is really needed, which means that partial trust code won't be affected.

This solved the first problem, the right prompt now appeared.

JNA Issue

The next problem appeared after I ran jruby.jar through ikvmc and ran the resulting jruby.exe. After exiting I would get an exception:

irb(main):001:0> exit
java.io.IOException: Cannot run program "C:\.virtual-ikvm-home/bin/java": The system cannot find the file specified
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:474)
    at java.lang.Runtime.exec(Runtime.java:610)
    at java.lang.Runtime.exec(Runtime.java:483)
    at com.sun.jna.Native$DeleteNativeLibrary.run(Native.java:761)
    at java.lang.Thread.threadProc(Thread.java:2297)
    at java.lang.Thread$1.Invoke(Thread.java:797)
    at cli.System.Threading.ThreadHelper.ThreadStart_Context(Unknown Source)
    at cli.System.Threading.ExecutionContext.Run(Unknown Source)
    at cli.System.Threading.ThreadHelper.ThreadStart(Unknown Source)
Caused by: java.io.IOException: The system cannot find the file specified
    at java.lang.ProcessImpl.<init>(ProcessImpl.java:126)
    at java.lang.ProcessImpl.start(ProcessImpl.java:54)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:467)
    ... 8 more

This didn't show up when running via ikvm.exe, because I recently added support for this, but when running jruby.exe the ikvm.exe file wasn't available. I debugged this because I was curious why a new JVM was being started at exit. It turns out that JNA extracts a JNI DLL into the temp directory and wants to delete that at exit. While the DLL is loaded by the JVM, it cannot be deleted so in a shutdown hook it fires up a new JVM to delete the temporary file. This is already pretty gross (and not 100% Pure Java :-)), but not the actual issue. Digging deeper I found that this is the alternative code path, because they first attempt to unload the DLL by explicitly calling java.lang.ClassLoader$NativeLibrary.finalize() via reflection on the NativeLibrary object that corresponds to their DLL. This is, of course, exceptionally evil. Nevertheless, I have now implemented support for this in IKVM.

Conclusion

Maybe 100% Pure Java isn't as useless as I thought a decade ago :-)

Tuesday, 10 March 2009 06:53:42 (W. Europe Standard Time, UTC+01:00)  #    Comments [5]
# Monday, 02 March 2009
New Development Snapshot

In keeping with the recent coincidental trend of releasing a development snapshot every ten days, here's the next one. This time a got a little side tracked and decided to do some optimizations.

A cute one is that Object.getClass() is now an intrinsic in two specific scenarios. The first scenario is trivial, when it is used to do a null reference check (javac uses this pattern, both in its source and in the code it generates), the second scenario is a little more interesting:

boolean equals(Object other)
{
  if (other.getClass() != this.getClass())
    return false;
  ...
}

Is now compiled into:

boolean equals(Object other)
{
  if (Type.GetTypeHandle(other).Value != Type.GetTypeHandle(this).Value)
    return false;
  ...
}

This removes the need to create (and lookup) the System.Type and java.lang.Class objects entirely.

Changes:

  • Added wildcard exports for assemblies referenced by non-main assembly in shared class loader group.
  • Intrinsified two uses of Object.getClass().
  • Minor optimization to TypeWrapper.ClassObject.
  • Added optimization to reflective instantiation to use Activator.CreateInstance() when that is almost as fast as our LCG based implementation (which has a far higher initial cost).
  • Removed some unnecessary initializations from java.lang.Class.
  • Changed runtime calls to AccessController.doPrivileged() to explicitly pass callerID.
  • Added support for multi level stack tracking to CodeEmitter.
  • Made most Pop emitting lazy to enable optimizing them away (together with corresponding push).
  • Made loading class literal lazy, to enable optimizing them away when they aren't used (e.g. because an atomic intrinsic).
  • Made Class.desiredAssertionStatus() into an intrinsic, to be able to optimize it away when -removeassertions is used.
  • Avoid constructing AssemblyName objects when reading the export map (it turns out that constructing an AssemblyName is pretty expensive).
  • Made the export map reading lazy.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3348.zip
 

Monday, 02 March 2009 06:43:55 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]