# Tuesday, 17 March 2009
New Development Snapshot

One major change: I rewrote exception handling to store exception state in java.lang.Throwable (for non-.NET exceptions) instead of using a weak map. I never got around to doing that earlier, because the overhead of exceptions on the CLR is such that the weak map didn't contribute significantly, but on Mono the situation was very different, as it turns out the weap map was very slow there and exception handling is much faster. Anyway, the result is that on Mono a particular microbenchmark went from 40 seconds to less than a second. On the CLR the performance is also a little bit better, but still horrible.

If you use exceptions for control flow, make sure to override fillInStackTrace (simply "return this") as this saves a lot of time in collecting the stack trace. On previous versions of IKVM this sometimes didn't help, but now it always does.

Changes:

  • Rewrote exception handling.
  • Moved PDB generating code (used by IKVM.Reflection.Emit.dll) into a separate assembly (IKVM.PdbWriter.dll) to work around missing ISymWrapper.dll on Mono.
  • Made java.lang.Object and java.lang.StackTraceElement serializable (in the .NET sense).
  • Added workaround for Mono bug (Type.IsGenericTypeDefinition throws for non-MonoType types).
  • Fixed ikvm.extensions.ExtensionMethods.printStackTrace() to unmap the exception after printing the stack trace.
  • Volker implemented a couple more AWT graphics functions.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3363.zip
 

Tuesday, 17 March 2009 07:10:27 (W. Europe Standard Time, UTC+01:00)  #    Comments [3]
# Thursday, 12 March 2009
New Development Snapshot

Various fixes and improvements.

Changes:

  • Made Mono compilation workarounds conditional on __MonoCS__.
  • Fixed Class.desiredAssertionStatus() instrinsic to use the RemoveAsserts flag from the right class loader.
  • Fixed several generics related bugs in IKVM.Reflection.Emit.
  • Implemented ikvm.internal.ClassLiteral<T> to allow for more efficient class literals.
  • Switched java.awt.print package from GNU Classpath to OpenJDK.
  • Switched java.awt.geom package from GNU Classpath to OpenJDK.
  • Switched java.awt.color package from GNU Classpath to OpenJDK.
  • Switched java.awt.event package from GNU Classpath to OpenJDK.
  • Switched java.awt.datatransfer package from GNU Classpath to OpenJDK.
  • Switched java.awt.dnd package from GNU Classpath to OpenJDK.
  • Added hack to skip running the static initializer when computing the serialVersionUID while running ikvmstub.
  • Added support for defining delegates in Java.
  • Fixed regression introduced in 0.38 that caused LinkageError to be thrown instead of ClassCircularyError.
  • Updated mscorlib.jar (to get the new default constructor in MulticastDelegate).
  • Added utility class ikvm.runtime.Delegates to make it easier to create Runnables and PrivilegeActions from .NET languages.
  • Fixed JNI constructor invocation on dynamically loaded class.
  • Added hack to java.io.FileDescriptor to support JRuby's tty detection.
  • Implemented java.lang.ClassLoader$NativeLibrary.finalize(). Note that under normal circumstances this is never called, because IKVM doesn't support class unloading and by default finalization doesn't run on exit.
  • Fixed IKVM.Reflection.Emit bug that caused NullReferenceException in Save when using the ikvmc -target:module option.

The last IKVM.Reflection.Emit fix isn't in cvs yet, because SourceForge cvs is down.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3358.zip

Thursday, 12 March 2009 06:41:37 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Wednesday, 11 March 2009
Defining Delegates in Java

I finally implemented support for defining delegates in Java. The functionality will be available in the next snapshot.

Here's how to do it:

import cli.System.AsyncCallback;
import cli.System.IAsyncResult;
import cli.System.MulticastDelegate;

public final class RunnableDelegate extends MulticastDelegate
{
    public RunnableDelegate(Method m) { }
    public native void Invoke();
    public native IAsyncResult BeginInvoke(AsyncCallback callback, Object obj);
    public native void EndInvoke(IAsyncResult res);
    public interface Method
    {
        void Invoke();
    }
}

This is the recommended pattern. There is a little flexibility, but there are a number of strict rules, if you violate any of these rules, the class will fail to load (or compile with ikvmc) with a java.lang.VerifyError.

Delegate rules:

  • Class must extend cli.System.MulticastDelegate.
  • Class must be final.
  • Class must have a single method named Invoke that is public, native, non-final and non-static.
  • Class must have a single public constructor taking an inner interface named Method and have an empty method body (actually it must call the base class default constructor using the canonical bytecode sequence).
  • Class may have BeginInvoke/EndInvoke methods, but if it has either one it must have both. The BeginInvoke and EndInvoke methods must have a singature that is implied by the signature of Invoke.
  • Class must have a public inner interface named Method with a single method named Invoke with a signature identical to the Invoke method in the class.
  • Class may not have any fields.
  • Class may not have any other native methods.
  • Class must be static if it is an inner class. (This isn't a real rule, it follows from the other rules.)

The delegate is usable from Java like other .NET delegates, but obviously it doesn't add much value there. The real value comes from being able to better interact with other .NET languages. For example, one cool trick is this:

public final class RunnableDelegate
  
 extends MulticastDelegate
    implements Runnable
{
    public RunnableDelegate(Method m) { }
    public native void Invoke();
    public interface Method
    {
        void Invoke();
    }
    public void run()
    {
        Invoke();
    }
}

Now you have a delegate that also implements java.lang.Runnable. Unfortunately, C# doesn't understand that delegates can implement interfaces, so we need a helper method:

package ikvm.runtime;

public final class Delegates
{
    public static Runnable toRunnable(RunnableDelegate delegate)
    {
        return delegate;
    }
}

Now you can do this in C#:

java.lang.Thread thread = new java.lang.Thread(
    ikvm.runtime.Delegates.toRunnable(delegate {
        Console.WriteLine("Hello World");
    }));
thread.start();

I've currently defined delegates for java.lang.Runnable and java.security.PrivilegedAction as inner classes of ikvm.runtime.Delegates. There are some other candidate interfaces in IKVM.OpenJDK.Core.dll, but I'll only add them if there is demand, so let me know.

Wednesday, 11 March 2009 06:44:02 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Tuesday, 10 March 2009
100% Pure Java

In the past weekend I tried running JRuby 1.2.0RC2's interactive command line jirb on IKVM.NET. It somehow reminded me of an experience I had a little over a decade ago getting an application certified for as 100% Pure Java.

At the time I didn't really see the point of the 100% Pure Java certification, but Sun convinced us (mostly by offering to pay the certification costs), so we went ahead and eventually got certified. As far as I remember we didn't have any major issues, mostly some hardcoded paths that included backslashes and some dependencies on a case insenstive file system (to make it easier to find these issues without having to buy Solaris, I actually patched the C runtime that came with the Windows JDK to make all file access case sensitive). The other problem was that in those days Swing was exceptionally unstable on Solaris, but they eventually conceded that this wasn't our fault ;-)

JRuby

As some may remember, I have been running JRuby tests for a while. This is an older version of JRuby (I don't even know which version) and not used interactively. In any case, running the most recent JRuby revealed a number of issues:

  1. IKVM.NET bug in JNI that caused constructing an object of a dynamically loaded class via JNI to fail.
  2. IKVM.NET regression introduced a couple of days ago.
  3. An infinite recursion bug in the CLR x64 JIT.
  4. JRuby Issue.
  5. JNA Issue (JNA is a library used by JRuby that provides P/Invoke like functionality).

JRuby Issue

The first issue was that when I started jirb, it would not come up with the irb(main):001:0> prompt but simply read lines, echo them back and execute them. After debugging this I found that JRuby uses reflection to access the private handle field in java.io.FileDescriptor (when on Windows). IKVM's FileDescriptor didn't have a handle field, because it uses .NET's System.IO.Stream instead of the native Win32 API.

I solved this by adding a handle property to FileDescriptor (not the actual code):

@ikvm.lang.Property(get = "get_handle")
private long handle;

private long get_handle() {
  if (stream instanceof cli.System.IO.FileStream) {
    cli.System.IO.FileStream fs = (cli.System.IO.FileStream)stream;
    return fs.get_Handle().ToInt64();
  } else if (this == in) {
    return GetStdHandle(-10).ToInt64();
  } else if (this == out) {
    return GetStdHandle(-11).ToInt64();
  } else if (this == err) {
    return GetStdHandle(-12).ToInt64();
  }
  return -1;
}

@DllImportAttribute.Annotation("kernel32")
private static native cli.System.IntPtr GetStdHandle(int nStdHandle);

Now when JRuby uses reflection to get the handle field, it ends up calling the get_handle method and it will get the right handle back. The nice thing about implementing this as a property it that it means the cost is virtually zero when you're not accessing it and it also delays the full trust requiring native method invocation until it is really needed, which means that partial trust code won't be affected.

This solved the first problem, the right prompt now appeared.

JNA Issue

The next problem appeared after I ran jruby.jar through ikvmc and ran the resulting jruby.exe. After exiting I would get an exception:

irb(main):001:0> exit
java.io.IOException: Cannot run program "C:\.virtual-ikvm-home/bin/java": The system cannot find the file specified
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:474)
    at java.lang.Runtime.exec(Runtime.java:610)
    at java.lang.Runtime.exec(Runtime.java:483)
    at com.sun.jna.Native$DeleteNativeLibrary.run(Native.java:761)
    at java.lang.Thread.threadProc(Thread.java:2297)
    at java.lang.Thread$1.Invoke(Thread.java:797)
    at cli.System.Threading.ThreadHelper.ThreadStart_Context(Unknown Source)
    at cli.System.Threading.ExecutionContext.Run(Unknown Source)
    at cli.System.Threading.ThreadHelper.ThreadStart(Unknown Source)
Caused by: java.io.IOException: The system cannot find the file specified
    at java.lang.ProcessImpl.<init>(ProcessImpl.java:126)
    at java.lang.ProcessImpl.start(ProcessImpl.java:54)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:467)
    ... 8 more

This didn't show up when running via ikvm.exe, because I recently added support for this, but when running jruby.exe the ikvm.exe file wasn't available. I debugged this because I was curious why a new JVM was being started at exit. It turns out that JNA extracts a JNI DLL into the temp directory and wants to delete that at exit. While the DLL is loaded by the JVM, it cannot be deleted so in a shutdown hook it fires up a new JVM to delete the temporary file. This is already pretty gross (and not 100% Pure Java :-)), but not the actual issue. Digging deeper I found that this is the alternative code path, because they first attempt to unload the DLL by explicitly calling java.lang.ClassLoader$NativeLibrary.finalize() via reflection on the NativeLibrary object that corresponds to their DLL. This is, of course, exceptionally evil. Nevertheless, I have now implemented support for this in IKVM.

Conclusion

Maybe 100% Pure Java isn't as useless as I thought a decade ago :-)

Tuesday, 10 March 2009 06:53:42 (W. Europe Standard Time, UTC+01:00)  #    Comments [5]
# Monday, 02 March 2009
New Development Snapshot

In keeping with the recent coincidental trend of releasing a development snapshot every ten days, here's the next one. This time a got a little side tracked and decided to do some optimizations.

A cute one is that Object.getClass() is now an intrinsic in two specific scenarios. The first scenario is trivial, when it is used to do a null reference check (javac uses this pattern, both in its source and in the code it generates), the second scenario is a little more interesting:

boolean equals(Object other)
{
  if (other.getClass() != this.getClass())
    return false;
  ...
}

Is now compiled into:

boolean equals(Object other)
{
  if (Type.GetTypeHandle(other).Value != Type.GetTypeHandle(this).Value)
    return false;
  ...
}

This removes the need to create (and lookup) the System.Type and java.lang.Class objects entirely.

Changes:

  • Added wildcard exports for assemblies referenced by non-main assembly in shared class loader group.
  • Intrinsified two uses of Object.getClass().
  • Minor optimization to TypeWrapper.ClassObject.
  • Added optimization to reflective instantiation to use Activator.CreateInstance() when that is almost as fast as our LCG based implementation (which has a far higher initial cost).
  • Removed some unnecessary initializations from java.lang.Class.
  • Changed runtime calls to AccessController.doPrivileged() to explicitly pass callerID.
  • Added support for multi level stack tracking to CodeEmitter.
  • Made most Pop emitting lazy to enable optimizing them away (together with corresponding push).
  • Made loading class literal lazy, to enable optimizing them away when they aren't used (e.g. because an atomic intrinsic).
  • Made Class.desiredAssertionStatus() into an intrinsic, to be able to optimize it away when -removeassertions is used.
  • Avoid constructing AssemblyName objects when reading the export map (it turns out that constructing an AssemblyName is pretty expensive).
  • Made the export map reading lazy.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3348.zip
 

Monday, 02 March 2009 06:43:55 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Friday, 20 February 2009
New Development Snapshot

Yet more changes to support -sharedclassloader and while I was at FOSDEM I tried running SPECjvm2008 so I did the fixes necessary to get that to run. Note that if you want to run SPECjvm2008, you'll have to generate rt.jar (ikvmstub -shared ikvm.openjdk.core && ren IKVM.OpenJDK.Core.jar rt.jar) and create an ikvm.exe.config file to set the boot class path to rt.jar:

<?xml version="1.0"?>
<configuration>
  <appSettings>
    <add key="ikvm:sun.boot.class.path" value="\ikvm\lib\rt.jar" />
  </appSettings>
</configuration>

Changes:

  • Implemented JPEGImageWriter.getDefaultWriteParam().
  • Added dummy implementations for java.awt.Graphics.setComposite/getComposite/getFontRenderContext.
  • Added supported for redirecting Runtime.exec("java ...") to ikvm.exe.
  • Small performance improvement to stack walking in ObjectInputStream.
  • Removed several JIT performance workarounds that are no longer needed as of .NET 2.0 SP2 (aka .NET 3.5 SP1) .
  • Added hack to ikvmstub (-shared option) to generate stubs for core class library.
  • Imported JMath (John F. Brophy's pure Java port of fdlibm) and use it for most Math functions instead of .NET Math API.
  • Made 32 bit floating point math more compatible by rounding to 32 bit after every operation. On x86 this takes a significant performance hit, but without it the differences were too large for the SPECjvm2008 sunflow benchmark to pass.
  • Set foreground/background colors for java.awt.Graphics created from Image correctly.
  • Added support for setting the JPEG compression level and for custom quantization (but not huffman) tables.
  • Moved Y correction to java.awt.Graphics2D float overload of drawString, so that it too positions the text (approx.) correctly.
  • Don't overwrite the thread context class loader if it has already been set when sun.misc.Launcher initializes. Thanks to Dawid Weiss for finding this.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3338.zip
 

Friday, 20 February 2009 06:04:20 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Tuesday, 10 February 2009
New Development Snapshot

More changes to support -sharedassemblyclassloader, a couple of fixes and Volker's JDBC-ODBC bridge implementation.

Changes:

  • Moved JDBC code into IKVM.OpenJDK.Jdbc.dll.
  • Added JDBC-ODBC-Bridge (based on System.Data.Odbc) to IKVM.OpenJDK.Jdbc.dll.
  • Fixed IKVM.Reflection.Emit's AssemblyBuilder.AssemblyName to always include Version, Culture and PublicKeyToken.
  • Fixed NullReferenceException in IKVM.Reflection.Emit's SignatureHelper.WriteType.
  • Implemented TypeWrapper.GetEnclosingMethod for ReflectionOnly assemblies (this allows ikvmstub to work on ikvmc generated assemblies loaded by specifying a path).
  • Fixed pointer type check for method return types.
  • When ikvmc loads a referenced assembly that is the main assembly of a sharedclassloader group, also pre-load the other assemblies in the group.
  • More fixes to deal with the fact that the AssemblyClassLoader <-> Assembly mapping is no longer a one to one relation.
  • Changed AssemblyClassLoader to cache assembly load failures (because the CLR binder also caches failures).
  • Disable String.toCharArray() intrinsic in dynamic mode, because it relies on defining global variables which aren't available in dynamic mode.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3328.zip

Tuesday, 10 February 2009 08:55:25 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Thursday, 05 February 2009
IKVM 0.38 Update 1 RC 1

A fairly serious (because of its potentially elusive nature) bug was found in 0.38 and I forgot to update the version number of IKVM.OpenJDK.ClassLibrary.dll in the previous release candidate, so here is a new release candidate.

Changes:

  • Changed version to 0.38.0.4.
  • Fixed bug in PassiveWeakDictionary that caused ghost arrays to lose their type after a while. Thanks to Dawid Weiss for reporting this issue.

Binaries available here: ikvmbin-0.38.0.4.zip
Sources (+ binaries): Sources: ikvm-0.38.0.4.zip, classpath-0.95-stripped.zip, openjdk6-b12-stripped.zip
 

Thursday, 05 February 2009 15:51:03 (W. Europe Standard Time, UTC+01:00)  #    Comments [2]
# Monday, 02 February 2009
IKVM 0.38 Update 1 RC 0

Since there isn't yet a schedule for when 0.40 will be available, I decided to release an update of 0.38 that includes recent bug fixes.

Changes:

  • Changed version to 0.38.0.3.
  • Fixed the stack trace when an unwrapped java.lang.Error (or subclass) escapes from a static initializer.
  • It turns out that we really should create an MBeanServer in sun.management.ManagementFactory.createPlatformMBeanServer(), even if we don't populate it with anything useful, applications might still want to register their own MBeans. This fix allows Derby 10.4.2.0 to work.
  • Added helpful message to ClassCastException generated for ghost array casts.
  • Added check for constructor with missing body in map.xml.
  • Removed over eager state checking from java.util.zip.Deflater. Fixes Lucene issue.
  • Fixed enclosing method discovery to work for ReflectionOnly assemblies. This allows ikvmstub to work with ikvmc generated assemblies.
  • Always emit an explicit method override if we've mangled the name/sig, because we can't predict whether it will be needed or not (without keeping track of the mangling in the base classes) and the cost is minimal since this doesn't happen all that often.
  • Miranda method should use mangled name (if the name is mangled).
  • Fixed pointer detection to work for types with multiple indirection levels.
  • If the last call site of a subroutine wasn't reachable, the return switch would fall through potentially causing the code to be unverifiable.
  • The check for unloadable types on the stack indexed the stack in the wrong order.
  • Fixed exception wrapping for java.security.AccessController.doPrivileged().
  • Fixed tracer to only add a trace listener in executables.

Binaries available here: ikvmbin-0.38.0.3.zip
Sources (+ binaries): Sources: ikvm-0.38.0.3.zip, classpath-0.95-stripped.zip, openjdk6-b12-stripped.zip
 

Monday, 02 February 2009 07:01:55 (W. Europe Standard Time, UTC+01:00)  #    Comments [4]
# Wednesday, 21 January 2009
New Development Snapshot

Volker Berlin continues working on integrating more OpenJDK awt/image related code. He also started on a fully managed JDBC / ODBC bridge implementation (using System.Data.Odbc). I did some bug fixes and few small improvements here and there.

Changes:

  • Fixed the stack trace when an unwrapped java.lang.Error (or subclass) escapes from a static initializer.
  • It turns out that we really should create an MBeanServer in sun.management.ManagementFactory.createPlatformMBeanServer(), even if we don't populate it with anything useful, applications might still want to register their own MBeans. This fix allows Derby 10.4.2.0 to work.
  • Various java.awt.image.BufferedImage improvements.
  • Use CallerID instead of stack walking in java.util.ResourceBundle.
  • Moved java.security.AccessController.doPrivileged() implementation to Java and use CallerID to avoid stack walk.
  • JPEG support added to ImageIO.
  • Switched to using OpenJDK ColorModel code.
  • Use CallerID instead of stack walking in java.sql.DriverManager.
  • Added helpful message to ClassCastException generated for ghost array casts.
  • Convert a Java filename to a .NET filename in NetToolkit.createImage(String).
  • Added workarounds to make IKVM.Reflection.Emit work on Mono (2.2 or higher required).
  • Made ISymWrapper.dll dependency conditional in build to make IKVM.Reflection.Emit compile on Mono.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3308.zip
 

Wednesday, 21 January 2009 06:52:54 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]