# Monday, 02 March 2009
New Development Snapshot

In keeping with the recent coincidental trend of releasing a development snapshot every ten days, here's the next one. This time a got a little side tracked and decided to do some optimizations.

A cute one is that Object.getClass() is now an intrinsic in two specific scenarios. The first scenario is trivial, when it is used to do a null reference check (javac uses this pattern, both in its source and in the code it generates), the second scenario is a little more interesting:

boolean equals(Object other)
{
  if (other.getClass() != this.getClass())
    return false;
  ...
}

Is now compiled into:

boolean equals(Object other)
{
  if (Type.GetTypeHandle(other).Value != Type.GetTypeHandle(this).Value)
    return false;
  ...
}

This removes the need to create (and lookup) the System.Type and java.lang.Class objects entirely.

Changes:

  • Added wildcard exports for assemblies referenced by non-main assembly in shared class loader group.
  • Intrinsified two uses of Object.getClass().
  • Minor optimization to TypeWrapper.ClassObject.
  • Added optimization to reflective instantiation to use Activator.CreateInstance() when that is almost as fast as our LCG based implementation (which has a far higher initial cost).
  • Removed some unnecessary initializations from java.lang.Class.
  • Changed runtime calls to AccessController.doPrivileged() to explicitly pass callerID.
  • Added support for multi level stack tracking to CodeEmitter.
  • Made most Pop emitting lazy to enable optimizing them away (together with corresponding push).
  • Made loading class literal lazy, to enable optimizing them away when they aren't used (e.g. because an atomic intrinsic).
  • Made Class.desiredAssertionStatus() into an intrinsic, to be able to optimize it away when -removeassertions is used.
  • Avoid constructing AssemblyName objects when reading the export map (it turns out that constructing an AssemblyName is pretty expensive).
  • Made the export map reading lazy.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3348.zip
 

Monday, 02 March 2009 06:43:55 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Friday, 20 February 2009
New Development Snapshot

Yet more changes to support -sharedclassloader and while I was at FOSDEM I tried running SPECjvm2008 so I did the fixes necessary to get that to run. Note that if you want to run SPECjvm2008, you'll have to generate rt.jar (ikvmstub -shared ikvm.openjdk.core && ren IKVM.OpenJDK.Core.jar rt.jar) and create an ikvm.exe.config file to set the boot class path to rt.jar:

<?xml version="1.0"?>
<configuration>
  <appSettings>
    <add key="ikvm:sun.boot.class.path" value="\ikvm\lib\rt.jar" />
  </appSettings>
</configuration>

Changes:

  • Implemented JPEGImageWriter.getDefaultWriteParam().
  • Added dummy implementations for java.awt.Graphics.setComposite/getComposite/getFontRenderContext.
  • Added supported for redirecting Runtime.exec("java ...") to ikvm.exe.
  • Small performance improvement to stack walking in ObjectInputStream.
  • Removed several JIT performance workarounds that are no longer needed as of .NET 2.0 SP2 (aka .NET 3.5 SP1) .
  • Added hack to ikvmstub (-shared option) to generate stubs for core class library.
  • Imported JMath (John F. Brophy's pure Java port of fdlibm) and use it for most Math functions instead of .NET Math API.
  • Made 32 bit floating point math more compatible by rounding to 32 bit after every operation. On x86 this takes a significant performance hit, but without it the differences were too large for the SPECjvm2008 sunflow benchmark to pass.
  • Set foreground/background colors for java.awt.Graphics created from Image correctly.
  • Added support for setting the JPEG compression level and for custom quantization (but not huffman) tables.
  • Moved Y correction to java.awt.Graphics2D float overload of drawString, so that it too positions the text (approx.) correctly.
  • Don't overwrite the thread context class loader if it has already been set when sun.misc.Launcher initializes. Thanks to Dawid Weiss for finding this.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3338.zip
 

Friday, 20 February 2009 06:04:20 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Tuesday, 10 February 2009
New Development Snapshot

More changes to support -sharedassemblyclassloader, a couple of fixes and Volker's JDBC-ODBC bridge implementation.

Changes:

  • Moved JDBC code into IKVM.OpenJDK.Jdbc.dll.
  • Added JDBC-ODBC-Bridge (based on System.Data.Odbc) to IKVM.OpenJDK.Jdbc.dll.
  • Fixed IKVM.Reflection.Emit's AssemblyBuilder.AssemblyName to always include Version, Culture and PublicKeyToken.
  • Fixed NullReferenceException in IKVM.Reflection.Emit's SignatureHelper.WriteType.
  • Implemented TypeWrapper.GetEnclosingMethod for ReflectionOnly assemblies (this allows ikvmstub to work on ikvmc generated assemblies loaded by specifying a path).
  • Fixed pointer type check for method return types.
  • When ikvmc loads a referenced assembly that is the main assembly of a sharedclassloader group, also pre-load the other assemblies in the group.
  • More fixes to deal with the fact that the AssemblyClassLoader <-> Assembly mapping is no longer a one to one relation.
  • Changed AssemblyClassLoader to cache assembly load failures (because the CLR binder also caches failures).
  • Disable String.toCharArray() intrinsic in dynamic mode, because it relies on defining global variables which aren't available in dynamic mode.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3328.zip

Tuesday, 10 February 2009 08:55:25 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Thursday, 05 February 2009
IKVM 0.38 Update 1 RC 1

A fairly serious (because of its potentially elusive nature) bug was found in 0.38 and I forgot to update the version number of IKVM.OpenJDK.ClassLibrary.dll in the previous release candidate, so here is a new release candidate.

Changes:

  • Changed version to 0.38.0.4.
  • Fixed bug in PassiveWeakDictionary that caused ghost arrays to lose their type after a while. Thanks to Dawid Weiss for reporting this issue.

Binaries available here: ikvmbin-0.38.0.4.zip
Sources (+ binaries): Sources: ikvm-0.38.0.4.zip, classpath-0.95-stripped.zip, openjdk6-b12-stripped.zip
 

Thursday, 05 February 2009 15:51:03 (W. Europe Standard Time, UTC+01:00)  #    Comments [2]
# Monday, 02 February 2009
IKVM 0.38 Update 1 RC 0

Since there isn't yet a schedule for when 0.40 will be available, I decided to release an update of 0.38 that includes recent bug fixes.

Changes:

  • Changed version to 0.38.0.3.
  • Fixed the stack trace when an unwrapped java.lang.Error (or subclass) escapes from a static initializer.
  • It turns out that we really should create an MBeanServer in sun.management.ManagementFactory.createPlatformMBeanServer(), even if we don't populate it with anything useful, applications might still want to register their own MBeans. This fix allows Derby 10.4.2.0 to work.
  • Added helpful message to ClassCastException generated for ghost array casts.
  • Added check for constructor with missing body in map.xml.
  • Removed over eager state checking from java.util.zip.Deflater. Fixes Lucene issue.
  • Fixed enclosing method discovery to work for ReflectionOnly assemblies. This allows ikvmstub to work with ikvmc generated assemblies.
  • Always emit an explicit method override if we've mangled the name/sig, because we can't predict whether it will be needed or not (without keeping track of the mangling in the base classes) and the cost is minimal since this doesn't happen all that often.
  • Miranda method should use mangled name (if the name is mangled).
  • Fixed pointer detection to work for types with multiple indirection levels.
  • If the last call site of a subroutine wasn't reachable, the return switch would fall through potentially causing the code to be unverifiable.
  • The check for unloadable types on the stack indexed the stack in the wrong order.
  • Fixed exception wrapping for java.security.AccessController.doPrivileged().
  • Fixed tracer to only add a trace listener in executables.

Binaries available here: ikvmbin-0.38.0.3.zip
Sources (+ binaries): Sources: ikvm-0.38.0.3.zip, classpath-0.95-stripped.zip, openjdk6-b12-stripped.zip
 

Monday, 02 February 2009 07:01:55 (W. Europe Standard Time, UTC+01:00)  #    Comments [4]
# Wednesday, 21 January 2009
New Development Snapshot

Volker Berlin continues working on integrating more OpenJDK awt/image related code. He also started on a fully managed JDBC / ODBC bridge implementation (using System.Data.Odbc). I did some bug fixes and few small improvements here and there.

Changes:

  • Fixed the stack trace when an unwrapped java.lang.Error (or subclass) escapes from a static initializer.
  • It turns out that we really should create an MBeanServer in sun.management.ManagementFactory.createPlatformMBeanServer(), even if we don't populate it with anything useful, applications might still want to register their own MBeans. This fix allows Derby 10.4.2.0 to work.
  • Various java.awt.image.BufferedImage improvements.
  • Use CallerID instead of stack walking in java.util.ResourceBundle.
  • Moved java.security.AccessController.doPrivileged() implementation to Java and use CallerID to avoid stack walk.
  • JPEG support added to ImageIO.
  • Switched to using OpenJDK ColorModel code.
  • Use CallerID instead of stack walking in java.sql.DriverManager.
  • Added helpful message to ClassCastException generated for ghost array casts.
  • Convert a Java filename to a .NET filename in NetToolkit.createImage(String).
  • Added workarounds to make IKVM.Reflection.Emit work on Mono (2.2 or higher required).
  • Made ISymWrapper.dll dependency conditional in build to make IKVM.Reflection.Emit compile on Mono.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3308.zip
 

Wednesday, 21 January 2009 06:52:54 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Monday, 29 December 2008
IKVM 0.38 Released

I've released IKVM 0.38 to SourceForge. The binaries are identical to the ones in release candidate 2.

Release Notes

This document lists the known issues and incompatibilities.

Runtime

  • Code unloading (aka class GC) is not supported.
  • In Java static initializers can deadlock, on .NET some threads can see uninitialized state in cases where deadlock would occur on the JVM.
  • JNI
    • Only supported in the default AppDomain.
    • Only the JNICALL calling convention is supported! (On Windows, HotSpot appears to also support the cdecl calling convention).
    • Cannot call string contructors on already existing string instances
    • A few limitations in Invocation API support
      • The Invocation API is only supported when running on .NET.
      • JNI_CreateJavaVM: init options "-verbose[:class|:gc|:jni]", "vfprintf", "exit" and "abort" are not implemented. The JDK 1.1 version of JavaVMInitArgs isn't supported.
      • JNI_GetDefaultJavaVMInitArgs not implemented
      • JNI_GetCreatedJavaVMs only returns the JavaVM if the VM was started through JNI or a JNI call that retrieves the JavaVM has already occurred.
      • DestroyJVM is only partially implemented (it waits until there are no more non-daemon Java threads and then returns JNI_ERR).
      • DetachCurrentThread doesn't release monitors held by the thread.
    • Native libraries are never unloaded (because code unloading is not supported).
  • The JVM allows any reference type to be passed where an interface reference is expected (and to store any reference type in an interface reference type field), on IKVM this results in an IncompatibleClassChangeError.
  • monitorenter / monitorexit cannot be used on unitialized this reference.
  • Floating point is not fully spec compliant.
  • A method returning a boolean that returns an integer other than 0 or 1 behaves differently (this also applies to byte/char/short and for method parameters).
  • Synchronized blocks are not async exception safe.
  • Ghost arrays don't throw ArrayStoreException when you store an object that doesn't implement the ghost interface.
  • Class loading is more eager than on the reference VM.
  • Interface implementation methods are never really final (interface can be reimplemented by .NET subclasses).
  • JSR-133 finalization spec change is not fully implemented. The JSR-133 changes dictate that an object should not be finalized unless the Object constructor has run successfully, but this isn't implemented.
  • When a java.lang.Error (or subclass) is thrown in (and escapes) a static initializer, the stack trace might be (partially) lost.

Static Compiler (ikvmc)

  • Some subtle differences with ikvmc compiled code for public members inherited from non-public base classes (so called "access stubs"). Because the access stub lives in a derived class, when accessing a member in a base class, the derived cctor will be run whereas java (and ikvm) only runs the base cctor.
  • Try blocks around base class ctor invocation result in unverifiable code (no known compilers produce this type of code).
  • Try/catch blocks before base class ctor invocation result in unverifiable code (this actually happens with the Eclipse compiler when you pass a class literal to the base class ctor and compile with -target 1.4).
  • Only code compiled in a single assembly fully obeys the JLS binary compatibility rules.
  • An assembly can only contain one resource with a particular name.
  • Passing incorrect command line options to ikvmc may result in an exception rather than a proper error messages.
  • Under specific circumstances ikvmc may die with an exception if you're compiling code that references missing classes. As a workaround, supply the missing classes (may be stubs).
  • Under specific circumstances ikvmc may produce unverifiable code if you're compiling code that references missing classes. As a workaround, supply the missing classes (may be stubs).

Class Library

Most class library code is based on OpenJDK 6 build 12. Below is a list of divergences and IKVM specific implementation notes.

com.sun.security.auth.module        Not implemented.
java.applet GNU Classpath implementation. Not implemented.
java.awt GNU Classpath implementation with partial System.Windows.Forms based back-end. Not supported.
java.io.Console Not implemented.
java.lang.instrument Not implemented.
java.lang.management Not implemented.
java.net No IPv6 support implemented.
java.net.ProxySelector Getting the default system proxy for a URL is not implemented.
java.text.Bidi GNU Classpath implementation. Not supported.
java.util.zip Partially based on GNU Classpath implementation.
javax.imageio.plugins.jpeg Not implemented.
javax.management Not implemented.
javax.print Not implemented.
javax.script Not implemented.
javax.smartcardio Not implemented.
javax.sound Not implemented.
javax.swing GNU Classpath implementation. Not supported.
javax.tools Not implemented.
org.ietfs.jgss Not implemented.
sun.jdbc.odbc Not implemented.
sun.net.www.content.audio Audio content handlers not implemented.
sun.net.www.content.image Image content handlers not implemented.

The entire public API is available, so "Not implemented." for javax.print, for example, means that the API is there but there is no back-end to provide the actual printing support. "Not supported." means that the code is there and probably works at least somewhat, but that I'm less likely to fix bugs reported in these areas.

Specific API notes:

  • java.lang.Thread.stop(Throwable t) doesn't support throwing arbitrary exceptions on other threads (only java.lang.ThreadDeath).
  • java.lang.Thread.holdsLock(Object o) causes a spurious notify on the object (this is allowed by the J2SE 5.0 spec).
  • java.lang.String.intern() strings are never garbage collected.
  • Weak/soft references and reference queues are inefficient and do not fully implement the required semantics.
  • java.lang.ref.SoftReference: Soft references are not guaranteed to be cleared before an OutOfMemoryError is thrown.
  • Threads started outside of Java aren't "visible" (e.g. in ThreadGroup.enumerate()) until they first call Thread.currentThread().
  • java.lang.Thread.getState() returns WAITING or TIMED_WAITING instead of BLOCKING when we're inside Object.wait() and blocking to re-acquire the monitor.
  • java.nio.channel.FileChannel.lock() shared locks are only supported on Windows NT derived operating systems.
  • java.lang.SecurityManager: Deprecated methods not implemented: classDepth(String), inClass(String), classLoaderDepth(), currentLoadedClass(), currentClassLoader(), inClassLoader()

Supported Platforms

This release has been tested on the following CLI implementations / platforms:

CLI Implementation       Architecture      Operating System
.NET 2.0 SP2 x86 Windows
.NET 2.0 SP2 x64 Windows


Partial Trust

There is experimental support for running in partial trust.
 

Monday, 29 December 2008 08:46:52 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Wednesday, 24 December 2008
New Development Snapshot

I've checked in all the changes required to split the class library into ten different assemblies. So here is the first snapshot that contains the split binaries.

This means that the -sharedclassloader ikvmc option has been implemented, but it isn't ready for prime time yet, for now I've only focussed on getting the core class library to build with it.

Changes:

  • Split IKVM.OpenJDK.ClassLibrary.dll into ten parts.
  • Added -sharedclassloader option to ikvmc.
  • Removed some GNU Classpath build leftovers.
  • Removed workaround for com.sun.beans.ObjectHandler.classForName2() that hopefully isn't necessary any more.
  • Made ikvmc emit a warning whenever it emits code that throws a hard error.
  • Fixed ikvmc to detect access to members in another assembly that expose non-public types from that assembly (the CLR doesn't allow this) and generate java.lang.IllegalAccessError (plus warning during compilation) instead of producing invalid code.
  • Volker Berlin checked in his first set of changes to replace java.awt.image.BufferedImage with the OpenJDK version.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3280.zip

Wednesday, 24 December 2008 07:14:18 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Friday, 19 December 2008
Modularization Status Update

I've been doing lots of modularization work, but I'm not quite ready to check it in yet. I've now split the class library assembly into ten assemblies and I've got "shared class loader" support working (at least for the core class library scenario).

I've been positively surprised by how many scenarios can be supported while only loading IKVM.OpenJDK.Core.dll. As the graph below shows, it has more (necessarily circular) dependencies than I would like, but for a number of scenarios I've carefully tweaked the set of classes in Core to enable lazy loading the dependencies only when they are really needed.

I'm not yet ready to commit to supported "Core-only" scenarios, but here's a flavor of some things I've been able to do:

  • Running "Hello, World!" in dynamic mode
  • Serialization
  • Reflection
  • File I/O & Socket I/O (both classic and nio)

Packages included in Core:

  • java.io
  • java.lang
  • java.lang.annotation
  • java.lang.ref
  • java.lang.reflection
  • java.math
  • java.net
  • java.nio
  • java.nio.channels
  • java.nio.channels.spi
  • java.nio.charset
  • java.nio.charset.spi
  • java.security
  • java.security.cert
  • java.util
  • java.util.concurrent
  • java.util.concurrent.atomic
  • java.util.concurrent.locks
  • java.util.regex
  • javax.net

Here is the assembly dependency graph (click the image to enlarge):

Here are the current assembly file sizes:

IKVM.OpenJDK.Core.dll 3,278,848
IKVM.OpenJDK.Security.dll 2,646,016
IKVM.OpenJDK.Util.dll 1,111,040
IKVM.OpenJDK.Xml.dll 8,497,664
IKVM.OpenJDK.SwingAWT.dll 3,040,768
IKVM.OpenJDK.Charsets.dll 5,017,088
IKVM.OpenJDK.Corba.dll 2,335,232
IKVM.OpenJDK.Management.dll      1,180,160
IKVM.OpenJDK.Misc.dll 2,668,032
IKVM.OpenJDK.Text.dll 628,736
  30,403,584

It's interesting to see that the total is slightly less than the previous size of IKVM.OpenJDK.ClassLibrary.dll (30,472,704).

Friday, 19 December 2008 08:49:00 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Monday, 08 December 2008
Modularizing OpenJDK

Recently Mark Reinhold started blogging about modularizing the JDK. I've been getting requests to split up IKVM.OpenJDK.ClassLibrary.dll for a long time. It's good to see that the Java platform is also moving in that direction, but we needn't wait for that.

I've been working on making ikvmc support "multi target" mode for a while now. Last week I used this mode to compile the OpenJDK classes into 743 different assemblies (one per package) . I then tried to use NDepend to dig through the dependencies, but it turns out that 743 assemblies is a bit too much to handle (NDepend handled it resonably well, but the dependency graph was way too large to be useful). So I started moving some obvious stuff together. The result was this NDepend report. Still a lot of data and a lot of dependencies, but some patterns are starting to emerge.

Here's my preliminary view of how things could be split:

IKVM.OpenJDK.Core.dll 5 MB
IKVM.OpenJDK.Security.dll 3 MB
IKVM.OpenJDK.Util.dll 1 MB
IKVM.OpenJDK.Xml.dll 8 MB
IKVM.OpenJDK.SwingAWT.dll 3 MB
IKVM.OpenJDK.Charsets.dll 5 MB
IKVM.OpenJDK.Corba.dll 2 MB
IKVM.OpenJDK.Management.dll      2 MB
IKVM.OpenJDK.Misc.dll 3 MB

(The sizes are only approximate.)

I had originally hoped to make IKVM.OpenJDK.Core.dll smaller by keeping java.util, java.net and java.io out of it, but it looks like you won't be able to run anything non-trivial without requiring classes from these packages, so it makes more sense to put them into the core assembly. The IKVM.OpenJDK.Security.dll and  IKVM.OpenJDK.Util.dll assemblies contain other util and security related packages that shouldn't be needed as often.

It is possible to split packages across assemblies (e.g. java.awt.AWTPermission will be in IKVM.OpenJDK.Core.dll because java.lang.SecurityManager depends on it), but given the potential for confusion my current thinking is that it is probably best to only move individual classes into Core should it be necessary (because, realistically, you can't develop without having a reference to Core you're less likely to be confused when trying to locate the class).

To avoid confusion or expectations that are too high: I haven't yet built the runtime infrastructure to support this. So while I can compile the class library into all these parts, the runtime won't actually be able to work correctly, because it still expects all the boot class loader classes in a single assembly.

As always, feedback on the proposed split is very welcome.

NDepend

I really like NDepend's ability to show the dependencies in different ways and the interactive dependency matrix that allows you to drill down into a dependency to see exactly where it comes from. Beyond dependency analysis it also has a power SQL like query language to allows use to query dependencies and compute all the code metrics you can come up with. It also includes a number of code metrics out of the box, but those aren't really my cup of tea, so I can't comment how useful they are.

One other small but really nice thing about it is that you can run it without installing. IMO this is very nice compared with installers that do who knows what to your system (and require administrator access).

An evaluation copy can be downloaded from their website.

Full disclosure: I was given a free copy of NDepend Professional Edition.

Monday, 08 December 2008 06:51:28 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]