# Monday, 10 December 2007
IKVM 0.36 Released

After what feels like an eternity, I've finally released IKVM 0.36. The binaries are identical to the ones in rc6.

Release Notes

This document lists the known issues and incompatibilities.

Runtime

  • Code unloading (aka class GC) is not supported.
  • In Java static initializers can deadlock, on .NET some threads can see uninitialized state in cases where deadlock would occur on the JVM.
  • JNI
    • Only supported in the default AppDomain.
    • Only the JNICALL calling convention is supported! (On Windows, HotSpot appears to also support the cdecl calling convention).
    • Cannot call string contructors on already existing string instances
    • A few limitations in Invocation API support
      • The Invocation API is only supported when running on .NET.
      • JNI_CreateJavaVM: init options "-verbose[:class|:gc|:jni]", "vfprintf", "exit" and "abort" are not implemented. The JDK 1.1 version of JavaVMInitArgs isn't supported.
      • JNI_GetDefaultJavaVMInitArgs not implemented
      • JNI_GetCreatedJavaVMs only returns the JavaVM if the VM was started through JNI or a JNI call that retrieves the JavaVM has already occurred.
      • DestroyJVM is only partially implemented (it waits until there are no more non-daemon Java threads and then returns JNI_ERR).
      • DetachCurrentThread doesn't release monitors held by the thread.
    • Native libraries are never unloaded (because code unloading is not supported).
  • The JVM allows any reference type to be passed where an interface reference is expected (and to store any reference type in an interface reference type field), on IKVM this results in an IncompatibleClassChangeError.
  • monitorenter / monitorexit cannot be used on unitialized this reference.
  • Floating point is not fully spec compliant.
  • Volatile fields are not marked with modopt(volatile).
  • A method returning a boolean that returns an integer other than 0 or 1 behaves differently (this also applies to byte/char/short and for method parameters).
  • Synchronized blocks are not async exception safe.
  • Ghost arrays cannot be distinguished from object arrays (isinstance and casting don't differentiate and ArrayStoreException isn't thrown).
  • Class loading is more eager than on the reference VM.
  • Object instances don't get created at new, but only at the invokespecial of the constructor. This causes observable differences with classes that have a finalizer when the constructor arg evaluation throws an exception [with the JSR-133 change in finalization, this is no longer a spec issue, but since the reference VM doesn't correctly implement JSR-133 it still is an implementation difference].
  • Interface implementation methods are never really final (interface can be reimplemented by .NET subclasses).
  • JSR-133 finalization spec change is not implemented. The JSR-133 changes dictate that an object should not be finalized unless the Object constructor has run successfully.

Static Compiler (ikvmc)

  • Some subtle differences with ikvmc compiled code for public members inherited from non-public base classes (so called "access stubs"). Because the access stub lives in a derived class, when accessing a member in a base class, the derived cctor will be run whereas java (and ikvm) only runs the base cctor.
  • Try blocks around base class ctor invocation result in unverifiable code (no known compilers produce this type of code).
  • Try/catch blocks before base class ctor invocation result in unverifiable code (this actually happens with the Eclipse compiler when you pass a class literal to the base class ctor and compile with -target 1.4).
  • Only code compiled in a single assembly fully obeys the JLS binary compability rules.
  • An assembly can only contain one resource with a particular name.
  • Passing incorrect command line options to ikvmc may result in an exception rather than a proper error messages.

Class Library

Most class library code is based on OpenJDK build 13. Below is a list of divergences and IKVM specific implementation notes.

com.sun.security.auth.module        Not implemented.
java.applet GNU Classpath implementation. Not implemented.
java.awt GNU Classpath implementation with partial System.Windows.Forms based back-end. Not supported.
java.io.Console Not implemented.
java.lang.instrument Not implemented.
java.lang.management Not implemented.
java.net No IPv6 support implemented.
java.net.ProxySelector Getting the default system proxy for a URL is not implemented.
java.security Not supported.
java.text.Bidi GNU Classpath implementation. Not supported.
java.util.zip Partially based on GNU Classpath implementation.
javax.crypto IcedTea implementation. Not supported.
javax.imageio.plugins.jpeg Not implemented.
javax.management Not implemented.
javax.print Not implemented.
javax.script Not implemented.
javax.security Not supported.
javax.smartcardio Not implemented.
javax.sound Not implemented.
javax.swing GNU Classpath implementation. Not supported.
javax.tools Not implemented.
org.ietfs.jgss Not implemented.
sun.jdbc.odbc Not implemented.
sun.net.www.content.audio Audio content handlers not implemented.
sun.net.www.content.image Image content handlers not implemented.

The entire public API is available, so "Not implemented." for javax.print, for example, means that the API is there but there is no back-end to provide the actual printing support. "Not supported." means that the code is there and probably works at least somewhat, but that I'm less likely to fix bugs reported in these areas.

In addition to the issues listed above, the extended charset providers are not included (the ones in charsets.jar in the JRE).

Specific API notes:

  • java.lang.Thread.stop(Throwable t) doesn't support throwing arbitrary exceptions on other threads (only java.lang.ThreadDeath).
  • java.lang.Thread.holdsLock(Object o) causes a spurious notify on the object (this is allowed by the J2SE 5.0 spec).
  • java.lang.String.intern() strings are never garbage collected.
  • Weak/soft references and reference queues are inefficient and do not fully implement the required semantics.
  • Threads started outside of Java aren't "visible" (e.g. in ThreadGroup.enumerate()) until they first call Thread.currentThread().
  • java.lang.Thread.getState() returns WAITING or TIMED_WAITING instead of BLOCKING when we're inside Object.wait() and blocking to re-acquire the monitor.
  • java.io.File.getCanonicalPath() is broken on Mono on Windows.
  • java.nio.channel.FileChannel.lock() shared locks are only supported on Windows NT derived operating systems.
  • java.lang.ref.SoftReference: Soft references are not guaranteed to be cleared before an OutOfMemoryError is thrown.
  • java.lang.SecurityManager: Deprecated methods not implemented: classDepth(String), inClass(String), classLoaderDepth(), currentLoadedClass(), currentClassLoader(), inClassLoader()

Supported Platforms

This release has been tested on the following CLI implementations / platforms:

CLI Implementation       Architecture      Operating System
Mono 1.2.5 x86* Windows
Mono 1.2.5 x86* Linux
.NET 1.1 x86 Windows
.NET 2.0 x86 Windows
.NET 2.0 x64 Windows
.NET 2.0 SP1 x86 Windows
.NET 2.0 SP1 x64 Windows

* Running on Mono has only been tested on x86, but Mono includes the native (platform specific) parts of IKVM, so any platform where Mono runs should in theory be able to run this IKVM release.

Partial Trust

Partial trust is not supported. The IKVM class library and runtime require Full Trust to be able to run.
 

Monday, 10 December 2007 08:50:56 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Friday, 07 December 2007
Mono Summit 2007

Last week I attended the Mono Summit 2007 in Madrid. It was a very productive three days and very much fun as well! I got to meet Zoltan, Wade, Paolo, Massi and Dick for the first time and enjoyed seeing Jb (of Cecil and monolinker fame), Rodrigo (of Boo fame) and Miguel again.

Jb created a small repro of the monolinker bug that caused it too fail on IKVM.OpenJDK.ClassLibrary.dll, so hopefully that'll be fixed soon. I also talked to him about using Cecil in ikvmc to free me from the shackles of Reflection.Emit. This is a major chunk of work, but I think it will be worth it.

Zoltan found some weirdness in my usage of DynamicMethod (the DynamicMethod signature for the constructor reflection helper had an extra unused argument, the CLR didn't mind because it matched up with the unused this reference, but Mono didn't like it). Unfortunately, fixing this didn't resolve the problems I had with DynamicMethod on Mono 1.2.5 and 1.2.6, so fast reflection will remain disabled while running on Mono for the time being (on Zoltan's current code base it did work, so when the Mono release after 1.2.6 is out I'll be able to enable it).

Rodrigo did a very interesting and entertaining presentation on Boo. I must admit that I haven't seriously played with Boo yet, but it looks extremely interesting. It is a statically typed language that also supports more dynamic typing (like Erik Meijer's "Static Typing Where Possible, Dynamic Typing When Needed"). It also supports some very cool meta programming capabilities. I definitely need to check this out and I recommend you do too.

Paolo and I had a tentative discussion about adding Java bytecode support to the Mono JIT, so that the expensive IKVM just-in-time bytecode to IL conversion can be skipped when running Java code in dynamic mode on Mono. This would be really nice to have, but there aren't any specific plans yet, so don't hold your breath yet.

Finally, I tried to convice Jim Purbrick (of SecondLife, who was there because he wants to use Mono to speed up LSL and possibly support other languages) that he really should support Java as well ;-)

Friday, 07 December 2007 07:42:20 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Tuesday, 04 December 2007
IKVM 0.36 Release Candidate 6

This time I really expect this to be the final 0.36 release candidate :-)

The list of changes is relative to the 0.36 rc5, only the DynamicMethod constructor fix is new relative to the 0.37 development snapshot.

Changes:

  • Implemented Custom Assembly Class Loaders.
  • Fixed URL.equals() for "ikvmres:" urls.
  • Fixed findResources() bug that caused resources in core assembly to show up twice if a .NET assembly explicitly referenced the core assembly.
  • Added check to prevent defineClass() on an assembly class loader from defining a class that already exists in the assembly.
  • Fixed Class.getModifiers() to mask out ACC_SUPER bit.
  • Fixed a bug in the DynamicMethod based reflection implementation that caused a NullPointerException when generating the setter for a final instance field for code that isn't compiled with -strictfinalfieldsemantics.
  • Fix to make sure that a ghost interface method call always goes thru the target reference wrapping path. (Calling java.lang.CharSequence methods through an interface that extends java.lang.CharSequence would generate incorrect code.)
  • Fixed ghost interface array casting to generate verifiable code.
  • Fixed FileChannelImpl to close the FileDescriptor after releasing the file locks.
  • Fixed DynamicMethod based constructor reflection invocation to have method match the delegate signature.

Binaries available here:ikvmbin-0.36.0.5.zip.

Sources (+ binaries):ikvm-0.36.0.5.zip

External sources (haven't changed since rc1):classpath-0.95-stripped.zip,openjdk-b13-stripped.zip

Tuesday, 04 December 2007 08:55:12 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Monday, 19 November 2007
Splitting the Class Library Assembly

In a comment to the previous post Peter asks:

Do you plan to split the classpath assembly into multiple smaller assemblies at some point? The OpenJDK-based one is getting *huge* and when you only use a small subset of JDK (as I think most people who don't use IKVM as an alternative to JRE, but use it so that they can embed Java code in .NET app, do), it's pretty high overhead...

Cyprien Noel posted a feature request:

Would it be possible to split the dll with the jvm classes in 2 or more
parts ? E.g. a basic dll with java.lang, util, math, io, net, text, and
another dll with the other features. Thanks

Others have asked similar things. I'm very sympathetic to this request, because the bloat of the Java API has annoyed me ever since JDK 1.2.

Unfortunately this is a very hard problem without upstream support. Since Sun has never had any incentive to reduce dependencies inside the class library (because up until now it has always shipped as a single file), there is a pretty tight coupling between many packages in the Java class library. For example, just the static dependencies of java.lang.Object amount to 1893 classes in the IKVM 0.36 release codebase. Note that these are just the static dependencies and this set of classes won't even allow you to run "Hello World". However, adding a few more classes (to a total of 2676) gives a, what appears to be, workable subset and yields an assembly of approx. 5.5MB.

That looks like a great start, doesn't it? However, the problem is that it isn't at all clear what this assembly supports. For example, it supports the http protocol, but not the https protocol. Simply adding the https protocol handler drags in an additional 1895 classes!

Suppose I put the https support in a separate assembly and when you test your application you only test with http URLs so everything works with just the core assembly and you decide to ship only the core assembly. Now a user of your code tries an https URL and it doesn't work. This may seem like an obvious thing, but there are probably other scenarios that aren't as obvious.

The key point here is that I have no idea of most of the inner workings of the class library, so it would be difficult (or extremely time consuming) for me to figure out the best way to split the classes and to give any guidance as to when you'd need to ship what assembly.

Why I'm Probably Still Going To Do It

There is another argument for splitting the class library assembly: Performance. For example, depending on the scenario, having several smaller assemblies that are demand loaded may be more efficient than a single huge assembly. Especially because the assemblies are strong named and the CLR verifies the assembly hash when it loads the assembly (if the assembly isn't in the GAC).

When?

As is usually the answer: I don't know. It'll be done when it's done. In the meantime you can always download the sources and build your own subset class library assembly :-)

Monday, 19 November 2007 08:51:30 (W. Europe Standard Time, UTC+01:00)  #    Comments [4]
# Thursday, 15 November 2007
0.36 Release Plan

Ben asked about the 0.36 release yesterday. Here's the situation: There will be at least one more 0.36 release candidate that includes the new custom assembly class loader support and the fixes I've done since rc5. I'm currently waiting for some test results from the sponsor of the 0.36 release, to make sure that everything that needs to be fixed is fixed before release.

Thursday, 15 November 2007 08:12:29 (W. Europe Standard Time, UTC+01:00)  #    Comments [2]
# Tuesday, 13 November 2007
Development Snapshot Update

Three more fixes and a complete set of binaries this time.

Changes:

  • Fix to make sure that a ghost interface method call always goes thru the target reference wrapping path. (Calling java.lang.CharSequence methods through an interface that extends java.lang.CharSequence would generate incorrect code.)
  • Fixed ghost interface array casting to generate verifiable code.
  • Fixed FileChannelImpl to close the FileDescriptor after releasing the file locks.

Binaries available here: ikvmbin-0.37.2873.zip.

Update: Fixed the link.

Tuesday, 13 November 2007 08:19:28 (W. Europe Standard Time, UTC+01:00)  #    Comments [3]
# Friday, 02 November 2007
Development Snapshot Update

Two fixes. This time I'm only releasing an updated version of IKVM.Runtime.dll as that is all that changed.

Changes:

  • AppDomainAssemblyClassLoader now skips AssemblyBuilders. Previous version would cause problems if any dynamically generated assemblies were present in the AppDomain.
  • Fixed a bug in the DynamicMethod based reflection implementation that caused a NullPointerException when generating the setter for a final instance field for code that isn't compiled with -strictfinalfieldsemantics.

Update available here: ikvmbin-upd-0.37.2862.zip.

Friday, 02 November 2007 08:02:15 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Friday, 26 October 2007
Writing a Custom Assembly Class Loader

A new development snapshot. It's now possible to write your own custom assembly class loader. This is the recommended pattern:

class MyCustomClassLoader extends ClassLoader
{
  MyCustomClassLoader(cli.System.Reflection.Assembly asm)
  {
    super(new ikvm.runtime.AssemblyClassLoader(asm));
  }

  protected Class findClass(String name)
  {
    // TODO: forward call to other class loader or load class and call defineClass()
  }

  protected URL findResource(String name)
  {
    // TODO: forward call to other class loader or find resource somewhere
  }

  protected Enumeration findResources(String name)
  {
    // TODO: forward call to other class loader or find resources somewhere
  } 
}

If you delegate to other class loaders, you should be careful to avoid loops in the delegation path. If you want to change the resource loading delegation order, you can override getResource[s]() instead of findResource[s]() relatively safely, but if you want to change the delegation order for classes you need to be more careful, because you cannot replace any classes that are defined in the assembly.

Of course, you can also reuse existing class loader classes. Here's an example with URLClassLoader:

class MyCustomClassLoader extends java.net.URLClassLoader
{
  MyCustomClassLoader(cli.System.Reflection.Assembly asm)
  {
    super(new java.net.URL[0], new ikvm.runtime.AssemblyClassLoader(asm));
    // see below why calling addURL() is safer than passing it to the super constructor
    addURL(new java.net.URL("..."));
  }
}

Note that custom assembly class loader classes and their single argument Assembly constructor must be accessible from the assembly that they are applied to.

Delegation Magic

The ikvm.runtime.AssemblyClassLoader instance that is passed in as the parent class loader is a bit funny. It supports loading classes and resources from the assembly, but it effectively does this by magically delegating back to the custom class loader (without actually calling it, otherwise you'd get infinite recursion.) This means that classes from the assembly can be loaded via the parent class loader, but they report the custom class loader instance as their class loader.

Construction Magic

The custom assembly class loader is constructed when someone calls getClassLoader() on a class defined in that assembly for the first time, but what should getClassLoader() return while the custom class loader is being constructed? For consistencies sake, I've decided to apply some magic here and to make it return the object that is currently being constructed. This implies that you should call the base class constructor ASAP, because as long as you haven't done this the ClassLoader instance is in an unitialized state. After calling the base class constructor you can do additional work to configure your custom class loader, but you should take into account that while you are setting up your instance data, if you do anything that causes a class or resource to be loaded via your class loader, that your methods may run while initialization has not yet been completed. Note that this can only happen for the initiating thread. All other threads that call getClassLoader() will be blocked while the constructor is runnning. If the constructor throws an exception, the initiating call to getClassLoader() will see the exception, but all other calls will return the partially initialized class loader object.

Too Much Magic?

I certainly hope not, but I have to admit this is a scary feature. I recommend only writing your own custom assembly class loader if you really understand Java class loaders and there is no alternative.

Finally, please do send me feedback on this feature (even if it's a simple "I like it" or "It sucks"). I plan on backporting this to the 0.36 release, so it's important to get this right.

Changes:

  • More changes to keep up the pretense that an AssemblyClassLoader has already loaded all classes in the assembly.
  • Made AssemblyClassLoader public and moved it to ikvm.runtime package.
  • Allow non-public custom assembly class loaders (if they are defined in the same assembly).
  • Fixed ClassLoader.getSystemClassLoader() to return the custom assembly class loader for the main executable, if one is defined, even if the java.class.path or java.ext.dirs system property is set.
  • Restructured AssemblyClassLoader.GetProtectionDomain() to avoid calling any code while holding the lock.

Binaries available here: ikvmbin-0.37.2855.zip.

Friday, 26 October 2007 11:25:23 (W. Europe Daylight Time, UTC+02:00)  #    Comments [4]
# Monday, 22 October 2007
New Development Snapshot

I've made a bunch of changes to support custom assembly class loaders. This should finally solve some of the issues caused by the new class loader architecture. Eventually it will be possible to write your own assembly class loader (hence the name custom assembly class loaders), but currently only the two provided ones are supported: ikvm.runtime.AppDomainAssemblyClassLoader and ikvm.runtime.ClassPathAssemblyClassLoader.

AppDomainAssemblyClassLoader

This approximates the pre-0.32 behavior by searching all currently loaded assemblies (after looking in the assembly itself and the assemblies it references).

ClassPathAssemblyClassLoader

This class loader supports dynamically loading classes and resources from the CLASSPATH (or java.class.path system property). It too first searches the assembly itself and any assemblies it references. This class loader makes the most sense as a replacement for the assembly class loader of the main executable, that way it will also be the system class loader. Note that if multiple assemblies use this class loader, they each will have a separate class loader instance that loads different classes (even though they may be based on the same class files).

How to Use

There is a new ikvmc option to specify a custom assembly class loader:

ikvmc -classloader:ikvm.runtime.AppDomainAssemblyClassLoader example.jar

For non-Java code you can apply a custom attribute to your code:

using System;

[assembly: IKVM.Attributes.CustomAssemblyClassLoader(typeof(ikvm.runtime.ClassPathAssemblyClassLoader))]

class SimpleJavaRunner
{
  public static void Main(string[] args)
  {
    string[] newArgs = new string[args.Length - 1];
    Array.Copy(args, 1, newArgs, 0, newArgs.Length);
    java.lang.Class.forName(args[0])
      .getMethod("main", typeof(string[]))
      .invoke(null, new object[] { newArgs });
  }
}

You can also specify them in the app.config file, this overrides the above methods:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="ikvm-classloader:MyExampleAssembly" value="ikvm.runtime.AppDomainAssemblyClassLoader, IKVM.OpenJDK.ClassLibrary, Version=0.37.0.0, Culture=neutral, PublicKeyToken=null" />
  </appSettings>
</configuration>

Here the key is ikvm-classloader: followed by the simple name of the assembly. The value must be the assembly qualified type name of the custom assembly class loader type.

As always, feedback is appreciated.

Changes:

  • Implemented Custom Assembly Class Loaders.
  • Fixed URL.equals() for "ikvmres:" urls.
  • Fixed findResources() bug that caused resources in core assembly to show up twice if a .NET assembly explicitly referenced the core assembly.
  • Added check to prevent defineClass() on an assembly class loader from defining a class that already exists in the assembly.
  • Fixed Class.getModifiers() to mask out ACC_SUPER bit.

Binaries available here: ikvmbin-0.37.2851.zip.

Monday, 22 October 2007 15:22:38 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Monday, 15 October 2007
Mono Summit 2007

I'll be at the Mono Summit 2007.

Monday, 15 October 2007 11:59:24 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]