# Wednesday, 09 June 2004
AppDomains, Domain Neutral Assemblies and a Framework Bug

I came across an nasty bug in the Microsoft .NET Framework 1.1 today.

Here is a program that reproduces the problem:

using System;
using System.Reflection;

class Class1
{
  [LoaderOptimization(LoaderOptimization.MultiDomain)]
  static void Main(string[] args)
  {
    if(AppDomain.CurrentDomain.FriendlyName != "My 2nd Domain")
    {
      AppDomain dom = AppDomain.CreateDomain("My 2nd Domain");
      dom.ExecuteAssembly(Assembly.GetEntryAssembly().Location);
    }
    Console.WriteLine(AppDomain.CurrentDomain.FriendlyName);
    Console.WriteLine(typeof(Class1).TypeHandle.Value);
    new System.Data.RowNotInTableException();
    foreach(Assembly a in AppDomain.CurrentDomain.GetAssemblies())
    {
      Console.WriteLine(a.FullName);
    }
  }
}

In the 2nd AppDomain (the first one that prints), the list of assemblies doesn't contain System.Data, even though it clearly is present in the AppDomain, after all, the new System.Data.RowNotInTableException() was just executed.

The problem only occurs when assemblies are shared across AppDomains (this is what the [LoaderOptimization(LoaderOptimization.MultiDomain)] attribute does). Comment out that line and the program works as expected.

I think that this effectively means that IKVM cannot be used inside of ASP.NET, which, I believe, always loads assemblies domain neutral.

In the May 2004 Community Technology Preview of Visual Studio 2005, the bug is fixed and the above program performs as expected.

Wednesday, 09 June 2004 16:33:52 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
Eclipse

Ultratoto14 asks (in the comments):

I tried to launch eclipse M8 and M9 with the latest binaries. It does not run.

I saw that the on april 15 you successfully started it.

Is there something special to do or is the latest bin cannot be used ?

Thanks in advance.

M8 runs, but M9 doesn't. For M8 to run, you do need to add the xalan and xerces jars to the bootclasspath (wrapped for readability):

eclipse -vm c:\ikvm\bin\ikvm.exe
          -vmargs
          -Xbootclasspath:C:\xalan-j_2_6_0\bin\xalan.jar;
                         C:\xalan-j_2_6_0\bin\xercesImpl.jar;
                         C:\xalan-j_2_6_0\bin\xml-apis.jar

Note that this is on Windows. I haven't tried running Eclipse on Mono/Linux.

Wednesday, 09 June 2004 12:13:50 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Monday, 07 June 2004
New Snapshot

I made a new snapshot with a couple of small fixes, but this is mainly to update the version of SharpZipLib to 0.6, because this is the version that the Mono Beta 2 ships with.

Here's what's new:

  • Verifier fix. Calling a method on an unloadable class could cause assertion to be fired.
  • Fixed ikvmc to tolerate multiple definitions of the same class (classes that have the same name as a class that was already processed are ignored now, with a warning instead of aborting compilation with a fatal error). I ran into a jar that contained multiple entries for the same class!
  • Compiled with 0.6 version of SharpZipLib to be compatible with the version shipped with Mono Beta 2.
  • Removed the IKVM runtime version number from the JavaModuleAttribute, because I realised the same information is available by using Assembly.GetReferencedAssemblies().
  • Fixed ikvmstub to print help message instead of crash, if invoked without argument.
  • Fixed ikvmstub.csproj (Visual Studio .NET project file) to generate ikvmstub.exe, instead of netexp.exe.
  • Fixed the bytecode compiler to emit verifiable code when it is compiling invalid code (e.g. code that produces NoSuchFieldError or IllegalAccessError). This was the last know issue, all non-JNI using code generated by ikvmc should now be verifiable. If you encounter unverifiable code, please let me know.

New snapshots: just the binaries and source plus binaries.

Monday, 07 June 2004 15:03:16 (W. Europe Daylight Time, UTC+02:00)  #    Comments [3]
# Tuesday, 01 June 2004
Compatibility List

On the ikvm-developers list, Kevin Gilpin suggested to create a list of Java applications and libraries that are compatible with IKVM. This is a great idea. Brian Sletten, who is working on the documentation, offered to collect the list.

So, if you have successfully (or unsuccessfully) run or used any Java application or library (open source or proprietary), please let us know, so it can be added to the list.

In addition to the name and version of the application or library, please supply the following information (not required, but it would be nice, if you have it available):

  • Version of IKVM you used.
  • Do you want your name and/or e-mail address in the list?
  • .NET runtime version used (e.g. MS .NET 1.1 or Mono 0.91).
  • Operating System version (e.g. Windows XP or Debian 3.0).
  • Whether you used the application/library in static (ikvmc.exe) or dynamic mode (ikvm.exe), or in a mixed scenario.
  • Any build issues you encountered.
  • Any other compatibility issues you encountered.

Please send your feedback to me or Brian, or leave a comment to this post.

Update: In the comments, Jamie Cansdale asks for unit test results for the application or library. That's a good point. If they are available please report the unit test results as well. The JUnit 3.8.1 command line test runner should work with IKVM (at least for the samples tests run successfully, apart from three failures that result from JUnit trying to use Runtime.exec() to start java.exe).

Tuesday, 01 June 2004 10:23:07 (W. Europe Daylight Time, UTC+02:00)  #    Comments [4]
# Thursday, 27 May 2004
Painful Changes

I renamed the assemblies and also took the opportunity to change the directory structure and Visual Studio project names accordingly. This is quite a painful process and also loses the cvs history for all the moved files (at least, I couldn't find any way of moving files in cvs, other than add/remove).

Hopefully this will be the only time I have to make these changes. Breaking all build scripts and what have you, isn't fun.

Note: To build this version, you require either Jikes 1.19 or 1.21.

Here's what's new:

  • Changed all assembly versions to 0.7.* (except for the IKVM.GNU.Classpath assembly, that now has version 0.9, to indicate the GNU Classpath version)
  • Removed the StackTraceInfo attribute (wasn't supported by the stack trace code anymore).
  • Made ExceptionHelper (in ExceptionHelper.cs) private.
  • Changed ThrowsAttribute to take a string array instead of a single string (to support reporting the throws clause in declaration order).
  • Changed ImplementsAttribute to take a string array instead of a single Type (to support reporting the implemented interfaces in declaration order).
  • Made handling of InnerClasses attribute more robust (this applies to ikvmc only, the dynamic runtime ignores this attribute).
  • Made the op_Implicit method that is added to classes that implement ghost interfaces hidden from reflection.
  • Added MethodAttributes.CheckAccessOnOverride to virtual method definitions, to prevent package private methods from being overridden in another assembly.
  • Fixed reflection on sealed .NET types to also add public method for private interface implementations.
  • Added a version string to the JavaModuleAttribute, to record the IKVM runtime version that was used to generate the module.
  • Removed unused NativeCode.java.io.File class.
  • Removed unused NativeCode.java.nio.channels.FileChannelImpl class.
  • Fixed a bug in the exception untangling code and cleaned the code up a bit.
  • Fixed FileChannelImpl to catch System.NotSupportedException and rethrow a FileNotFoundException.
  • Added explicit call to Environment.Exit to ikvmc to workaround background threads starting up due to static initializers running (due to .NET Framework reflection bug).
  • Fixed ikvmc regression that caused ArgumentException when compiling a bunch of classes with a wildcard expression.
  • Added dummy ftp protocol handler to work around Classpath bug (when it sees a file: url with a host, it treats it as an ftp url, but since the ftp protocol doesn't exist, the code gets stuck in an infinite loop).

New snapshots: just the binaries and source plus binaries.

Thursday, 27 May 2004 15:27:23 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Monday, 10 May 2004
Generics Again

I have a hard time truly understanding the Java generics model. I understand the implementation and I think it's a hack and this is probably making it harder for me to see the design objectively.

After looking at the recent JDK 1.5 beta, there was one method that particularly puzzled me:

public T cast(Object obj)

In java.lang.Class. Why would you need a method to dynamically cast? If you understand how generics work under the covers, you know that T is actually java.lang.Object, so calling this cast method doesn't actually buy you anything (the compiler will insert a real cast after your call).

Before diving in, if your knowledge of how Java generics work under the covers is a bit rusty, I recommend reading the paper on GJ: Making the future safe for the past: Adding Genericity to the Java Programming Language. While I was re-reading it, a few things jumped out: "Adding generics to an existing language is almost routine.." In fact, it's so easy that today in 2004 we still don't have generics in Java, even though the design was already done in 1998. Another quote: "GJ comes with a cast-iron guarantee: no cast inserted by the compiler will ever fail. (Caveat: this guarantee is void if the compiler generates an 'unchecked' warning, which may occur if legacy and parametric code is mixed without benefit of retrofitting.)" Please note that the caveat applies when you use one of the principal features of GJ: "GJ contains a novel language feature, raw types, to capture the correspondence between generic and legacy types, and a retrofitting mechanism to allow generic types to be imposed on legacy code."

Another interesting paper (that is referenced in the GJ paper as NextGen), is Compatible Genericity with Run-time Types for the Java TM Programming Language.

Anyway, let's look at some code and try to figure out what the purpose of the new cast method is:

Class<String> c = String.class;;
String s = c.cast("foo");;

This is compiled as:

ldc_w java/lang/String
astore_1
aload_1
ldc "foo"
invokevirtual java/lang/Class cast(Ljava/lang/Object;)Ljava/lang/Object;
checkcast java/lang/String
astore_2

Note in particular the checkcast that I highlighted. This is inserted by the compiler and is key to how Java generics work. Presumably the Class.cast() method also checks that the passed in object is actually castable to the type, so effectively you get two casts for the price of one (or rather, you pay twice for the same cast).

Why would you want (or need) that? The answer became clear (?) to me when I was contrasting the Java generics with the .NET generics model.

Let's look at some more code (C# 2.0 this time):

T LameFactory<T>() where T : new()
{
  return new T();
}

This method generically constructs instances. It's not relevant to my point, but it is interesting to note that the C# compiler uses reflection under the covers to implement this. Most C# generics constructs are actually supported by the CLR, but instantation isn't. The compiler generates something like this:

T LameFactory<T>()
{
  // If T is a Value Type, we don't need
  // to use Activator.CreateInstance.
  if((object)T.default != null)
  {
    return T.default;
  }
  return (T)Activator.CreateInstance(typeof(T));
}

How would you do something similar in Java? Here's how:

T LameFactory(Class<T> factory)
{
  return factory.newInstance();
}

In essence, what you're doing here is the same as what the CLR is doing under the covers (passing an extra parameter with the type information, at least conceptually). Class.newInstance returns an appropriately typed reference, because Class is a generic type. Suppose it hadn't returned the appropriate type, but simply Object like in the good old days. You could have used Class.cast() to the the downcast instead! Admittedly this isn't the greatest example for explaining the existence of Class.cast(), but I do understand now that it provides real functionality that would have been impossible to (safely) get in any order way. Note that the obvious:

T LameFactory(Class factory)
{
  Object o = factory.newInstance();
  return (T)o;
}

Isn't the right answer. When compiling this, the compiler rightfully warns: Note: cast.java uses unchecked or unsafe operations. The cast is unsafe, because it dissolves at compile time. Note that this doesn't break type safety (in the VM/security sense), because the caller of LameFactory will have inserted its own cast to T, but it does (allow you to) break compile time type safety. If you value compile time type safety, it's a good idea to stay away from code that generates this warning.

Monday, 10 May 2004 12:07:15 (W. Europe Daylight Time, UTC+02:00)  #    Comments [4]
# Friday, 07 May 2004
Assembly Names

To clean up the assembly names, I propose to change to the following names for the next snapshot:

Current Name New Name
awt.dll IKVM.AWT.WinForms.dll
classpath.dll IKVM.GNU.Classpath.dll
ik.vm.jni.dll IKVM.JNI.CLR-Win32.dll
ik.vm.net.dll IKVM.Runtime.dll
ikvm.exe ikvm.exe
ikvmc.exe ikvmc.exe
Mono.IKVM.JNI.dll IKVM.JNI.Mono.dll
netexp.exe ikvmstub.exe
OpenSystem.Java.dll (will be removed)


I'm removing OpenSystem.Java.dll for the time being, because interop with dotGNU hasn't really happened so far and at this point it's just pre-mature design for reuse. Hopefully, in the future when they continue their work on Java support we can work together to make the two systems interoperable.

Any comments on the assembly names?

[Update: Stuart makes a good point in the comments. I've changed the suggested new name of netexp to ikvmstub.exe]

Friday, 07 May 2004 15:52:52 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Wednesday, 05 May 2004
Mono Beta 1

The Mono Beta 1 was released today and I've prepared a new snapshot that works with this version to test interoperability.

What's new?

  • Fixed ik.vm.jni.build to create output directory.
  • Added Profiler.Count calls to ByteCodeHelper.cs for dynamic helper methods.
  • Fixed TypeWrapper.IsSamePackage to handle case where one type is array and other isn't.
  • Fixed bug introduced in last snapshot in bytecode compiler, that caused NullReferenceException if exception block starts on an unreachable instructions.
  • Added exception handling to VMRuntime.exec.
  • Added -Xwait option to ikvm, to keep the process hanging around (this is just a hack so I can look at the console window that contains the profiling statistics and trace messages after Eclipse has exited).
  • Fixed ikvmc to ignore Main-Class attribute when building a library.
  • Implemented JNI methods AllocObject and ReleasePrimitiveArrayCritical (on Windows JNI provider only).
  • Fixed resource loading from -Xbootclasspath.
  • Fixed a minor bug in remapped type handling.
  • Changed map.xml format to support future metadata annotations (thanks Stuart!).
  • Fixed handling of final fields in ikvmc.
  • Changed handling of shadow types (e.g. cli.System.Object now extends java.lang.Object and cli.System.String exists side by side with java.lang.String).
  • Fixed support for deserializing final fields.
  • Fixed java.lang.Class/VMClass to no longer require VMClass instances (Class now contains direct reference to TypeWrapper).
  • Added the managed part of the Mono JNI provider to CVS and the build script (and the snapshot).
  • Updated the SharpZipLib reference to the latest version.
  • Locked to GNU Classpath 0.09 and changed classpath build directory to ../../classpath-0.09
  • Fixed Thread.sleep() to support larger values than Integer.MAX_VALUE

Linux Lamer

Since I'm a Linux Lamer (tm), I'm documenting the steps here to get JNI to work with Mono (primarily for my own use).

  • cd /usr/local/src/classpath-0.09
  • ./configure --with-jikes --disable-gtk-peer
  • make
  • cp include/jni.h /usr/local/include/
  • cp include/jni_md.h /usr/local/include/
  • cd ../mono-0.91/ikvm-jni/
  • make && make install
  • cd ../jnitest
  • vi test.java
    class test {
      public static void main(String[] args) {
        System.loadLibrary("test");
        nativeMethod();
      }
      static native void nativeMethod();
    }
  • jikes -cp ../classpath-0.09:../classpath-0.09/vm/reference:../ikvm/classpath test.java
  • vi test.c
    #include <stdio.h>
    #include <jni.h>
    JNIEXPORT void JNICALL Java_test_nativeMethod(
    JNIEnv* env, jclass clazz) { printf("Hello from JNI\n"); }
  • gcc -shared test.c -o libtest.so
  • export LD_LIBRARY_PATH=.
  • ikvm test

New snapshots: just the binaries and source plus binaries.

The GNU Classpath release that this is based on can be downloaded from ftp://ftp.gnu.org/pub/gnu/classpath/classpath-0.09.tar.gz

Note that anon cvs for SourceForge is apparently still lagging behind, so at the moment I'm posting this, anon cvs does not yet contain the latest changes.

Wednesday, 05 May 2004 11:06:43 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Thursday, 15 April 2004
Progress

While it's been quiet on the blog and the mailing list, there has been quite a lot of progress behind the scenes.

  • The next Mono release will contain the C half of the IKVM JNI provider and the next IKVM snapshot will contain the C# half of the Mono JNI provider. This means that JNI will work out of the box on Mono (for the parts of JNI that are actually implemented). Thanks to Zoltan and Miguel for this.
  • I'm planning an IKVM 0.8 release to coincide with the Mono 1.0 release.
  • John Luke added IKVM support to MonoDevelop. Read about it here or see the screenshot here.
  • I successfully started up Eclipse 3.0 M8 for the first time yesterday. Thanks to Michael Koch for his work on GNU Classpath's java.nio implementation and all the other GNU Classpath hackers, of course.
  • In the comments, Jesus Garcia point me to SwingWT. An SWT based implementation of AWT and Swing by Robin Rawson-Tetley. Very cool stuff! It doesn't run on the latest snapshot due to a JNI bug, but I have it running and it's very cool to see the SwingSet demo running on IKVM.

I hope to do a new snapshot in the first week of May and after that to work towards the 0.8 release.

Thursday, 15 April 2004 10:06:15 (W. Europe Daylight Time, UTC+02:00)  #    Comments [4]
# Monday, 29 March 2004
Backward Branch Constraints

The last snapshot was totally broken on Mono (*blush*). My apologies to the Mono users. I should test my snapshots on Mono before releasing them, but I'm lazy so this sometimes slips through the cracks.

The breakage was kind of interesting though. As I wrote last time, I rewrote the bytecode compiler to emit CIL in the same order as the Java bytecode. This caused invalid (per the ECMA spec) CIL to be generated in some cases. The interesting thing is that the code isn't really invalid, the only reason it is invalid is because the spec says so:

Partition III -- 1.7.5 Backward Branch Contraints

It must be possible, with a single forward-pass through the CIL instruction stream for any method, to infer the exact state of the evaluation stack at every instruction (where by “state” we mean the number and type of each item on the evaluation stack).

In particular, if that single-pass analysis arrives at an instruction, call it location X, that immediately follows an unconditional branch, and where X is not the target of an earlier branch instruction, then the state of the evaluation stack at X, clearly, cannot be derived from existing information. In this case, the CLI demands that the evaluation stack at X be empty.

(Note that the section numbering seems to change with each version, this is from the working draft of June 2003.)

Java bytecode has no such requirment, so my straight forward translation caused Java bytecode that is only reachable through a backward branch to be translated into invalid CIL.

The Microsoft verifier and JIT don't require this constraint to be met and they will happily verify and JIT code that violates this constraint. However, the Mono JIT relies on it and so it was unable to handle some of the CIL that IKVM generated.

I fixed the IKVM bytecode compiler to emit ECMA compliant code (at least for this particular issue, who knows what else is wrong). I also fixed exception mapping, which didn't work on Mono as it relied on a currently unimplemented feature (I think, I didn't really investigate).

I think that, realistically, the Mono JIT will also have to be "fixed" to support the broken code, as you can be sure that there will be compilers that emit broken code, because it works on the Microsoft runtime.

New snapshots: just the binaries and source plus binaries.

Monday, 29 March 2004 12:56:28 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]