# Monday, 07 June 2004
New Snapshot

I made a new snapshot with a couple of small fixes, but this is mainly to update the version of SharpZipLib to 0.6, because this is the version that the Mono Beta 2 ships with.

Here's what's new:

  • Verifier fix. Calling a method on an unloadable class could cause assertion to be fired.
  • Fixed ikvmc to tolerate multiple definitions of the same class (classes that have the same name as a class that was already processed are ignored now, with a warning instead of aborting compilation with a fatal error). I ran into a jar that contained multiple entries for the same class!
  • Compiled with 0.6 version of SharpZipLib to be compatible with the version shipped with Mono Beta 2.
  • Removed the IKVM runtime version number from the JavaModuleAttribute, because I realised the same information is available by using Assembly.GetReferencedAssemblies().
  • Fixed ikvmstub to print help message instead of crash, if invoked without argument.
  • Fixed ikvmstub.csproj (Visual Studio .NET project file) to generate ikvmstub.exe, instead of netexp.exe.
  • Fixed the bytecode compiler to emit verifiable code when it is compiling invalid code (e.g. code that produces NoSuchFieldError or IllegalAccessError). This was the last know issue, all non-JNI using code generated by ikvmc should now be verifiable. If you encounter unverifiable code, please let me know.

New snapshots: just the binaries and source plus binaries.

Monday, 07 June 2004 15:03:16 (W. Europe Daylight Time, UTC+02:00)  #    Comments [3]
# Tuesday, 01 June 2004
Compatibility List

On the ikvm-developers list, Kevin Gilpin suggested to create a list of Java applications and libraries that are compatible with IKVM. This is a great idea. Brian Sletten, who is working on the documentation, offered to collect the list.

So, if you have successfully (or unsuccessfully) run or used any Java application or library (open source or proprietary), please let us know, so it can be added to the list.

In addition to the name and version of the application or library, please supply the following information (not required, but it would be nice, if you have it available):

  • Version of IKVM you used.
  • Do you want your name and/or e-mail address in the list?
  • .NET runtime version used (e.g. MS .NET 1.1 or Mono 0.91).
  • Operating System version (e.g. Windows XP or Debian 3.0).
  • Whether you used the application/library in static (ikvmc.exe) or dynamic mode (ikvm.exe), or in a mixed scenario.
  • Any build issues you encountered.
  • Any other compatibility issues you encountered.

Please send your feedback to me or Brian, or leave a comment to this post.

Update: In the comments, Jamie Cansdale asks for unit test results for the application or library. That's a good point. If they are available please report the unit test results as well. The JUnit 3.8.1 command line test runner should work with IKVM (at least for the samples tests run successfully, apart from three failures that result from JUnit trying to use Runtime.exec() to start java.exe).

Tuesday, 01 June 2004 10:23:07 (W. Europe Daylight Time, UTC+02:00)  #    Comments [4]
# Thursday, 27 May 2004
Painful Changes

I renamed the assemblies and also took the opportunity to change the directory structure and Visual Studio project names accordingly. This is quite a painful process and also loses the cvs history for all the moved files (at least, I couldn't find any way of moving files in cvs, other than add/remove).

Hopefully this will be the only time I have to make these changes. Breaking all build scripts and what have you, isn't fun.

Note: To build this version, you require either Jikes 1.19 or 1.21.

Here's what's new:

  • Changed all assembly versions to 0.7.* (except for the IKVM.GNU.Classpath assembly, that now has version 0.9, to indicate the GNU Classpath version)
  • Removed the StackTraceInfo attribute (wasn't supported by the stack trace code anymore).
  • Made ExceptionHelper (in ExceptionHelper.cs) private.
  • Changed ThrowsAttribute to take a string array instead of a single string (to support reporting the throws clause in declaration order).
  • Changed ImplementsAttribute to take a string array instead of a single Type (to support reporting the implemented interfaces in declaration order).
  • Made handling of InnerClasses attribute more robust (this applies to ikvmc only, the dynamic runtime ignores this attribute).
  • Made the op_Implicit method that is added to classes that implement ghost interfaces hidden from reflection.
  • Added MethodAttributes.CheckAccessOnOverride to virtual method definitions, to prevent package private methods from being overridden in another assembly.
  • Fixed reflection on sealed .NET types to also add public method for private interface implementations.
  • Added a version string to the JavaModuleAttribute, to record the IKVM runtime version that was used to generate the module.
  • Removed unused NativeCode.java.io.File class.
  • Removed unused NativeCode.java.nio.channels.FileChannelImpl class.
  • Fixed a bug in the exception untangling code and cleaned the code up a bit.
  • Fixed FileChannelImpl to catch System.NotSupportedException and rethrow a FileNotFoundException.
  • Added explicit call to Environment.Exit to ikvmc to workaround background threads starting up due to static initializers running (due to .NET Framework reflection bug).
  • Fixed ikvmc regression that caused ArgumentException when compiling a bunch of classes with a wildcard expression.
  • Added dummy ftp protocol handler to work around Classpath bug (when it sees a file: url with a host, it treats it as an ftp url, but since the ftp protocol doesn't exist, the code gets stuck in an infinite loop).

New snapshots: just the binaries and source plus binaries.

Thursday, 27 May 2004 15:27:23 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Monday, 10 May 2004
Generics Again

I have a hard time truly understanding the Java generics model. I understand the implementation and I think it's a hack and this is probably making it harder for me to see the design objectively.

After looking at the recent JDK 1.5 beta, there was one method that particularly puzzled me:

public T cast(Object obj)

In java.lang.Class. Why would you need a method to dynamically cast? If you understand how generics work under the covers, you know that T is actually java.lang.Object, so calling this cast method doesn't actually buy you anything (the compiler will insert a real cast after your call).

Before diving in, if your knowledge of how Java generics work under the covers is a bit rusty, I recommend reading the paper on GJ: Making the future safe for the past: Adding Genericity to the Java Programming Language. While I was re-reading it, a few things jumped out: "Adding generics to an existing language is almost routine.." In fact, it's so easy that today in 2004 we still don't have generics in Java, even though the design was already done in 1998. Another quote: "GJ comes with a cast-iron guarantee: no cast inserted by the compiler will ever fail. (Caveat: this guarantee is void if the compiler generates an 'unchecked' warning, which may occur if legacy and parametric code is mixed without benefit of retrofitting.)" Please note that the caveat applies when you use one of the principal features of GJ: "GJ contains a novel language feature, raw types, to capture the correspondence between generic and legacy types, and a retrofitting mechanism to allow generic types to be imposed on legacy code."

Another interesting paper (that is referenced in the GJ paper as NextGen), is Compatible Genericity with Run-time Types for the Java TM Programming Language.

Anyway, let's look at some code and try to figure out what the purpose of the new cast method is:

Class<String> c = String.class;;
String s = c.cast("foo");;

This is compiled as:

ldc_w java/lang/String
astore_1
aload_1
ldc "foo"
invokevirtual java/lang/Class cast(Ljava/lang/Object;)Ljava/lang/Object;
checkcast java/lang/String
astore_2

Note in particular the checkcast that I highlighted. This is inserted by the compiler and is key to how Java generics work. Presumably the Class.cast() method also checks that the passed in object is actually castable to the type, so effectively you get two casts for the price of one (or rather, you pay twice for the same cast).

Why would you want (or need) that? The answer became clear (?) to me when I was contrasting the Java generics with the .NET generics model.

Let's look at some more code (C# 2.0 this time):

T LameFactory<T>() where T : new()
{
  return new T();
}

This method generically constructs instances. It's not relevant to my point, but it is interesting to note that the C# compiler uses reflection under the covers to implement this. Most C# generics constructs are actually supported by the CLR, but instantation isn't. The compiler generates something like this:

T LameFactory<T>()
{
  // If T is a Value Type, we don't need
  // to use Activator.CreateInstance.
  if((object)T.default != null)
  {
    return T.default;
  }
  return (T)Activator.CreateInstance(typeof(T));
}

How would you do something similar in Java? Here's how:

T LameFactory(Class<T> factory)
{
  return factory.newInstance();
}

In essence, what you're doing here is the same as what the CLR is doing under the covers (passing an extra parameter with the type information, at least conceptually). Class.newInstance returns an appropriately typed reference, because Class is a generic type. Suppose it hadn't returned the appropriate type, but simply Object like in the good old days. You could have used Class.cast() to the the downcast instead! Admittedly this isn't the greatest example for explaining the existence of Class.cast(), but I do understand now that it provides real functionality that would have been impossible to (safely) get in any order way. Note that the obvious:

T LameFactory(Class factory)
{
  Object o = factory.newInstance();
  return (T)o;
}

Isn't the right answer. When compiling this, the compiler rightfully warns: Note: cast.java uses unchecked or unsafe operations. The cast is unsafe, because it dissolves at compile time. Note that this doesn't break type safety (in the VM/security sense), because the caller of LameFactory will have inserted its own cast to T, but it does (allow you to) break compile time type safety. If you value compile time type safety, it's a good idea to stay away from code that generates this warning.

Monday, 10 May 2004 12:07:15 (W. Europe Daylight Time, UTC+02:00)  #    Comments [4]
# Friday, 07 May 2004
Assembly Names

To clean up the assembly names, I propose to change to the following names for the next snapshot:

Current Name New Name
awt.dll IKVM.AWT.WinForms.dll
classpath.dll IKVM.GNU.Classpath.dll
ik.vm.jni.dll IKVM.JNI.CLR-Win32.dll
ik.vm.net.dll IKVM.Runtime.dll
ikvm.exe ikvm.exe
ikvmc.exe ikvmc.exe
Mono.IKVM.JNI.dll IKVM.JNI.Mono.dll
netexp.exe ikvmstub.exe
OpenSystem.Java.dll (will be removed)


I'm removing OpenSystem.Java.dll for the time being, because interop with dotGNU hasn't really happened so far and at this point it's just pre-mature design for reuse. Hopefully, in the future when they continue their work on Java support we can work together to make the two systems interoperable.

Any comments on the assembly names?

[Update: Stuart makes a good point in the comments. I've changed the suggested new name of netexp to ikvmstub.exe]

Friday, 07 May 2004 15:52:52 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Wednesday, 05 May 2004
Mono Beta 1

The Mono Beta 1 was released today and I've prepared a new snapshot that works with this version to test interoperability.

What's new?

  • Fixed ik.vm.jni.build to create output directory.
  • Added Profiler.Count calls to ByteCodeHelper.cs for dynamic helper methods.
  • Fixed TypeWrapper.IsSamePackage to handle case where one type is array and other isn't.
  • Fixed bug introduced in last snapshot in bytecode compiler, that caused NullReferenceException if exception block starts on an unreachable instructions.
  • Added exception handling to VMRuntime.exec.
  • Added -Xwait option to ikvm, to keep the process hanging around (this is just a hack so I can look at the console window that contains the profiling statistics and trace messages after Eclipse has exited).
  • Fixed ikvmc to ignore Main-Class attribute when building a library.
  • Implemented JNI methods AllocObject and ReleasePrimitiveArrayCritical (on Windows JNI provider only).
  • Fixed resource loading from -Xbootclasspath.
  • Fixed a minor bug in remapped type handling.
  • Changed map.xml format to support future metadata annotations (thanks Stuart!).
  • Fixed handling of final fields in ikvmc.
  • Changed handling of shadow types (e.g. cli.System.Object now extends java.lang.Object and cli.System.String exists side by side with java.lang.String).
  • Fixed support for deserializing final fields.
  • Fixed java.lang.Class/VMClass to no longer require VMClass instances (Class now contains direct reference to TypeWrapper).
  • Added the managed part of the Mono JNI provider to CVS and the build script (and the snapshot).
  • Updated the SharpZipLib reference to the latest version.
  • Locked to GNU Classpath 0.09 and changed classpath build directory to ../../classpath-0.09
  • Fixed Thread.sleep() to support larger values than Integer.MAX_VALUE

Linux Lamer

Since I'm a Linux Lamer (tm), I'm documenting the steps here to get JNI to work with Mono (primarily for my own use).

  • cd /usr/local/src/classpath-0.09
  • ./configure --with-jikes --disable-gtk-peer
  • make
  • cp include/jni.h /usr/local/include/
  • cp include/jni_md.h /usr/local/include/
  • cd ../mono-0.91/ikvm-jni/
  • make && make install
  • cd ../jnitest
  • vi test.java
    class test {
      public static void main(String[] args) {
        System.loadLibrary("test");
        nativeMethod();
      }
      static native void nativeMethod();
    }
  • jikes -cp ../classpath-0.09:../classpath-0.09/vm/reference:../ikvm/classpath test.java
  • vi test.c
    #include <stdio.h>
    #include <jni.h>
    JNIEXPORT void JNICALL Java_test_nativeMethod(
    JNIEnv* env, jclass clazz) { printf("Hello from JNI\n"); }
  • gcc -shared test.c -o libtest.so
  • export LD_LIBRARY_PATH=.
  • ikvm test

New snapshots: just the binaries and source plus binaries.

The GNU Classpath release that this is based on can be downloaded from ftp://ftp.gnu.org/pub/gnu/classpath/classpath-0.09.tar.gz

Note that anon cvs for SourceForge is apparently still lagging behind, so at the moment I'm posting this, anon cvs does not yet contain the latest changes.

Wednesday, 05 May 2004 11:06:43 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Thursday, 15 April 2004
Progress

While it's been quiet on the blog and the mailing list, there has been quite a lot of progress behind the scenes.

  • The next Mono release will contain the C half of the IKVM JNI provider and the next IKVM snapshot will contain the C# half of the Mono JNI provider. This means that JNI will work out of the box on Mono (for the parts of JNI that are actually implemented). Thanks to Zoltan and Miguel for this.
  • I'm planning an IKVM 0.8 release to coincide with the Mono 1.0 release.
  • John Luke added IKVM support to MonoDevelop. Read about it here or see the screenshot here.
  • I successfully started up Eclipse 3.0 M8 for the first time yesterday. Thanks to Michael Koch for his work on GNU Classpath's java.nio implementation and all the other GNU Classpath hackers, of course.
  • In the comments, Jesus Garcia point me to SwingWT. An SWT based implementation of AWT and Swing by Robin Rawson-Tetley. Very cool stuff! It doesn't run on the latest snapshot due to a JNI bug, but I have it running and it's very cool to see the SwingSet demo running on IKVM.

I hope to do a new snapshot in the first week of May and after that to work towards the 0.8 release.

Thursday, 15 April 2004 10:06:15 (W. Europe Daylight Time, UTC+02:00)  #    Comments [4]
# Monday, 29 March 2004
Backward Branch Constraints

The last snapshot was totally broken on Mono (*blush*). My apologies to the Mono users. I should test my snapshots on Mono before releasing them, but I'm lazy so this sometimes slips through the cracks.

The breakage was kind of interesting though. As I wrote last time, I rewrote the bytecode compiler to emit CIL in the same order as the Java bytecode. This caused invalid (per the ECMA spec) CIL to be generated in some cases. The interesting thing is that the code isn't really invalid, the only reason it is invalid is because the spec says so:

Partition III -- 1.7.5 Backward Branch Contraints

It must be possible, with a single forward-pass through the CIL instruction stream for any method, to infer the exact state of the evaluation stack at every instruction (where by “state” we mean the number and type of each item on the evaluation stack).

In particular, if that single-pass analysis arrives at an instruction, call it location X, that immediately follows an unconditional branch, and where X is not the target of an earlier branch instruction, then the state of the evaluation stack at X, clearly, cannot be derived from existing information. In this case, the CLI demands that the evaluation stack at X be empty.

(Note that the section numbering seems to change with each version, this is from the working draft of June 2003.)

Java bytecode has no such requirment, so my straight forward translation caused Java bytecode that is only reachable through a backward branch to be translated into invalid CIL.

The Microsoft verifier and JIT don't require this constraint to be met and they will happily verify and JIT code that violates this constraint. However, the Mono JIT relies on it and so it was unable to handle some of the CIL that IKVM generated.

I fixed the IKVM bytecode compiler to emit ECMA compliant code (at least for this particular issue, who knows what else is wrong). I also fixed exception mapping, which didn't work on Mono as it relied on a currently unimplemented feature (I think, I didn't really investigate).

I think that, realistically, the Mono JIT will also have to be "fixed" to support the broken code, as you can be sure that there will be compilers that emit broken code, because it works on the Microsoft runtime.

New snapshots: just the binaries and source plus binaries.

Monday, 29 March 2004 12:56:28 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Friday, 26 March 2004
Debugging Support

I finished and fixed the local variable analysis and made various improvements to the debugging experience (in Visual Studio .NET). In the process I discovered some quirks in jikes' debugging tables. It associates two line numbers with the same bytecode address when that bytecode address is also the start of an exception block. It also (I think incorrectly) starts a local variable scope before the store instruction that first initializes that local variable. Javac starts the local variable scope immediately after the first store to that local, this makes more sense to me, but also required a hack in the local variable debugging support to make sure that the IL variable scope starts at the right position (i.e. before the first store to that local variable).

To make local variable scopes work, I had to change the bytecode compiler to emit IL in the same order as the original Java bytecode. For some reason I originally wrote the compiler to do the same code flow analysis as the verifier and this resulted in a somewhat arbitrary order of the generated IL. I finally fixed that and I also removed the recursion from the compiler (it now uses an explicit stack to keep track of exception blocks).

Note that unreachable code is still not compiled (it can't be verified, so it can't be compiled), but there will be nop IL instructions for each unreachable bytecode instruction, so you can set breakpoints and move the instruction pointer there. This could be a bit confusing and I probably should fix it at some point (by not emitting the nops).

I also had to reintroduce the automatic downcasting of locals, because I realised that it is possible to write bytecode by hand that depends on this (Java source will never require it). In non-debug builds of classpath.dll this introduces 12 (unnecessary) downcasts, so that's not really worth optimising.

What's new?

  • When compiling with -debug option, dead store are not optimized out anymore.
  • When compiling with -debug option, local variables are merged based on LocalVariableTable information.
  • When compiling with -debug option, the first instruction of a method will always (except when compiling with -Xmethodtrace) have an associated line number now (to enable stepping into the method).
  • When compiling with -debug option, local variables are now scoped based on LocalVariableTable information. Different locals with the same name are now handled correctly in the debugger.
  • Made helper method MethodInfo caching more consistent.
  • Method arguments that are value types are now boxed on method entry and not on each individual load, this makes them behave consistent with local variables.
  • Reusing method argument local variable slots (with different types) is now fully supported. I've never seen a Java compiler do this, but it is legal.
  • lookupswitch/tableswitch branches into exception blocks are now handled correctly (exception block is split around branch target). This completes branch/exception block handling. I originally thought that this would never happen, but apparently it does.
  • Added a helper method to emit Ldc_I4 that always uses the optimal encoding.
  • Added a class loader check to the System.arraycopy optimization check, to make sure that we're in fact calling System.arraycopy on the bootstrap version of java.lang.System.
  • Many fixes to the local variable analysis.
  • Added -srcpath option to ikvmc. If specified, this prepends the specified path and the package name to the source file name in the debugging information. In most cases this will be the correct location for the source file and will allow the debugger to automatically load the correct source.
  • Added AWT peer for Window.

New snapshots: just the binaries and source plus binaries.

Friday, 26 March 2004 11:30:36 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Thursday, 25 March 2004
Q & A

In the comments on the previous entry, Stuart asks:

Given an instance of System.Type, how do I determine what the typename is in Java?

This is tricky and my (lame) answer is that you shouldn't have to. Do you have a specific scenario in mind? Having said that, in your comment you're on the right track, there is no one-to-one mapping between System.Type instances and java.lang.Class instances. What I think you're after is the fact that you'll never encounter cli.System.Object or cli.System.Exception as field types or in method signatures. The only way to get the class object for these types, is by calling getClass() on an instance or by calling Class.getSuperclass() on a subclass of them.

Note that the above behaviour isn't actually implemented yet. At the moment cli.System.Object and cli.System.Exception are simply helper classes with static methods and you'll never encounter them as field types, in method signatures or as instance types.

As to which one to use from Java, you're right when you say java.lang.* is the answer is most cases. The only times you want to use cli.System.Object or cli.System.Exception is when you subclass them from Java to make your Java class look more .NET like to .NET consumers.

What would be really nice would be if there were something in the IK.VM.NET.dll that would let me answer this question authoritatively with a simple call...

There is the NativeCode.java.lang.VMClass.getClassFromType() method that is used by the runtime internally (it's public, so you could call it), but I can't really guarantee that it'll stay around or behave consistently over time. At some point in the future there'll probably be a utility class in the ikvm.lang package that will contain conversion method to go from System.Type to java.lang.Class and vice versa (but it probably will require you to specify to context for the mapping).

Oh, one other thing that occurred to me would be a nice feature: if a class implemented in Java could put itself into the cli.* package and thereby make itself look to other Java code as if it were a .NET type, without actually putting it in a cli.* namespace in .NET. In other words, a Java class cli.Foo.Bar would be compiled as namespace Foo.Bar *without* the attribute that preserves its name in Java, so that Foo.Bar then gets translated back to cli.Foo.Bar when Java code sees it.

Sort of the inverse of the attribute for turning name mangling off on the .NET side.

Can you explain why and when you'd want to use this?

Mike asks:

Are there any tasks for a CS scrub like myself to work on with IKVM?

Here is a partial list of things that need to be done:

  • Implement the AWT peers
  • Implement the missing JNI methods
  • Build a framework to test the verifier / bytecode compiler
  • Write a Java implementation of String.valueOf(float) and String.valueOf(double)
  • Search the source for // TODO for things that seem doable (or write test cases that show the current code is broken)
  • Write documentation
  • Design a logo
  • Design a website

If you (or anyone) decides to work on something, please send e-mail to the ikvm-developers list, so we can coordinate.

Thursday, 25 March 2004 10:28:51 (W. Europe Standard Time, UTC+01:00)  #    Comments [2]