# Monday, 23 January 2006
IKVM 0.24 rc2

João Saraiva reported a bug that I considered serious enough to fix for the 0.24 release, so here's a new release candidate. I also snuck in another AWT related fix.

Changes:

  • Changed WinForms event loop thread to STA.
  • Fixed handling of open generic types (by making sure they are invisible to Java code).

Files are available here: ikvm-0.24.0.1.zip (sources + binaries), ikvmbin-0.24.0.1.zip (binaries), ikvmbin-generics-0.24.0.1.zip (binaries built from generics branch)

Monday, 23 January 2006 11:15:40 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Monday, 16 January 2006
IKVM 0.24 rc1

A new release candidate based on GNU Classpath 0.20 that was released this weekend. The GNU Classpath progress has been truly amazing. Updated japi results are available here. If no major bugs are found in this rc it will turn into a release, probably by the end of the week or early next week.

Note that while I did a little work to improve compatibility with the .NET Compact Framework, it is not supported and does not work. There is also no commitment from my end to work on this.

Changes:

  • Integrated GNU Classpath 0.20
  • Changed nant build files to use NAnt 0.85-rc3 syntax and features
  • Removed support for compiling classpath with jikes
  • Added support for targetting both .NET 1.1 and .NET 2.0 to the build files
  • Removed assembly names from type names in map.xml (to better support targetting a different version of the CLR)
  • Added conditional compilation support to map.xml parser (based on target CLR version)
  • Changed map.xml Object.hashCode implementation to use RuntimeHelpers.GetHashCode when targetting .NET 2.0 (RuntimeHelpers.GetHashCode is broken on .NET 1.1 and the non-virtual call trick is no longer verifiable on .NET 2.0)
  • Added map.xml Object.equals implementation instead of relying on System.Object.Equals.
  • Changed map.xml VMSystem.identityHashCode to use RuntimeHelpers.GetHashCode when targetting .NET 2.0
  • Hardened VMSystemProperties to work better on the Compact Framework (where several methods are missing)
  • Fixed several NIO socket bugs
  • Changed FileChannelImpl to work on the Compact Framework (stdin, stdout and stderr are missing on the Compact Framework)
  • Hardened VMClassLoader to work better on the Compact Framework
  • Changed static compiler to use ReflectionOnly context for loading and generating assemblies (when compiled for .NET 2.0)
  • Added conditional compilation #ifs to the runtime to work better on the Compact Framework
  • Changed compilation of invokespecial bytecode to be verifiable on .NET 2.0
  • Added IKVM.Runtime.Util.GetInstanceTypeFromClass() to go from java.lang.Class to System.Type. Note that due to the object model mapping issues there is no one-to-one correspondence from Class to Type, so this method returns the "instance" type, which is logically equivalent to doing Class.newInstance().GetType().
  • Fixed ikvm.exe to give a proper error message when using the -jar option on a jar that doesn't have a manifest or a manifest that doesn't have a Main-Class attribute.
  • Added several hacks to ikvmc, ikvmstub and the runtime to support loading assemblies in the ReflectionOnly context.
  • Implemented optimization in compiler to remove redundant box/unbox operation is many cases.
  • Changed compiler to explicitly implement all inherited interfaces, for compatibility with the Compact Framework.
  • Fixed DotNetTypeWrapper.LoadTypeFromLoadedAssemblies to support generic type instantiations.
  • Optimized compiler to push/pop only the rightmost requires arguments when doing method argument conversions.
  • Implement "this" reference tracking in verifier, to enable code generator to emit more efficient (and verifiable on .NET 2.0) non-virtual base class method calls.
  • Optimized compiler to use short encoding when possible for backward branches.
  • Removed unnecessary verifier hack branch at the end of methods that don't use exception blocks and used short form of the branch.
  • Fixed Math.pow(1.0, Double.INFINITY) result on .NET 2.0.
  • Added SourceFileAttribute to module when running in dynamic mode, to enable source file reporting when running on .NET 2.0.
  • Changed code generator to leave out redundant branches.
  • Fixed infinite recursion in resource loading when SecurityManager is installed.

Files are available here: ikvm-0.24.0.0.zip (sources + binaries), ikvmbin-0.24.0.0.zip (binaries), ikvmbin-generics-0.24.0.0.zip (binaries built from generics branch)

Monday, 16 January 2006 15:15:07 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Saturday, 24 December 2005
Getting Incorrect Code to Run Correctly

One of the big challenges of doing a new implementation of an already existing platform is compatibility with the existing platform. Even though there is usually documentation or maybe even a specification that describes the functionality of the platform (and the things you can and cannot depend upon), in practice it turns out that much code that is written against a particular implementation will depend on implementation details. A typical response to this problem is: "That code is incorrect, because it makes assumptions that aren't guaranteed to hold."

The truth of the matter is, of course, that this issue isn't black and white. Sometimes the documentation or specification is vague or ambiguous, sometimes you need to depend on implementation details, because the documented interface simply isn't good enough or in many cases the dependency may simply be a bug in the code, but it's not always easy, desirable or even possible to fix the code.

Ultimately, when a customer has a piece of code that runs correctly on one implementation and it doesn't run on another implementation, the customer has a problem (and will in many cases assume that the problem is caused by the alternative implementation, since that's only thing that is being varied), so from a customer's point of view, supporting “incorrect” code can be a very important feature.

I've spent a lot of effort in IKVM to support incorrect code. Sometimes this can be frustrating, but I realize that the value of IKVM is to a large degree proportional to its compatibility with Java.

Some people in the GNU Classpath community think that we should implement the specification as correctly and efficiently as possible, but I'm often arguing for compatibility with the Sun implementation, even if this means duplicating buggy or inefficient behavior. Obviously this is a tricky issue, because Sun fixes bugs and sometimes makes backward compatibility breaking changes as well, so in every case we need to figure out whether it is likely that Sun will ever fix the bug (or change the implementation) and whether the benefit outweighs the cost.

It's important to always keep in mind though, that “The Right Thing” from our geeky developer oriented view doesn't always align with our customer's expectations and I think that most FOSS projects could benefit from a little bit more customer oriented thinking every now and then.

Saturday, 24 December 2005 13:01:15 (W. Europe Standard Time, UTC+01:00)  #    Comments [3]
# Thursday, 24 November 2005
IKVM 0.22 Released

I released 0.22 to SourceForge.

Thursday, 24 November 2005 10:50:03 (W. Europe Standard Time, UTC+01:00)  #    Comments [3]
# Monday, 14 November 2005
IKVM 0.22 rc1

It took a little longer than usual, but I finally managed to put together rc1 of IKVM 0.22 based on GNU Classpath 0.19. For the first time, this release also includes binaries built from the GNU Classpath generics branch. Many thanks to Andrew John Hughes for doing the work that makes this possible.

Update japi results are available here.

Changes:

  • Integrated GNU Classpath 0.19
  • Fixed bug 1310397.
  • Implemented reading package metadata from MANIFEST.MF for ikvmc compiled code.
  • Fixed ClassCastException in ExceptionHelper when Throwable instance methods are called on not remapped .NET exception that was caught in non-Java code and passed through a Java exception handler.
  • Fixed bug reported in support request 1280333.
  • Implemented RemappedTypeWrapper.Finish().
  • Implemented wakeup and waiting for empty list in classpath/gnu/java/nio/SelectorImpl.java
  • Fixed ikvmc resource compression bug that could cause last couple of bytes to fall off.
  • Added a META-INF/MANIFEST.MF to IKVM.GNU.Classpath.dll, so that Class.getPackage() returns the proper info for system classes.
  • Relaxed requirements for field and methods names (as per third edition VM specification).
  • Fixed runtime not to invoke user class loaders when instantiating an array type.
  • Improved weak reference support (improved performance and SoftReferences are not cleared as aggresively as before).
  • Fixed ikvmstub regression that caused .NET 2.0 generic types used by exported types not to be exported.

Files are available here: ikvm-0.22.0.0.zip (sources + binaries), ikvmbin-0.22.0.0.zip (binaries), ikvmbin-generics-0.22.0.0.zip (binaries built from generics branch)

Monday, 14 November 2005 12:08:37 (W. Europe Standard Time, UTC+01:00)  #    Comments [3]
# Saturday, 01 October 2005
GNU Classpath generics branch snapshot

As promised a while ago, here is the first IKVM snapshot based on the GNU Classpath generics branch. Note that this does not indicate a commitment to base the next release on the generics branch. I will only start basing my releases on the generics branch when GNU Classpath developer snapshots for the generics branch will be made. I don't know of there is any plan to do so at this time.

New in this release is the ability to reflect on the Java 1.5 generic type information. Based on these new APIs ikvmstub can now roundtrip generic type information (remember, this is Java generics, not .NET generics) and thanks to Stuart's work on Japi we can now get a status of the GNU Classpath library and at the same time test the IKVM/Classpath reflection infrastructure and ikvmstub. Results are available here.

The IKVM source + binaries snapshot is available here. This was build by specifying the “generics” target. Some of the changes in this version are only available when you build this target.

Changes:

  • Method.getModifiers() on ikvmc compiled code didn't return Modifier.SYNCHRONIZED for static synchronized methods.
  • Added support ceq IL instruction to remapper.
  • Added adhoc support for .NET type signatures in <call /> remapper instruction.
  • Improved build file support for building generics branch.
  • Added SignatureAttribute to various map.xml classes and methods.
  • Updated SharpZipLib to 0.84.
  • Added .NET attributes and support for capturing Java Signature and EnclosingMethod attributes.
  • Added support for mapping Java varargs methods onto .NET vararg methods (and v.v.).

Changes (“generics” build target only):

  • Implemented support for 1.5 generic reflection methods.

  • Added support to ikvmstub for roundtripping (Java) generics, enums, annotations, varargs and synthetic marker.

Saturday, 01 October 2005 13:27:27 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Sunday, 11 September 2005
IKVM 0.20 Released

I released 0.20 to SourceForge.

Sunday, 11 September 2005 16:39:33 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Wednesday, 07 September 2005
IKVM 0.20 rc1

A new release candidate based on yesterday's GNU Classpath 0.18 release. Unless any showstoppers are found, these bits will become the official 0.20 release in a few days.

Update japi results are available here.

Changes:

  • Integrated GNU Classpath 0.18
  • Fixed (previously unused) CountingILGenerator.BeginFaultBlock().
  • Added reusing temporary locals to compiler (to reduce file size).
  • Added optimization to compiler to remove recursive exception handlers that make synchronized block exit async safe (see http://weblog.ikvm.net/PermaLink.aspx?guid=3af9548e-4905-4557-8809-65a205ce2cd6).
  • Added optimization to compiler to turn synchronized block exit exception handlers into fault blocks.
  • Interned all member names and signatures (to save memory).
  • Interned type names.
  • Made some minor optimizations to class file parsing.
  • Stopped "normalizing" tableswitch instruction to lookupswitch (since the compiler treated them differently already).
  • Corrected "cannot throw" bytecode metadata flag for ifnonnull instruction.
  • Changed ByteCodeMetaData.CanThrowException() to take NormalizedByteCode.
  • Removed non-normalized bytecode from Instruction structure.
  • Added support for throwing NoClassDefFoundError when accessing a type that previously failed static initialization.
  • Changed ClassLoader.findLoadedClass to also find array classes made of classes that are already loaded (for compatibility with Sun JDK 1.4).
  • Changed ClassLoader <-> ClassLoaderWrapper association from hashtable to using the vmdata field in ClassLoader.
  • Rewrote large parts of class caching by the class loaders to be more spec compliant and more compatible with the Sun implementation for unspecified behavior.
  • Added class and baseclass name to VerifyError message when a class extends a final class.
  • Removed broken mechanism that tried to doom entire classes when a method was unverifiable, instead of throwing the VerifyError when the method is actually called.
  • Fixed bug that caused critical error when compiling a class that contains a package private method that overrides a protected method in a base class.
  • Added more info to some of the loader constraints violated exception messages.
  • Fixed ikvmc to continue compilation when encountering a class that cannot be loaded (due to inaccessible or final base classes).
  • Fixed various bugs in FileChannel.lock()/tryLock().
  • Fixed a verifier bug.
  • Added error message when a typename occurs in multiple referenced assemblies.
  • Fixed VMFile.mkdir NullPointerException when trying to create a root directory.
  • Added ClassFile.GetClassName() to be able to sniff class name from class definition without fully parsing the class definition.
  • Removed ability to parse java.lang.Object from ClassFile.
  • Removed OuterClass property from ClassFile.
  • Moved accidental Finish trigger detection from JavaTypeImpl.Finish to ClassLoaderWrapper.OnTypeResolve, to facilitate new "finish as we go" strategy used by ikvmc.
  • Fixed a couple of places in FieldWrappers where FieldInfo.FieldType was used (which isn't available anymore after we bash the contents of the FieldBuilders to save memory).
  • Enabled Reflection.Emit private field bashing hack to conserve memory for static compilation as well.
  • Introduced IKVM_DISABLE_TYPEBUILDER_HACK environment variable to disable the hack that bashes Reflection.Emit private fields to conserve memory.
  • Changed the way inner classes are resolved by ikvmc to facilitate new "finish as we go" strategy used by ikvmc.
  • Fixed subtle miranda method bug when dealing with loader constraint violations.
  • Fixed method generator to set MethodAttributes.NewSlot when a mangled method name is used to override a method.
  • Introduced JVM.FinishingForDebugSave to save memory, simplify code and fix a theoretical bug.
  • Moved bytecode error handling from compiler to verifier, so that the verifier can now accurately track what code is reachable (bytecode that gets compiled as an exception (e.g. NoSuchMethodError) typically results in unreachable code following that instruction). This change allows "broken" constructors to be compiled into verifiable CIL.
  • Moved array cloning hack (to workaround a bug javac) from compiler to verifier.
  • Fixed some unloadable type bugs.
  • Restructed verifier into two separate steps. The first step computes the stack and local variables types and the second step does the actual verification and reachability determination.
  • Changed jniproxy.dll (created with -Xsave) from module to assembly to make the main assembly verifiable.
  • Fixed stack walking to skip HideFromJava methods.
  • Fixed bug 1257044.
  • Moved Name and Signature to MemberWrapper.
  • Renamed MirandaMethodAttribute to HideFromReflectionAttribute.
  • Fixed ikvmc to only generate __Fields inner classes for public interfaces.
  • Added IKVM.Runtime.Util.GetFriendlyClassFromType() method. This is dumb name and subject to change.
  • Added (limited) support to remapper for adding methods to Java classes.
  • Added implicit conversion operator to java.lang.Class to convert from System.Type.
  • Optimized line number table encoding yet again.
  • Fixed bytecode compile to trigger class initializer for "new" bytecode and for accessing constant fields.
  • Changed class loading to reduce chance of inadvertantly hiding exceptions caused by bugs in the IKVM runtime.
  • Improved replacing of ClassNotFoundException with NoClassDefFoundError during class loading.
  • Made delegate helper interfaces private. This solves an incompatibility between ikvmc generated assemblies and C++/CLI, but ultimately this wasn't the reason for this change, the helper interfaces don't have type identity, so they shouldn't be used in public interfaces (preferably they should only be used in delegate instantiation).
  • Fixed class loading bugs that caused Class.forName("cli.java.lang.Object") and Class.forName("System.EventHandler$Method") to work.
  • Fixed ikvmc to no longer generate unused delegate helper interfaces.
  • Changed java.version system property to 1.4.2.
  • Fixed ikvmc to open input files as read-only.
  • Changed JNI methods GetPrimitiveArrayCritical/ReleasePrimitiveArrayCritical to allow up to two arrays to be pinned instead of copied.
  • Fixed compiler to mark all methods with MethodAttributes.HideBySig, to use correct method name resolution semantics when using an ikvmc generated assembly from C#.
  • Removed "ikvm.cleanstacktrace" system property and introduced IKVM_DISABLE_STACKTRACE_CLEANING environment variable to solve initialization order problem.

Files are available here: ikvm-0.20.0.0.zip (sources + binaries), ikvmbin-0.20.0.0.zip (binaries)

Wednesday, 07 September 2005 09:46:47 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Tuesday, 06 September 2005
Mono users meeting at the Microsoft PDC
For some reason, this year's Mono BoF proposal was again not accepted. Fortunately, Miguel organized an alternative get together. If you're going to the PDC and you're interested in running your .NET applications on non-Microsoft platforms or simply interested in the state of the leading Open Source CLI implementation, this is a must attend event.
Tuesday, 06 September 2005 15:22:39 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Monday, 15 August 2005
Revenge of the Non-Public Base Class

I think I've previously written about the stupidity of allowing public classes to extend non-public classes and implement non-public interfaces, but unfortunately in Java we have to live with this.

Last week rnaylor filed a bug that ikvmc created invalid code when calling public methods inherited from a non-public base class from another class (in another package).

A good example that is (sort of) equivalent to the problem in the bug report are the fields in the java.util.zip.ZipConstants interface. I'd link to the Javadoc for this interface, but it is a private type so there is no public documentation. However, this type still leaves it's marks on the public API, because several of the zip classes implement this interface. For example, ZipFile implements this interface and as a consequence it inherits a whole bunch of constants (i.e. public static final fields) that, as far as I can tell, serve no purpose whatsoever in the public API.

Before I checked in my fix yesterday, ikvmc didn't do anything special with these inherited fields or methods and because of this if you referenced these fields or methods from another assembly ikvmc would generate invalid code to access these members on the private base class and the .NET runtime would complain about this (by throwing System.MethodAccessException or System.FieldAccessException).

The fix was quite involved (the diff was about a thousand lines), because stub methods and field accessors need to be added to any public class that extends a non-public class or implements a non-public interface, however these stubs should not be visible to Java reflection, because that could potentially change the semantics of the code (e.g. for things like calculating the serial version UID). Additionally, while most fields should be exposed as properties (that read/write the field in the base class), constant fields need to be copied to retain their "constantness" when accessing them from another language (e.g. so that you can you say case ZipFile.CENATT: in C#). Another non-obvious consequence of these stub methods is that the stack walking code (in the security system and the code that generates the stack traces) needs to filter out these methods, because the system should function as if these methods weren't there.

The lesson here is that you have to be very careful when designing a class library in Java. I believe that the fields that the ZipConstants interface exposes on ZipFile, ZipEntry, etc. were actually an accident that the Sun developers failed to spot before shipping the original JDK. The general advice should be, don't have public classes extend non-public base classes and don't implement non-public interfaces on any of your public classes. Or at least have some tools in place that check for these inherited public members, to make you aware of them before shipping your library.

Finally, this problem doesn't occur in C#, because C# doesn't allow you to create a public class that extends a non-public base class. It does allow implementing non-public interfaces, but since interfaces can't have fields in C# that isn't a problem. An amusing note is that when you call an inherited public method (in a non-public base class) in an assembly that was generated with the broken version of ikvmc, the C# compiler will happily compile the call for you and the generated (invalid) code will again fail at runtime with a System.MethodAccessException.

Monday, 15 August 2005 19:09:27 (W. Europe Daylight Time, UTC+02:00)  #    Comments [3]