# Wednesday, 19 November 2008
Multi Target Support

I've now enabled "multi target" support. This means that you can compile multiple assemblies at once. One of the advantages this has is that it makes dealing with circular dependencies much easier, but even if there aren't any circular dependencies, it can make building much easier because you don't have to do any dependency analysis.

As an example I compiled the jars from the JBoss lib directory:

ikvmc -target:library { commons-codec.jar } { commons-httpclient.jar } { commons-logging.jar } { concurrent.jar } { getopt.jar } { jboss-common.jar } { jboss-jmx.jar } { jboss-system.jar } { jboss-xml-binding.jar } { log4j-boot.jar }

The curly braces define the targets. This is all it takes to compile these jars into seperate assemblies that automatically reference eachother, as required.

Using NDepend (which Patrick Smacchia kindly gave me a free copy of) you can easily graph the dependencies of the resulting assemblies:

In this case there aren't any cycles, but had there been, it would have made no differences.

The multi target feature is by no means done. I want to add a -sharedclassloader option to enable multiple assemblies to share a single Java class loader. I'm also considering some options to select which classes/packages go in which assembly. This would be helpful for splitting the core class library into multiple assemblies.

Changes:

  • Added support to ikvmstub for automatically exporting non-vector array types.
  • Implemented/fixed support for pointers, by ref and non-vector arrays in IKVM.Reflection.Emit.
  • Fixed pointer detection to work for types with multiple indirection levels.
  • Changed proxy stub name mangling to work around ildasm bug.
  • Fixed IKVM.Reflection.Emit to not write token of FAULT pseudo exception that is used as a marker for fault blocks.
  • Fixed class loading to take ClassLoader lock.
  • Changed ikvmc not to generate warning when it loads ikvmstub generated classes.
  • Removed our version of System.Runtime.CompilerServices.ExtensionAttribute and instead add System.Core.jar to the compilation, this will allow us to reference the real ExtensionAttribute when it is available and yet the build will still work (albeit with a warning and without the ExtensionAttribute) when it is not available (i.e. when building on .NET 2.0).
  • Did some clean up and restructuring in IKVM.Reflection.Emit.
  • Fixed sorting of InterfaceImpl table in IKVM.Reflection.Emit.
  • Removed hard coded public keys from JniInterface.cs and Configuration.java.
  • Changed build process so that version number for all assemblies only has to be specified in CommonAssemblyInfo.cs.
  • Enabled multi target support.
  • Fixed typo in IKVM.Reflection.Emit that caused assembly minor version to be set to major version.
  • Added assembly version to ikvmstub.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3245.zip

Wednesday, 19 November 2008 07:46:06 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Monday, 17 November 2008
.NET Array Weirdness

One of the lesser known features of the .NET runtime is the support for multidimensional and/or non-zero-based arrays. The typical arrays that you use (and that are the same as the arrays in Java) are called vectors in the CLI specification, but unfortunately the non-vector arrays don't have a specific name (they are simply called arrays in the CLI specification).

In C# you can easily create a non-vector array:

int[,] arr1 = new int[4, 4]

This creates a two dimensional array of integers with 16 elements. This array is zero based. If you want to create a non-zero-based array, you have to use the API, because C# doesn't directly support that:

int[,] arr2 = (int[,])Array.CreateInstance(typeof(int), new[] { 4, 4 }, new[] { 1, 1 });

This also creates a two dimensional array of integers with 16 elements, but in this case the indexes run from 1 through 5.

So far, so good. Now let's look at how things look at the IL level. If you use ildasm to look at the first C# based instantation you'll see:

ldc.i4.1
ldc.i4.1
newobj instance void int32[0...,0...]::.ctor(int32, int32)

Most of this looks straightforward if you're familiar with IL, except for the [0...,0...] part. It looks like the lower bounds are part of the type, but a simple experiment shows that this is not the case:

Console.WriteLine(arr1.GetType() == arr2.GetType());

This prints out True. This implies that the lower bounds are not part of the type (and the CLI specification confirms this). So why are they part of the signature? I don't know for sure, but it does have an interesting consequence. You can overload methods based on this:

.assembly extern mscorlib { }
.assembly Test { }
.module Test.exe

.class public Program
{
  .method public static void Foo(int32[0...,0...] foo)
  {
    ret
  }

  .method public static void Foo(int32[1...,1...] foo)
  {
    ret
  }

  .method private static void Main()
  {
    .entrypoint
    ldc.i4.1
    ldc.i4.1
    newobj instance void int32[0...,0...]::.ctor(int32, int32)
    call void Program::Foo(int32[0...,0...])
    ret
  }
}

This is a valid and verifiable application. Unfortunately, using the  .NET reflection API it is impossible to see the difference in signatures between the two Foo methods.

This limitation in reflection means that code compiled with the new IKVM.Reflection.Emit based ikvmc won't be able to call methods that have non-zero-based array parameters. It is also impossible to override these methods, but that was already the case with the previous System.Reflection.Emit based implementation as well.

Finally, in the above text I talk about the lower bounds, but the same thing applies to the upper bounds of the array.

Monday, 17 November 2008 07:08:46 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Friday, 14 November 2008
Introducing IKVM.Reflection.Emit

Over the past two months I've been working on reimplementing a large portion of the Reflection.Emit API from scratch. After finally growing tired of the System.Reflection.Emit bugs and limitations and not finding Mono.Cecil satisfactory either, I decided to build my own implementation.

I started out with these design goals:

  • API compatible with System.Reflection.Emit (as much as possible).
  • Implement only the Emit part and be compatible with System.Reflection.
  • Only implement functionality required by ikvmc, but not implemented functionality shouldn't silently fail (i.e. it should throw a NotSupportedException or NotImplementedException). This also means that the API is mostly write-only.
  • Efficient implementation, optimized for ikvmc specific scenarios.

I think I've met or exceeded all of the design goals. Without doing any significant performance work on my Ref.Emit implementation (other than the design), ikvmc became so much faster that it is rather emberassing for the Microsoft System.Reflection.Emit implementation.

I've only had to make a couple of minor changes to the ikvmc sources (apart from changing using System.Reflection.Emit; to using IKVM.Reflection.Emit; in every file) to account for the fact that IKVM.Reflection.Emit.ModuleBuilder and IKVM.Reflection.Emit.AssemblyBuilder unfortunately cannot extend System.Reflection.Module and System.Reflection.Assembly. However, it looks like this is fixed in the .NET 4.0 CTP.

Here are some random statistics about compiling IKVM.OpenJDK.ClassLibrary.dll on .NET 2.0 SP2 x64:

  System.Reflection.Emit     IKVM.Reflection.Emit  
File size 31,645,696 30,480,896   bytes
CPU time used 272 35   seconds
Peak virtual memory 1,433,399,296 1,035,018,240   bytes
Generation 0 GCs 770 896  
Generation 1 GCs 201 240  
Generation 2 GCs 11 8  

(The huge memory usage is not because it actually needs that much memory, but simply the result of the fact that garbage collection is more efficient if you have more memory available and that my system had about 1.5GB of free memory while running these tests.)

The smaller file size is because System.Reflection.Emit always uses fat method headers and IKVM.Reflection.Emit uses tiny method headers whenever possible.

There is still some work left to do, I've only spent limited time on debugging support and there is no support for Mono's .mdb format yet. I also haven't done any testing on Mono yet.

BTW, thanks to Sebastien Pouliot for code I lifted from Mono to parse strong name CAPI key blobs.

Other changes in this snapshot:

  • Dropped support for Visual Studio 2005.
  • Added error message when map.xml references non-existing constructor.
  • Added more statistics to ikvmc -time option output.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3240.zip

Friday, 14 November 2008 10:01:50 (W. Europe Standard Time, UTC+01:00)  #    Comments [2]
# Wednesday, 12 November 2008
IKVM 0.38 Release Candidate 2

A codegen bug was found (not a regression, so there will be a 0.36 update as well) so here's another release candidate.

Changes:

  • Changed version to 0.38.0.2.
  • Fixed openjdk.build BOM issue on Linux.
  • Fixed jsr verifier bug that caused incorrect codegen under very specific circumstances (thanks to Brian Schwallier for tracking down a repro).

Binaries available here: ikvmbin-0.38.0.2.zip
Sources (+ binaries): Sources: ikvm-0.38.0.2.zip, classpath-0.95-stripped.zip, openjdk6-b12-stripped.zip

Wednesday, 12 November 2008 06:13:29 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Wednesday, 05 November 2008
IKVM 0.36 Update 2 Release Candidate 2

As announced when 0.36 was released, I will periodically release updates to 0.36 as long as there is enough interest in .NET 1.1 support.

This is the second release candidate of the second update.

Changes (all are back ported fixes):

  • Changed version to 0.36.0.13.
  • Fixed ikvmc not to open the key file for write access.
  • Added more efficient float/double to/from int/long bits converters.
  • Fixed libikvm-native.so build to include reference to gmodule-2.0 library.
  • Fixed ikvmc not to open the key file for write access.
  • Fixed Graphics2D.rotate() to convert rotation angle from radians (Java) to degrees (.NET).
  • Applied awt patch #1979656 by Daniel Wilson.
  • Fixed three String bugs found by OpenJDK string tests.
  • Fixed ldc <class> where <class> is a ghost array.
  • Fixed bug in instanceof <class> where <class> is a Serializable[].
  • Removed incorrect DataFormatException thrown in java.util.zip.InflaterHuffmanTree.
  • Fixed #2001802 contributed by Andy Malakov.
  • Fixed #2001799.
  • Fixed #2006953.
  • Made finalize() and clone() methods in cli.System.Object and cli.System.Exception final.

Binaries available here: ikvmbin-0.36.0.13.zip
Sources (+ binaries): ikvm-0.36.0.13.zip

Wednesday, 05 November 2008 10:08:46 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Tuesday, 04 November 2008
IKVM 0.38 Release Candidate 1

A couple of minor fixes.

Changes since RC0:

  • Updated version to 0.38.0.1
  • Hide nested types generated by callerID support
  • Made finalize() and clone() methods in cli.System.Object and cli.System.Exception final
  • Fixed reflection to work on .NET 2.0 RTM

Note that even though I fixed reflection to work on .NET 2.0 RTM, it still isn't a supported platform, I strongly recommend .NET 2.0 SP1 or higher.

Binaries available here: ikvmbin-0.38.0.1.zip

Sources: ikvm-0.38.0.1.zip, classpath-0.95-stripped.zip, openjdk6-b12-stripped.zip

Tuesday, 04 November 2008 06:36:51 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Friday, 26 September 2008
More Reflection.Emit Brokenness

Pardon me if I sound a little bitter today, but I just wasted almost a full day trying to work around this bug only to be stopped by yet another bug that makes it impossible to generate two mutually dependent assemblies with Reflection.Emit.

Also, while debugging I noticed another mind bogglingly stupid bug in the System.Reflection.Assembly source:

public override int GetHashCode() { return base.GetHashCode(); }

Why would you want to do that? Oh, of course! It's to get rid of a compiler warning... If you override Equals() you should also override GetHashCode(), BUT NOT LIKE THIS.

Here's small program that demonstrates the problem:

using System;
using System.Reflection;
using System.Reflection.Emit;

class Program
{
  static void Main()
  {
    AssemblyBuilder ab1 = AppDomain.CurrentDomain.DefineDynamicAssembly(new AssemblyName("A"),
                            AssemblyBuilderAccess.Run);
    ModuleBuilder mod1 = ab1.DefineDynamicModule("A.dll");
    TypeBuilder tb1 = mod1.DefineType("T");
    Type type = tb1.CreateType();
    Console.WriteLine(ab1.Equals(type.Assembly));
    Console.WriteLine(ab1.GetHashCode() == type.Assembly.GetHashCode());
  }
}

This prints out:

True
False

That clearly violates the Object.GetHashCode() contract.

Friday, 26 September 2008 08:47:51 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Wednesday, 24 September 2008
IKVM 0.38 Release Candidate 0

As with previous release candidates, this release includes strong named binaries and is considered to be (nearly) ready for production use. Please test this version and give feedback as soon as possible.

Changes since previous snapshot:

  • Changed version to 0.38.0.0 and strong named binaries.
  • Added missing HTMLEntities.res resource.
  • Re-introduced workaround for .NET JIT bug that causes .cctor not to run when a DynamicMethod invokes a method or gets/sets a field.

Binaries available here: ikvmbin-0.38.0.0.zip

Sources: ikvm-0.38.0.0.zip, classpath-0.95-stripped.zip, openjdk6-b12-stripped.zip

Wednesday, 24 September 2008 08:07:13 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Monday, 22 September 2008
Running JaC64

JaC64 is a open source Commodore 64 emulator written in Java. I have many fond childhood memories of my C64, so I spent a little time fixing a couple of AWT issues and hacking together some sound support for ikvm. The sound patch is here, but it won't go in because it is essentially hard coded for JaC64 and even then it doesn't really work, because it turns out that .NET has no decent sound API. The only API available is SoundPlayer, but it has an unacceptable latency (and can only play one sample at a time, so you can't hide the latency). JaC64 generates samples that are 0.25 seconds long and then plays these back to back. This means that you hear sound, but it is very choppy.

Two obligatory screen shots, first the emulator application and second just the C64 screen of my favorite game:

Monday, 22 September 2008 07:19:53 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
New Development Snapshot

This is the final development snapshot before the first 0.38 release candidate.

Changes since previous development snapshot:

  • Updated to OpenJDK 6 b12.
  • Updated IKVM.OpenJDK.ClassLibrary.dll copyright notices.
  • Removed hardcoded PublicKey from build process.
  • Fixed ikvmc regression that caused using .NET generic types not to work.
  • Added support to ikvmc for recognizing "access" bridge methods, so that they aren't hidden from other .NET code.
  • Removed warnings from IKVM.OpenJDK.ClassLibrary ikvmc build step.

WARNING: THIS IS A DEVELOPMENT SNAPSHOT, NOT AN OFFICIAL RELEASE.

Development snapshots are intended for evaluating and keeping track of where the project is going, not for production usage. The binaries have not been extensively tested and are not strong named.

This version supports .NET 2.0 SP1 and later. The binaries will run on Mono 2.0, but building on Mono 2.0 is not supported due an open bug.

Binaries available here: ikvmbin-0.37.3187.zip

The OpenJDK 6 b12 (re)source file needed to build from source are available here: openjdk6-b12-stripped.zip

Monday, 22 September 2008 06:51:27 (W. Europe Daylight Time, UTC+02:00)  #    Comments [3]