# Monday, 21 May 2007
Case Studies

A while ago I gave a talk about IKVM at WL | Delft Hydraulics and I convinced them to describe how they use IKVM, for a case study on the IKVM website. My thanks to Fedor and David for taking the time to do this.

If you or your company uses IKVM and would like to write about it for publication on the IKVM website, I would appreciate it very much. Please contact me by e-mail at jeroen@frijters.net.

Monday, 21 May 2007 17:41:19 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Friday, 11 May 2007
Merging the First OpenJDK Code and Another x64 CLR Bug

While working on integrating Sun's recently GPLed code to parse and toString floats and doubles (so that this long standing IKVM bug can finally be fixed) I ran into another x64 CLR bug, this time in the JIT. It was a little bit disturbing to see a Double.parseDouble() test fail with the Sun code that previously worked with my lame and hacked up parsing code, but fortunately the workaround was easy. The IKVM bytecode compiler now obfuscates any -0.0 literals it encounters to prevent the x64 JIT from "optimizing" them.

On a more general note, I haven't yet figured out how to go about integrating the whole of OpenJDK, but I have discovered it'll be quite a bit of work. For every class that uses native code to interface with the VM I'll have to decide whether to implement the existing native methods in C#, implement them in map.xml or rewrite the class to add IKVM specific code. Each option has different trade-offs, for example while it sucks to fork a class due to the added work of merging in changes, adding the required IKVM specific modifications inline is often the cleanest and most efficient way of implementing the required functionality.

I think I'm going to dip my toe into the water by changing my java.lang.reflect.* classes, which are currently modified versions of the GNU Classpath classes, into versions derived from the Sun code.

One final note. The OpenJDK is supposedly licensed under the GPL + "Classpath" exception, but the license text says that only files that explicitly opt-in to the "Classpath" exception are covered by the exception. That implies that I'll have to examine each source file to make sure it is covered, before including it with IKVM.

Friday, 11 May 2007 16:17:15 (W. Europe Daylight Time, UTC+02:00)  #    Comments [3]
# Wednesday, 09 May 2007
Custom Attribute Annotations

Custom attributes are a powerful capability of the .NET Framework and unlike Java annotations, there are quite a few custom attributes that are understood directly by the CLR runtime engine, for example things like P/Invoke, declarative Code Access Security and declaring thread or context local static variables can all be done by applying custom attributes. In addition to this there are also lots of reflection based applications of custom attributes because they've been part of the platform since day one.

Because of the importance of custom attributes, it makes sense to try to support them in ikvm. It has been possible for quite a while to apply custom attributes to Java classes, methods and fields by specifying them in the map.xml file and previously I used this when compiling IKVM.GNU.Classpath.dll in a few places (e.g. for declaring a thread local static variable), but given the similarity between custom attributes and Java annotations (which, like many other Java 5 features, were obviously "inspired" by .NET) it only makes sense to try to map custom attributes to annotations whenever possible.

I started working on this a while ago and recently checked in major improvements to this. The feature is now started to take shape nicely.

Limitations

Although conceptually very similar, there are significant differences between .NET custom attributes and Java annotations. The most important one is that custom attributes are classes while annotations are interfaces, as a direct result of that custom attributes use a constructor invocation syntax (plus the ability to set fields and properties), while annotations just name the type and list a number of key/value pairs (very similar to custom attribute properties). This means that in .NET, a custom attribute can specify which parameters are required and which are optional, by making the required parameters constructor arguments and the optional ones properties (or fields) and additionally, because of constructor overloading multiple combinations are possible. Annotation can specify for each individual property if it is required or optional, but there is no mechanism similar to constructor overloading.

This leads to the first limitation: Currently only custom attributes that have a default constructor or a single one-argument constructor (taking a supported type) are supported.

The second limitation is that not all parameter types are supported, only the Java primitive types (where Java's byte maps to System.Byte), type literals, strings and enumerations are supported (and single dimensional arrays of these types). Custom attribute parameters typed as Object are not supported.

Nested Annotation Type

When viewed from Java, every supported custom attribute class will appear to have a nested annotation type named Annotation. Here's an example of applying a simple custom attribute annotation to a field:

public class Foo
{
  @cli.System.ThreadStaticAttribute.Annotation
  private static int threadLocalCounter;
}

I toyed with the idea of naming the annotation by chopping of the Attribute part, to make it consistent with common C# usage, but I ultimately I decided the nested type approach was better. It's a bit more verbose, but it's consistent with how delegates are handled and it greatly reduces the chance of name clashes.

Single Argument Constructor

If a custom attribute has a single argument constructor, that will be translated into an annotation property named value. This allows the simplified syntax to be used:

@cli.System.CLSCompliantAttribute.Annotation(true)
public class Foo { ... }

If the custom attribute also has a default constructor, the value property will be optional.

Enums

Since .NET enums are value types and behave like integral types most of the time, they are not mapped to Java enums, but for use in annotations they have to be mapped to Java enums, so every .NET enum type has a nested type __Enum that is a Java enum. This allows enum values to be used in annotations:

import cli.System.*;

@AttributeUsageAttribute.Annotation({
  AttributeTargets.__Enum.Class,
  AttributeTargets.__Enum.Interface
})
public class MyCustomAttribute extends Attribute
{ ... }

The above example also shows that an enum that has a FlagsAttribute (as AttributeTargets does) will be translated into an array of that enum, to allow combining the flags. The exception to this is that if the parameter type is already an array, it will not be turned into a two dimensional array, because annotations only support one dimensional arrays.

ValidOn & AllowMultiple

Custom attribute annotations are annotated with the Target and Retention annotations. The target element types are based on the ValidOn property of the AttributeUsageAttribute of the custom attribute and Retention always has a RetentionPolicy.RUNTIME.

The ValidOn property of AttributeUsageAttribute is of type AttributeTargets. Here's a table that shows how the values map to the Java ElementType enum:

System.AttributeTargets java.lang.annotation.ElementType
Assembly, Class, Delegate, Enum, Interface, Struct     TYPE
Constructor CONSTRUCTOR
Method METHOD
Field FIELD
Parameter PARAMETER

Unlike custom attributes, annotations cannot be applied to a method's return value. The workaround for that is to nest them inside a special __ReturnValue annotation that is applied to the method:

public class Foo
{
  @cli.System.CLSCompliantAttribute.Annotation.__ReturnValue(
     @cli.System.CLSCompliantAttribute.Annotation(true)
  )
  public int foo() { ... }
}

Another custom attribute feature not supported by annotations is the ability to specify multiple instances of a custom attributes (controlled by the AllowMultiple property of AttributeUsageAttribute). This is handled in a similar way:

import cli.System.Security.Permissions.*;

@UIPermissionAttribute.Annotation.__Multiple({
  @UIPermissionAttribute.Annotation(value = SecurityAction.__Enum.Demand,
    Window = UIPermissionWindow.__Enum.SafeTopLevelWindows),
  @UIPermissionAttribute.Annotation(value = SecurityAction.__Enum.LinkDemand,
    Clipboard = UIPermissionClipboard.__Enum.AllClipboard)
})
public class Foo { ... }

Note that __Multiple is never nested inside a __ReturnValue, when a custom attribute has AllowMultiple and can be applied to return values, the __ReturnValue annotation simply has an array value property.

Type Literals

Type literals can be specified as class literals:

import cli.System.Runtime.CompilerServices.*;

@TypeForwardedToAttribute(Foo.class)
interface assembly {}

Note that the assembly type in the default package is treated specially by ikvm and acts as a placeholder for custom attributes that should be applied to the assembly.

Default Values

This table lists the default values for each supported type:

Type Default Value
boolean      false
byte (byte)0
char (char)0
short (short)0
int 0
float 0.0F
long 0L
double 0.0D
String ""
Class ikvm.internal.__unspecified.class
Enum __unspecified
array { }

The __unspecified hacks are necessary because annotation property values cannot be null. Every __Enum has an __unspecified field specifically for this purpose. Note that these defaults do not necessarily correspond to the real default values used by the custom attribute. These are simply the values that are reported by Method.getDefaultValue() and put into the stubs generated by ikvmstub. If you don't specify a value for an optional property, the resulting custom attribute won't have the property set and hence it will have the default value as defined by the implementation of the custom attribute.

Error Handling

Not yet implemented, but the idea is to ignore invalid annotations (most likely due to version conflicts between the stub jar that the Java code was compiled against and the assembly used at runtime). In the case of ikvmc, a warning message should be emitted as well.

Wednesday, 09 May 2007 08:15:26 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
New Snapshot

I expect that I'll be busy the next couple of weeks working on integrating the OpenJDK libraries that were released yesterday, so I thought I'd release a new development snapshot to reflect the current state. There are quite a few bug fixes, most of them triggered by running test suites suggested by Albert Strasheim. Many thanks to him for running the test suites and passing the interesting ones on to me (including instructions on how to install/build/run them, which is usually the most time consuming and frustrating part.)

This snapshot also includes much improved support for applying .NET custom attributes as Java annotations. The next blog entry will describe the changes in detail.

Changes:

  • Integrated current GNU Classpath cvs version.
  • Fixed exception handling to continue working during AppDomain finalization for unload.
  • Implement major chunk of custom attribute as annotations support.
  • Added support for applying custom attributes to return values.
  • Added support for applying AllowMultiple custom attributes multiple times to the same element.
  • Restructured ParameterBuilder handling.
  • Added system property "ikvm.apartmentstate" to enable setting the COM ApartmentState for threads created in Java code.
  • Added hack to support Double.MIN_VALUE and Double.MAX_VALUE toString/parse roundtripping.
  • Fixed Throwable.printStackTrace() to call Throwable.printStackTrace(OutputStream) to support exception classes that only override printStackTrace(OutputStream).
  • Fixed Throwable.printStackTrace(…) to use PrintWriter/PrintStream.println() to trigger flushing on auto-flush writers/streams.
  • Fixed Throwable constructor to set cause correctly if an exception was instantiated but not thrown immediately.
  • Moved ikvmc warning handling to fix missing warnings bug (previously depending on compilation order, some warnings might not be shown).
  • Fixed verifier/compiler to support dup_x2 form 2. Found by Derby test suite.
  • Implemented JSR 133 rule that says that finalize cannot run before constructor is finished.
  • Fixed ArrayIndexOutOfBoundsException in generic metadata reflection on dynamically compiled types.
  • Fixed PlainDatagramSocketImpl to set initial SO_BROADCAST option to enabled.
  • Fixed PlainSocketImpl.getOption(SO_OOBLINE) to return Boolean instead of Integer.
  • Fixed DatagramChannelImpl to reflect connectedness on underlying DatagramSocket.
  • Fixed SocketChannelImpl.read(ByteBuffer) to handle -1 return values from implRead() for non byte array backed buffers.

Binaries available here: ikvmbin-0.35.2685.zip

Wednesday, 09 May 2007 08:13:44 (W. Europe Daylight Time, UTC+02:00)  #    Comments [3]
# Saturday, 05 May 2007
IKVM 0.34 Released

I released 0.34.0.2 to SourceForge (same bits as rc3).

Saturday, 05 May 2007 09:35:10 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Wednesday, 02 May 2007
More Good Tests

Albert Strasheim mailed me the suggestion to run the Apache Commons Lang test suite (BTW, Albert was also the one who pointed me to the JRuby and MTJ test suites).

That also resulted in a number of bugs fixed/filed:

NestableErrorTestCase.testPrintStackTrace failure         Various IKVM java.lang.Throwable fixes
CharSetTest.testSerialization failure Mono reflection bug
NumberUtilsTest.testCompareDouble failure Added hack to IKVM's Double.toString() and Double.parseDouble() to support Double.MIN_VALUE and Double.MAX_VALUE
NumberUtilsTest.testIsNumber failure GNU Classpath BigDecimal bug
DateUtilsTest.testAddYears failure GNU Classpath GregorianCalendar bug

This list doesn't explain all failures, some are caused by the same issues as the ones mentioned and I think there are a few more GNU Classpath calendar/date/time etc. related bugs and a few tests are not quite "correct". For example, the ToStringBuilderTest.testReflectionHierarchyArrayList test assumes that the private member that contains the array of java.util.ArrayList is named elementData, but in GNU Classpath's implementation it is named data.

Wednesday, 02 May 2007 08:32:19 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
Groovy
Wednesday, 02 May 2007 06:47:19 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Sunday, 29 April 2007
IKVM 0.34 rc3

While working on NIO pipe support in the development branch, I found a few socket bugs and I decided to back port them to the 0.34 release. I've also back ported the NIO pipe support itself, since that's a low risk change.

Changes since rc2:

  • Implemented NIO pipe support.
  • Added support for socket connect with timeout.
  • Fixed Socket.bind() to set local port after binding.
  • Fixed SocketChannel.read() to return -1 if other side closed the socket.
  • Fixed ServerSocketChannel.accept() to properly set the state in the returned socket, to fix the socket from breaking if you called certain Socket methods on the SocketChannel.socket(). This was a regression introduced during the 0.33 development.

Files are available here: ikvm-0.34.0.2.zip (source + binaries) and ikvmbin-0.34.0.2.zip (binaries).

Sunday, 29 April 2007 11:01:44 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Wednesday, 25 April 2007
IKVM 0.34 rc2

A new release candidate that fixes a regression in String.lastIndexOf(String,int) that I introduced in rc1.

Files are available here: ikvm-0.34.0.1.zip (source + binaries) and ikvmbin-0.34.0.1.zip (binaries).

Also a clarification: As of this release there is no longer a separate “generics” build, because the 1.5 specific stuff has been merged into the GNU Classpath main release.

Wednesday, 25 April 2007 05:52:45 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Monday, 23 April 2007
IKVM 0.34 rc1

GNU Classpath 0.95 has been released! Here's the corresponding IKVM release candidate. I've updated the Japi status page.

Changes since previous snapshot:

  • Integrated GNU Classpath 0.95 release.
  • Fixed ByteBuffer.allocateDirect() to zero initialize the memory it allocated.
  • Fixed various String methods (indexOf, lastIndexOf, startsWith, endsWith, contains, replace) to use ordinal semantics instead of culture dependent word matching. Thanks to Louis Boydstun for tracking this bug down.
  • Fixed potential deadlock when a dying thread is interrupted.

Files are available here: ikvm-0.34.0.0.zip (source + binaries) and ikvmbin-0.34.0.0.zip (binaries).

As usual these are the strong named binaries and if no major issues are found these files will be released and placed on the SourceForge download page.

Monday, 23 April 2007 17:14:08 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]