# Friday, 29 January 2010
New Development Snapshot

I've modified ikvmc to use IKVM.Reflection and largely rewritten ikvmstub to directly work with the ikvm internals instead of using the java reflection API. Both ikvmc and ikvmstub can now process assemblies independent from the .NET runtime they run on. This opens up the possibility to start investigating the possibility of Silverlight support.

Changes:

  • Drag-n-drop fix by Nat.
  • Fixed regression introduced in previous development snapshot, related to field accessors.
  • Removed caching of inner classes.
  • Fix for bug #2908683.
  • Various AWT fixes by Volker.
  • Changed JNI to use standard caller ID mechanism.
  • Various JNI optimizations.
  • Fixed http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41696
  • Fixed exception sorter bug exposed by recent Mono sorting change.
  • Fixed Thread.getAllStackTraces() to resume threads that it suspends.
  • Integrated new IKVM.Reflection implementation.
  • Added AllowMultiple = true to RemappedClassAttribute.
  • Fixed atomic update helper nested types to be invisible from Java.
  • Removed support for "ikvm.stubgen.serialver" property that is no longer needed now that ikvmstub doesn't use the runtime to generate stubs.
  • Removed pre-generated stub jars from cvs and modified build process to generate them during the build.
  • Removed "constant" instance field support (which was only used by ikvmstub and doesn't make any sense anyway).
  • Removed ReflectionOnly support from runtime. Now that ikvmstub no longer requires it, there's no good reason to allow Java code to see ReflectionOnly types.

Binaries available here: ikvmbin-0.43.3681.zip

Friday, 29 January 2010 09:35:13 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Monday, 25 January 2010
Introducing IKVM.Reflection

In November 2008 I introduced IKVM.Reflection.Emit, today I'm introducing IKVM.Reflection. It superseded IKVM.Reflection.Emit and also includes the ability to read managed assemblies. In addition, I've also added many other features that aren't directly needed for ikvmc, but are useful for other applications. Almost the complete reflection API has now been implemented and there are several API extensions to support managed PE features that reflection doesn't support (well).

Why?

When I started on IKVM.Reflection.Emit, it wasn't at all clear to me that it would be possible to re-implement the System.Reflection.Emit namespace without also re-implementing the System.Reflection namespace, but it turned out it was. I did run into a few snags, such as the inability to subclass Module and Assembly (this was fixed in .NET 4.0) and a couple of Mono bugs, but on the whole IKVM.Reflection.Emit was very successful. So why then re-implement the System.Reflection namespace as well? The main reasons are Silverlight and .NET 4.0. For ikvmc to be able to target versions of the runtime different from the one it is currently running on, it is necessary to avoid using System.Reflection, because System.Reflection can only ever work with the mscorlib version of the current runtime.

ikvmc & ikvmstub

In the coming time, I plan on integrating IKVM.Reflection into ikvmc (most of the work for this has already been done, if you've been following the ikvm-commit list, you may have seen some changes go in and wondered why they are necessary) and ikvmstub (I haven't started on this yet). Currently, ikvmstub uses java reflection to expose members of the .NET types in an assembly. I chose this because it was the easiest way to make sure that what ikvmstub generated matched the ikvm runtime behavior (because it simply used the ikvm runtime to do the mapping). There are two downsides to this approach. The first is the same as mentioned above with ikvmc, you can only generate mscorlib stubs for the runtime you're currently running on. The second is more philosophical, it introduces a cycle in the build process. To build the IKVM.OpenJDK.*.dll assemblies, you need mscorlib.jar and System.jar, but to generate these stubs you need a compiled class library. To solve both these issues, I plan to rewrite ikvmstub to work directly on the internal ikvm runtime representations (with conditional compilation, like ikvmc does).

Features

This list is not exhaustive, but here are some interesting features of IKVM.Reflection (that are not in System.Reflection):

  • There is no AppDomain global state. Everything is contained in an instance of the IKVM.Reflection.Universe class.
  • No thread safety. If you want thread safety, you'll have to lock the universe object during every operation.
  • You can choose what version of mscorlib to load in the universe (using Universe.LoadMscorlib()) and by implementing a Universe.AssemblyResolve handler you can decide the framework assembly unification policy.
  • Support for querying and emitting .NET 2.0 style declarative security.
  • Support for defining unmanaged resources from a byte array (in .NET, AssemblyBuilder.DefineUnmanagedResources(byte[]) is broken, only the overload that accepts a filename works).
  • The ability to read and emit .NET 1.1, .NET 2.0 and .NET 4.0 assemblies (while running on, for example, .NET 2.0).
  • Full support for vararg calling convention.
  • Support for reading field RVA data (e.g. for the fields that are used by the C# compiler to initialize arrays).
  • The ability to enable/disable "exception block assistance", or get "clever" assistance.
  • Support for querying methodimpl mappings.
  • Support for reference, pointer and array types with custom modifiers (this CLR feature is used by C++/CLI).

Missing Features

Some things are still missing. The most notable being the Emit differences. The emit code was based on the IKVM.Reflection.Emit code and likewise still lacks some of the querying support (for baked types), although the new code is much better than the code in IKVM.Reflection.Emit.dll in this respect.

Here's a list of methods that can still throw a NotImplementedException:

  • FieldBuilder.__GetDataFromRVA()
  • ModuleBuilder.ResolveType()
  • ModuleBuilder.ResolveMethod()
  • ModuleBuilder.ResolveField()
  • ModuleBuilder.ResolveMember()
  • ModuleBuilder.ResolveString()
  • ModuleBuilder.__ResolveOptionalParameterTypes()
  • ModuleBuilder.GetArrayMethod()
  • GenericTypeParameterBuilder.BaseType
  • GenericTypeParameterBuilder.__GetDeclaredInterfaces()
  • GenericTypeParameterBuilder.GetGenericParameterConstraints()
  • GenericTypeParameterBuilder.GenericParameterAttributes
  • TypeBuilder.CreateType() (when invoked a second time)
  • TypeBuilder.__GetDeclaredFields()
  • TypeBuilder.__GetDeclaredEvents()
  • TypeBuilder.__GetDeclaredProperties()
  • ISymbolDocumentWriter.SetCheckSum()
  • ISymbolDocumentWriter.SetSource()
  • Most methods in ISymbolWriter
  • ManifestResourceInfo.ResourceLocation (for resources located in another assembly)
  • ManifestResourceInfo.ReferencedAssembly
  • ManifestResourceInfo.FileName
  • MethodBase.GetMethodBody() (if the method data contains unexpected sections)
  • Type.IsAssignableFrom()

Missing members:

  • Module.GetSignerCertificate()
  • Type.GUID
  • ParameterBuilder.GetToken()
  • PropertyBuilder.PropertyToken
  • MethodBuilder.Signature
  • ConstructorBuilder.Signature
  • MethodBuilder.SetSymCustomAttribute()
  • ModuleBuilder.SetSymCustomAttribute()
  • ConstructorBuilder.SetSymCustomAttribute()
  • ModuleBuilder.DefineResource()
  • AssemblyBuilder.ModuleResolve
  • Assembly.GetManifestResourceStream()
  • Assembly.GetSatelliteAssembly()
  • Assembly.GetFile()
  • Assembly.GetFiles()
  • MethodBuilder.SetMarshal() (obsolete, use MarshalAsAttribute instead)
  • ParameterBuilder.SetMarshal() (obsolete, use MarshalAsAttribute instead)
  • FieldBuilder.SetMarshal() (obsolete, use MarshalAsAttribute instead)
  • ModuleBuilder.DefineUnmanagedResource(byte[]) (because it is broken)
  • AssemblyBuilder.DefineUnmanagedResource(byte[]) (because it is broken)
  • MethodBuilder.CreateMethodBody()
  • Everything that doesn't make sense in a ReflectionOnly context.

Concepts that are not implemented:

  • Most metadata tokens returned by Emit objects are not properly typed (and can't be used for anything, other than comparing them against other metadata tokens).
  • When defining debugging symbols, a single method can only point to a single source document.
  • All type/member lookup operations are case sensitive.
  • Implementing a custom System.Reflection.Binder is not supported.
  • Modules with unsorted metadata tables are not supported.
  • When Type.GetMethods() (or __GetDeclaredMethods) is called on Array types it throws a NotImplementedException, instead of returning the special array accessor methods.
  • Managed function pointer types are not supported. Like System.Reflection, they are returned as System.IntPtr instead.

Linker Prototype

To see if I did miss any important CLR features, I wrote a prototype assembly linker. It is pretty capable, but should not be confused for something that is usable for anything other than exploring. I've used it with C++/CLI (compiled with /clr:pure) to test the more esoteric CLR features. The source for the linker prototype is in the zip linked to below.

The IKVM.Reflection source code is available in cvs. If you just want the binary, the LinkerPrototype.zip contains it.

Monday, 25 January 2010 08:28:20 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Tuesday, 12 January 2010
0.42 Update 1 RC 0

Yesterday, I released 0.42 and as seemingly always happens a bug was reported right after that. So with that we're now on the road to the 0.42 Update 1 release. This is release candidate 0.

Changes:

Binaries available here: ikvmbin-0.42.0.4.zip

Sources: ikvm-0.42.0.4.zip, openjdk6-b16-stripped.zip

Tuesday, 12 January 2010 18:11:43 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Monday, 11 January 2010
IKVM 0.42 Released
I've released IKVM 0.42 to SourceForge. The binaries are identical to the ones in release candidate 3.

Release Notes

This document lists the known issues and incompatibilities.

Runtime

  • Code unloading (aka class GC) is not supported.
  • In Java static initializers can deadlock, on .NET some threads can see uninitialized state in cases where deadlock would occur on the JVM.
  • JNI
     
    • Only supported in the default AppDomain.
    • Only the JNICALL calling convention is supported! (On Windows, HotSpot appears to also support the cdecl calling convention).
    • Cannot call string contructors on already existing string instances
    • A few limitations in Invocation API support
       
      • The Invocation API is only supported when running on .NET.
      • JNI_CreateJavaVM: init options "-verbose[:class|:gc|:jni]", "vfprintf", "exit" and "abort" are not implemented. The JDK 1.1 version of JavaVMInitArgs isn't supported.
      • JNI_GetDefaultJavaVMInitArgs not implemented
      • JNI_GetCreatedJavaVMs only returns the JavaVM if the VM was started through JNI or a JNI call that retrieves the JavaVM has already occurred.
      • DestroyJVM is only partially implemented (it waits until there are no more non-daemon Java threads and then returns JNI_ERR).
      • DetachCurrentThread doesn't release monitors held by the thread.
    • Native libraries are never unloaded (because code unloading is not supported).
  • The JVM allows any reference type to be passed where an interface reference is expected (and to store any reference type in an interface reference type field), on IKVM this results in an IncompatibleClassChangeError.
  • monitorenter / monitorexit cannot be used on unitialized this reference.
  • Floating point is not fully spec compliant.
  • A method returning a boolean that returns an integer other than 0 or 1 behaves differently (this also applies to byte/char/short and for method parameters).
  • Synchronized blocks are not async exception safe.
  • Ghost arrays don't throw ArrayStoreException when you store an object that doesn't implement the ghost interface.
  • Class loading is more eager than on the reference VM.
  • Interface implementation methods are never really final (interface can be reimplemented by .NET subclasses).
  • JSR-133 finalization spec change is not fully implemented. The JSR-133 changes dictate that an object should not be finalized unless the Object constructor has run successfully, but this isn't implemented.

Static Compiler (ikvmc)

  • Some subtle differences with ikvmc compiled code for public members inherited from non-public base classes (so called "access stubs"). Because the access stub lives in a derived class, when accessing a member in a base class, the derived cctor will be run whereas java (and ikvm) only runs the base cctor.
  • Try blocks around base class ctor invocation result in unverifiable code (no known compilers produce this type of code).
  • Try/catch blocks before base class ctor invocation result in unverifiable code (this actually happens with the Eclipse compiler when you pass a class literal to the base class ctor and compile with -target 1.4).
  • Only code compiled in a single assembly fully obeys the JLS binary compatibility rules.
  • An assembly can only contain one resource with a particular name.
  • Passing incorrect command line options to ikvmc may result in an exception rather than a proper error messages.

Class Library

Most class library code is based on OpenJDK 6 build 16. Below is a list of divergences and IKVM specific implementation notes.

com.sun.security.auth.module        Not implemented.
java.applet GNU Classpath implementation. Not implemented.
java.awt Partial System.Windows.Forms based back-end. Not supported.
java.io.Console Not implemented.
java.lang.instrument Not implemented.
java.lang.management Not implemented.
java.net No IPv6 support implemented.
java.net.ProxySelector Getting the default system proxy for a URL is not implemented.
java.text.Bidi GNU Classpath implementation. Not supported.
java.util.zip Partially based on GNU Classpath implementation.
javax.imageio.plugins.jpeg Partial implementation. JPEGs can be read and written, but there is no metadata support.
javax.management Not implemented.
javax.print Not implemented.
javax.script Not implemented.
javax.smartcardio Not implemented.
javax.sound Not implemented.
javax.swing Not supported.
javax.tools Not implemented.
org.ietfs.jgss Not implemented.
sun.jdbc.odbc Implementation based on .NET ODBC managed provider.
sun.net.www.content.audio Audio content handlers not implemented.
sun.net.www.content.image Image content handlers not implemented.

The entire public API is available, so "Not implemented." for javax.print, for example, means that the API is there but there is no back-end to provide the actual printing support. "Not supported." means that the code is there and probably works at least somewhat, but that I'm less likely to fix bugs reported in these areas.

Specific API notes:

  • java.lang.Thread.stop(Throwable t) doesn't support throwing arbitrary exceptions on other threads (only java.lang.ThreadDeath).
  • java.lang.Thread.holdsLock(Object o) causes a spurious notify on the object (this is allowed by the J2SE 5.0 spec).
  • java.lang.String.intern() strings are never garbage collected.
  • Weak/soft references and reference queues are inefficient and do not fully implement the required semantics.
  • java.lang.ref.SoftReference: Soft references are not guaranteed to be cleared before an OutOfMemoryError is thrown.
  • Threads started outside of Java aren't "visible" (e.g. in ThreadGroup.enumerate()) until they first call Thread.currentThread().
  • java.lang.Thread.getState() returns WAITING or TIMED_WAITING instead of BLOCKING when we're inside Object.wait() and blocking to re-acquire the monitor.
  • java.nio.channel.FileChannel.lock() shared locks are only supported on Windows NT derived operating systems.
  • java.lang.SecurityManager: Deprecated methods not implemented: classDepth(String), inClass(String), classLoaderDepth(), currentLoadedClass(), currentClassLoader(), inClassLoader()

Supported Platforms

This release has been tested on the following CLI implementations / platforms:

CLI Implementation       Architecture      Operating System
.NET 2.0 SP2 x86 Windows
.NET 2.0 SP2 x64 Windows


Partial Trust

There is experimental support for running in partial trust.

Monday, 11 January 2010 06:36:26 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]