# Monday, 03 May 2010
IKVM 0.42 Update 1 Released

I've promoted 0.42 Update 1 RC 2 to an official release.

Changes (Update 1 RC 0 + RC 1 + RC 2):

  • Added fix to mangle all artificial type names if they clash with Java type names in the same assembly.
  • Fix for http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41696.
  • Fixed exception sorter to be correct when invoked with two references to the same object.
  • Fix for bug #2946842.
  • Fixed ikvmstub to not emit stubs for generic type definitions.
  • Fixed several incorrect usages of Type.IsArray when we only want to deal with vectors.
  • Fixed timezone handing bug for unrecognized timezone names.
  • Several partial trust fixes.

The 0.42 release notes can be found here.

If you want to build from source, you need openjdk6-b16-stripped.zip from the 0.42.0.3 folder.

Monday, 03 May 2010 07:10:51 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Friday, 23 April 2010
New Development Snapshot

A fairly massive set of changes this time, mostly around .NET 4.0 suport and IKVM.Reflection fixes.

Targetting .NET 4.0

Please note that the work isn't done yet (in particular, I haven't checked in all the changes required for the .NET 4.0 security model), but it is now possible to build assemblies with ikvmc that target and reference .NET 4.0 while running ikvmc on the .NET 2.0 runtime.

C:\j>ikvmc test.class -nostdlib -lib:\windows\microsoft.net\framework\v4.0.30319 -r:mscorlib.dll
Note IKVMC0001: found main method in class "test"
Note IKVMC0002: output file is "test.exe"

C:\j>test
Hello World, running on CLR 4.0.30319.1


Note that (like the C# compiler) you can also reference .NET 4.0 assemblies while targetting .NET 2.0, but when you run the resulting assembly on .NET 2.0 and it tries to load the .NET 4.0 assembly that will fail with a BadImageFormatException.

The algorithm that ikvmc uses to load assemblies now resembles what csc uses and does not involve the CLR anymore. The same switches are used (-nostdlib, -lib and -reference), but there are some subtle differences. The most important being that ikvmc's -nostdlib also removes the current CLR runtime directory from the search path and that the -reference option (like it always has) has a runtime consequence (because it affects class loader delegation) and the assemblies that aren't explicitly required by the code being compiled will be silently loaded from the library search path (which is: current directory, CLR directory (except if -nostdlib is specified), -lib:<dir> directories, LIB environment directories).

Breaking Change

The above mentioned changes to assembly loading by ikvmc result in some (potential) breaking changes for build scripts. In particular, the -reference option could previously be used with a partial name (which would be resolved by the CLR with the Assembly.LoadWithPartialName() method), but this is no longer supported (to ease transitioning you can still specify the simple name of the assembly and the ikvmc assembly resolver will append ".dll" to it, but it will issue a warning about this.)

Changes:

  • Started implementing the various required changes for the .NET 4.0 security model. This is not finished yet, so running on .NET 4.0 with this snapshot is not yet supported.
  • .NET 4.0 fix: When class GC is enabled, don't intrinsify ThreadLocal.
  • .NET 4.0 support: When class GC is enabled, throw a VerifyError when custom attributes as used that aren't allowed in RunAndCollect assemblies.
  • Added parameter validation to ikvmc's -version option (bug #2987144).
  • Added System.Core target to build a tiny custom version of System.Core.dll that contains only System.Runtime.CompilerServices.ExtensionAttribute and will allow us to build on .NET 2.0 without having to figure out where System.Core.dll lives (or without it even being present).
  • Removed lib/security/local_policy.jar from vfs.zip (because, by default, OpenJDK builds the restricted version) and instead make a vfs alias for it that points to the unrestricted US_export_policy.jar.
  • Changed ikvm.runtime.Startup.setProperties() to take an IDictionary instead of Hashtable. This allows a Dictionary<string, string> to be passed in now and avoids future problems if/when we want to target Silverlight (which doesn't have System.Collections.Hashtable).
  • Rewrote assembly loading for ikvmc and ikvmstub (and unified it). It now no longer depends on the runtime to do assembly name to path resolution and behaves more csc like.
  • Added -nostdlib and -lib options to ikvmc and ikvmstub.
  • IKVM.Reflection: Fixed stack height updating for jmp instruction.
  • IKVM.Reflection: Don't crash when a DllImportAttribute doesn't have an ImportScope (which can happen for C++ code).
  • IKVM.Reflection: Version parts should be treated as unsigned.
  • IKVM.Reflection: Don't loop infinitely when field RVA lies outside of the file.
  • IKVM.Reflection: Don't try to return a MethodBody if the method isn't in IL.
  • IKVM.Reflection: Implemented ModuleRef ResolutionScope for TypeRef.
  • IKVM.Reflection: Fixed type name parsing bug (thanks to Jb Evain for reporting this). Generic type parameter type names can be without assembly name and then need to be resolved in context.
  • IKVM.Reflection: Added workaround for broken compiler(s) that add terminating NUL to type names in custom attribute data.
  • IKVM.Reflection: Added support for custom modifiers in generic type instantions (in signatures).
  • IKVM.Reflection: Having PinvokeImpl set doesn't necessarily imply having an ImplMap record (for mixed mode assemblies).
  • IKVM.Reflection: Made __GetDataFromRVA more general by taking an offset and a length (to be able to reuse the byte array) and this also removes the need for the field type to have a StructLayout.
  • IKVM.Reflection: A Version object that only has Major.Minor set will return -1 for Build and Revision. Handle that case by setting these parts to zero, instead of casting to 65535.
  • IKVM.Reflection: Added support for delay signing (i.e. setting the public key without having the private key).
  • IKVM.Reflection: Dynamic assemblies should also be visible in the universe.
  • IKVM.Reflection: Fixed default assembly resolver to throw/not throw the right exceptions.
  • IKVM.Reflection: Made method/field signature reading lazy. This makes building assemblies with circular dependencies easier.
  • IKVM.Reflection: Fixed method signature handling of custom modifiers (required and optional were mixed up).

Binaries available here: ikvmbin-0.43.3765.zip

Friday, 23 April 2010 09:51:39 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Tuesday, 06 April 2010
0.42 Update 1 RC 2

I back ported a couple more fixes to the stable release.

Changes:

  • Updated version to 0.42.0.6.
  • Fixed ikvmstub to not emit stubs for generic type definitions.
  • Fixed several incorrect usages of Type.IsArray when we only want to deal with vectors.
  • Fixed timezone handing bug for unrecognized timezone names.
  • Several partial trust fixes.

Binaries available here: ikvmbin-0.42.0.6.zip

Sources: ikvm-0.42.0.6.zip, openjdk6-b16-stripped.zip

Tuesday, 06 April 2010 07:40:36 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Friday, 02 April 2010
New Development Snapshot

I've integrated OpenJDK 6 b18. If you're building IKVM from source, you need to download openjdk6-b18-stripped.zip.

Changes:

  • Integrated OpenJDK 6 b18.
  • Fixed IKVM.Reflection bug in version number handling (for version parts > 32K).
  • Added support for generic parameter custom attributes to IKVM.Reflection (this is missing from June 2006 ECMA CLI spec).
  • Fixed IKVM.Reflection Type.FullName bug. Nested types can also have a namespace (not in the C# sense, but in the CLR sense).

Binaries available here: ikvmbin-0.43.3744.zip

Friday, 02 April 2010 06:11:40 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Monday, 29 March 2010
New Development Snapshot

It's been a while and enough changes have accumulated to warrant a new development snapshot.

Changes:

  • Volker implemented dumping a list of threads when Ctrl-Break is pressed (Windows only).
  • Fixed class loader caching in CallerID (thanks to Mainsoft for reporting this).
  • Added workaround to ikvmc for Assembly.Location differing in case when the assembly is loaded from DEVPATH.
  • Added error handling to ikvmc -key: and -keyfile: options.
  • Added code gen optimization to remove some unnecessary explicit class initialization triggers.
  • Several IKVM.Reflection fixes/improvements.
  • Added support to IKVM.Reflection for mcs specific AssemblyBuilderAccess flag (0x800 aka COMPILER_ACCESS) to allow access to members of unbaked TypeBuilders.
  • Added hack to ikvmc to automatically load OpenJDK assemblies from the same location as OpenJDK.Core (when a non-default OpenJDK.Core assembly is specified).
  • Introduced a base class for intrisified AtomitcReferenceFieldUpdaters. This reduces the size of the generates classes somewhat and allows for a public type to represent all intrinsified instances (instead of the HideFromJava nested type).
  • Intrinsified ThreadLocal (under specific circumstances).
  • Fix for bug #2946842.
  • Removed ../../openjdk6-b16 path from response files.
  • Fixed ikvmstub to not export generic type definitions, because the resulting class is final | abstract and that (intentionally) isn't legal.
  • Fixed ClassLoader to not allow unitialized class loader to be used as parent.
  • Replaced incorrect usages of Type.IsArray with ReflectUtil.IsVector().
  • Added workaround for Mono bug #583669.
  • Fixed bug in handling of unrecognized time zones.
  • When running on .NET 3.5 or later, use TimeZoneInfo.Id to identify timezone, because that maps better to the Win32 names that the Java name mapping is based on.
  • Fix for partial trust regression in assembly class loader initialization.
  • Partial trust: File.listRoots() now uses Environment.GetLogicalDrives() instead of Directory.GetLogicalDrives(). Both methods are semantically identical, but the former requires EnvironmentPermission(Unrestricted = true) and the latter SecurityPermission(UnmanagedCode = true). We also now swallow a SecurityException, should it occur.
  • Partial trust: Getting the host name now falls back to "localhost" if we don't have permission to query the name (or if anything else causes GetHostName to fail).

Binaries available here: ikvmbin-0.43.3736.zip

Monday, 29 March 2010 09:00:57 (W. Europe Daylight Time, UTC+02:00)  #    Comments [3]
IE9 Preview disables CLR DEVPATH feature

(This is not related to IKVM.NET, but since I spent several hours tracking this down, I thought I'd write it up here in the hope anyone else struggling with this will find it.)

One of the lesser known (development) features of the CLR is the ability to override the regular Fusion assembly loading rules by setting the DEVPATH environment variable (and adding an entry to the machine.config to enable this).

It turns out that this feature can be disabled by setting HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\DevOverrideEnable to 1.

When the IE9 Platform Preview is installed it adds this value to the registry.

To re-enable DEVPATH, set the value to 0 or delete it.

Monday, 29 March 2010 08:20:22 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Friday, 19 February 2010
0.42 Update 1 RC 1

I fixed a major bug in the automagic .NET serialization support.

Changes:

  • Updated version to 0.42.0.5.
  • Fix for bug #2946842.

Binaries available here: ikvmbin-0.42.0.5.zip

Sources: ikvm-0.42.0.5.zip, openjdk6-b16-stripped.zip

Friday, 19 February 2010 09:40:08 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Friday, 29 January 2010
New Development Snapshot

I've modified ikvmc to use IKVM.Reflection and largely rewritten ikvmstub to directly work with the ikvm internals instead of using the java reflection API. Both ikvmc and ikvmstub can now process assemblies independent from the .NET runtime they run on. This opens up the possibility to start investigating the possibility of Silverlight support.

Changes:

  • Drag-n-drop fix by Nat.
  • Fixed regression introduced in previous development snapshot, related to field accessors.
  • Removed caching of inner classes.
  • Fix for bug #2908683.
  • Various AWT fixes by Volker.
  • Changed JNI to use standard caller ID mechanism.
  • Various JNI optimizations.
  • Fixed http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41696
  • Fixed exception sorter bug exposed by recent Mono sorting change.
  • Fixed Thread.getAllStackTraces() to resume threads that it suspends.
  • Integrated new IKVM.Reflection implementation.
  • Added AllowMultiple = true to RemappedClassAttribute.
  • Fixed atomic update helper nested types to be invisible from Java.
  • Removed support for "ikvm.stubgen.serialver" property that is no longer needed now that ikvmstub doesn't use the runtime to generate stubs.
  • Removed pre-generated stub jars from cvs and modified build process to generate them during the build.
  • Removed "constant" instance field support (which was only used by ikvmstub and doesn't make any sense anyway).
  • Removed ReflectionOnly support from runtime. Now that ikvmstub no longer requires it, there's no good reason to allow Java code to see ReflectionOnly types.

Binaries available here: ikvmbin-0.43.3681.zip

Friday, 29 January 2010 09:35:13 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Monday, 25 January 2010
Introducing IKVM.Reflection

In November 2008 I introduced IKVM.Reflection.Emit, today I'm introducing IKVM.Reflection. It superseded IKVM.Reflection.Emit and also includes the ability to read managed assemblies. In addition, I've also added many other features that aren't directly needed for ikvmc, but are useful for other applications. Almost the complete reflection API has now been implemented and there are several API extensions to support managed PE features that reflection doesn't support (well).

Why?

When I started on IKVM.Reflection.Emit, it wasn't at all clear to me that it would be possible to re-implement the System.Reflection.Emit namespace without also re-implementing the System.Reflection namespace, but it turned out it was. I did run into a few snags, such as the inability to subclass Module and Assembly (this was fixed in .NET 4.0) and a couple of Mono bugs, but on the whole IKVM.Reflection.Emit was very successful. So why then re-implement the System.Reflection namespace as well? The main reasons are Silverlight and .NET 4.0. For ikvmc to be able to target versions of the runtime different from the one it is currently running on, it is necessary to avoid using System.Reflection, because System.Reflection can only ever work with the mscorlib version of the current runtime.

ikvmc & ikvmstub

In the coming time, I plan on integrating IKVM.Reflection into ikvmc (most of the work for this has already been done, if you've been following the ikvm-commit list, you may have seen some changes go in and wondered why they are necessary) and ikvmstub (I haven't started on this yet). Currently, ikvmstub uses java reflection to expose members of the .NET types in an assembly. I chose this because it was the easiest way to make sure that what ikvmstub generated matched the ikvm runtime behavior (because it simply used the ikvm runtime to do the mapping). There are two downsides to this approach. The first is the same as mentioned above with ikvmc, you can only generate mscorlib stubs for the runtime you're currently running on. The second is more philosophical, it introduces a cycle in the build process. To build the IKVM.OpenJDK.*.dll assemblies, you need mscorlib.jar and System.jar, but to generate these stubs you need a compiled class library. To solve both these issues, I plan to rewrite ikvmstub to work directly on the internal ikvm runtime representations (with conditional compilation, like ikvmc does).

Features

This list is not exhaustive, but here are some interesting features of IKVM.Reflection (that are not in System.Reflection):

  • There is no AppDomain global state. Everything is contained in an instance of the IKVM.Reflection.Universe class.
  • No thread safety. If you want thread safety, you'll have to lock the universe object during every operation.
  • You can choose what version of mscorlib to load in the universe (using Universe.LoadMscorlib()) and by implementing a Universe.AssemblyResolve handler you can decide the framework assembly unification policy.
  • Support for querying and emitting .NET 2.0 style declarative security.
  • Support for defining unmanaged resources from a byte array (in .NET, AssemblyBuilder.DefineUnmanagedResources(byte[]) is broken, only the overload that accepts a filename works).
  • The ability to read and emit .NET 1.1, .NET 2.0 and .NET 4.0 assemblies (while running on, for example, .NET 2.0).
  • Full support for vararg calling convention.
  • Support for reading field RVA data (e.g. for the fields that are used by the C# compiler to initialize arrays).
  • The ability to enable/disable "exception block assistance", or get "clever" assistance.
  • Support for querying methodimpl mappings.
  • Support for reference, pointer and array types with custom modifiers (this CLR feature is used by C++/CLI).

Missing Features

Some things are still missing. The most notable being the Emit differences. The emit code was based on the IKVM.Reflection.Emit code and likewise still lacks some of the querying support (for baked types), although the new code is much better than the code in IKVM.Reflection.Emit.dll in this respect.

Here's a list of methods that can still throw a NotImplementedException:

  • FieldBuilder.__GetDataFromRVA()
  • ModuleBuilder.ResolveType()
  • ModuleBuilder.ResolveMethod()
  • ModuleBuilder.ResolveField()
  • ModuleBuilder.ResolveMember()
  • ModuleBuilder.ResolveString()
  • ModuleBuilder.__ResolveOptionalParameterTypes()
  • ModuleBuilder.GetArrayMethod()
  • GenericTypeParameterBuilder.BaseType
  • GenericTypeParameterBuilder.__GetDeclaredInterfaces()
  • GenericTypeParameterBuilder.GetGenericParameterConstraints()
  • GenericTypeParameterBuilder.GenericParameterAttributes
  • TypeBuilder.CreateType() (when invoked a second time)
  • TypeBuilder.__GetDeclaredFields()
  • TypeBuilder.__GetDeclaredEvents()
  • TypeBuilder.__GetDeclaredProperties()
  • ISymbolDocumentWriter.SetCheckSum()
  • ISymbolDocumentWriter.SetSource()
  • Most methods in ISymbolWriter
  • ManifestResourceInfo.ResourceLocation (for resources located in another assembly)
  • ManifestResourceInfo.ReferencedAssembly
  • ManifestResourceInfo.FileName
  • MethodBase.GetMethodBody() (if the method data contains unexpected sections)
  • Type.IsAssignableFrom()

Missing members:

  • Module.GetSignerCertificate()
  • Type.GUID
  • ParameterBuilder.GetToken()
  • PropertyBuilder.PropertyToken
  • MethodBuilder.Signature
  • ConstructorBuilder.Signature
  • MethodBuilder.SetSymCustomAttribute()
  • ModuleBuilder.SetSymCustomAttribute()
  • ConstructorBuilder.SetSymCustomAttribute()
  • ModuleBuilder.DefineResource()
  • AssemblyBuilder.ModuleResolve
  • Assembly.GetManifestResourceStream()
  • Assembly.GetSatelliteAssembly()
  • Assembly.GetFile()
  • Assembly.GetFiles()
  • MethodBuilder.SetMarshal() (obsolete, use MarshalAsAttribute instead)
  • ParameterBuilder.SetMarshal() (obsolete, use MarshalAsAttribute instead)
  • FieldBuilder.SetMarshal() (obsolete, use MarshalAsAttribute instead)
  • ModuleBuilder.DefineUnmanagedResource(byte[]) (because it is broken)
  • AssemblyBuilder.DefineUnmanagedResource(byte[]) (because it is broken)
  • MethodBuilder.CreateMethodBody()
  • Everything that doesn't make sense in a ReflectionOnly context.

Concepts that are not implemented:

  • Most metadata tokens returned by Emit objects are not properly typed (and can't be used for anything, other than comparing them against other metadata tokens).
  • When defining debugging symbols, a single method can only point to a single source document.
  • All type/member lookup operations are case sensitive.
  • Implementing a custom System.Reflection.Binder is not supported.
  • Modules with unsorted metadata tables are not supported.
  • When Type.GetMethods() (or __GetDeclaredMethods) is called on Array types it throws a NotImplementedException, instead of returning the special array accessor methods.
  • Managed function pointer types are not supported. Like System.Reflection, they are returned as System.IntPtr instead.

Linker Prototype

To see if I did miss any important CLR features, I wrote a prototype assembly linker. It is pretty capable, but should not be confused for something that is usable for anything other than exploring. I've used it with C++/CLI (compiled with /clr:pure) to test the more esoteric CLR features. The source for the linker prototype is in the zip linked to below.

The IKVM.Reflection source code is available in cvs. If you just want the binary, the LinkerPrototype.zip contains it.

Monday, 25 January 2010 08:28:20 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Tuesday, 12 January 2010
0.42 Update 1 RC 0

Yesterday, I released 0.42 and as seemingly always happens a bug was reported right after that. So with that we're now on the road to the 0.42 Update 1 release. This is release candidate 0.

Changes:

Binaries available here: ikvmbin-0.42.0.4.zip

Sources: ikvm-0.42.0.4.zip, openjdk6-b16-stripped.zip

Tuesday, 12 January 2010 18:11:43 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]