# Monday, 11 March 2013
New Development Snapshot

I'm still working on issues resulting from the big change. There are some more known issues and probably a few unknown ones. However, I couldn't help myself and decided to a little bit of work on Java 8 support. If you define the IKVM_EXPERIMENTAL_JDK_8 environment variable, you can now run Java 8 class files that use the new static and default interface method features.

There is no interop with other .NET languages yet, so you won't be able to call static interface methods from (e.g.) C# or take advantage of default interface method implementations. The interop story for default interface methods isn't going to be great even when it is done. You'll have to manually call the default methods in your C# code. Something like this:

class MyCSharpClass : SomeJavaInterface {
   public void method() {
      SomeJavaInterface.__DefaultMethods.method(this);
   }
}

Changes:

  • Optimized some dynamic binding operations.
  • Switched dynamic field binding to method handles.
  • Removed "effectively final" optimization because it breaks the ability to dynamically load subclasses.
  • Added experimental support for static methods in interfaces (for Java 8 class files).
  • Added experimental support for default interfaces methods (for Java 8 class files).
  • Fixed an ikvmc crash when a member is accessed on a type from a missing assembly.
  • Fixed transparency artefact with AlphaComposite.Src and and anti-aliased text.
  • Bug fix. Don't implement interfaces that aren't accessible.
  • Changed TextArea peer from TextBox to RichTextBox (to handle LF correctly) and implemented replaceRange.
  • Tweaked f2i, f2l, d2i and d2l bytecodes to be a bit faster in the common case.

Binaries available here: ikvmbin-7.3.4817.zip

Monday, 11 March 2013 08:35:25 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Monday, 04 March 2013
New Development Snapshot

Another week, another snapshot.

Changes:

  • Fixed several annotation encoding issues in new stub class generator.
  • Added back support for exporting annotations in .NET assembly stub classes.
  • Fixed ikvmc to not add ObsoleteAttribute to deprecated types/members that already have an explicit ObsoleteAttribute annotation.
  • Fixed a typo in ikvm.runtime.Startup.addBootClassPathAssembly() method name.
  • Bug fix. .NET enums used in Java annotations were not usable from .NET custom attribute.
  • Added license headers to build scripts.
  • Made boot class package handling simpler (more OpenJDK based). The package information is now read from the manifest instead of hard coded.

Binaries available here: ikvmbin-7.3.4811.zip

Monday, 04 March 2013 09:20:59 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Monday, 25 February 2013
IKDASM Update

I finally created a github repository for ikdasm. A couple of weeks ago I fixed some ildasm compatibility issues and changed pinvokeimpl and marshalas handling to use the IKVM.Reflection specific APIs instead of decoding the pseudo custom attributes (I also used the new IKVM.Reflection feature to disable generating pseudo custom attributes). I added external module support and fixed some other small issues.

The primary purpose of ikdasm is to make sure that the IKVM.Reflection API is complete. I believe it now exposes all relevant Managed PE file features. The secondary feature is to test IKVM.Reflection and to make this easy ikdasm replicates (almost) all ildasm quirks to enable comparing the output files. I disassembled a large number of files (including C++/CLI and Managed C++ files) and compared the results.

Only a small subset of ildasm functionality has been cloned. There is no GUI and most command line options are also not implemented.

Pull requests are welcome.

Monday, 25 February 2013 11:58:14 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
New Development Snapshot

Still more changes to better support what I'll start calling "mixed mode" (i.e. ikvmc compiled assemblies that use dynamically loaded classes or use dynamic binding to classes in another assembly).

Another change is that runtime stub class generation is now based on the ikvmstub class file writer, instead of the very old code that was reflection based. This means that stubs can now acurately be generated even when some of the types involved are not available.

Changes:

  • Refactored assembly class loading.
  • Added ikvm.runtime.EnumerationWrapper to expose an IEnumerable as a java.util.Enumeration.
  • Removed the old runtime Java stub class generator and replaced it with the ikvmstub core.
  • Allow dynamic class loading from boot class "path" and referenced assemblies.
  • Regression fix. The previous custom assembly class loader construction rewrite introduced a bug.
  • Bug fix. MethodHandle should be able to call dynamic only methods.
  • Bug fix. MethodHandle to Object.clone/finalize should be special cased.
  • Reimplemented dynamic binding on top of MethodHandles.

Binaries available here: ikvmbin-7.3.4804.zip

Monday, 25 February 2013 07:41:17 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Wednesday, 20 February 2013
New Development Snapshot

A quick update because the previous snapshot had a bug that caused ikvmc to be completely broken on the CLR x86 JIT.

Changes:

  • Changed build to explicitly exclude the classes that are already defined in map.xml.
  • Changed ikvmc to add uncompilable classes loaded from the file system to classes.jar.
  • Changed ikvmc to copy zip file comment.
  • Changed ikvmc to emit a warning if a remapped type duplicates a loaded class.
  • Removed CallerID optimization special casing since we can now call internal members from dynamic assemblies.
  • Added some version information to the Internal Compiler Error message.
  • Added some version information to runtime critical failure.
  • Fixed AttributeHelper to have a deterministic class constructor.
  • Bug fix. Disallow invalid class names in AssemblyClassLoader.loadClass().
  • Removed the special casing of generic type definition loading as we've since exposed the generic type definitions to Java.
  • Some beginnings of class loading refactoring.

Binaries available here: ikvmbin-7.3.4799.zip

Wednesday, 20 February 2013 08:24:44 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Monday, 18 February 2013
New Development Snapshot

Lot's of changes in just a week. The most interesting one being that ikvmc and the runtime now work together to project all the classes that were in the original jars compiled by ikvmc are now visible in the same (virtual) jars at runtime.

To avoid making the generated assemblies much bigger, this is implemented by having ikvmc generate a compressed file called --ikvm-classes--/ in the resources jar that contains the names of the classes compiled from this jar. This typically adds about 0.5% size overhead. At runtime when that jar is read in by java.util.zip.ZipFile it expands the list of class names to create virtual ZipEntries. Only when a virtual class resource is accessed is the class stub generated.

Changes:

  • Fixed ikvmc to give a proper error message if it fails to write to the output file.
  • If a class can't be statically compiled due to a missing base class/interface, include it as a resource.
  • Modified the assembly class loader to try to load classes from resources.
  • Made dynamic binding to unloadable types the default (for ikvmc).
  • Added -static option to ikvmc to disable dynamic binding.
  • Unified all ikvmc filename validation.
  • Implemented StandardGlyphVector.getGlyphOutline().
  • Bug fix. When looking for a method to override we should handle internal access methods.
  • Fixed some runtime memory model issues (for non-x86 architectures).
  • Bug fix. Custom attribute annotation should skip indexer properties.
  • Bug fix. The available() method of the InputStream returned by ZipFile.getInputStream() was based on ZipEntry passed in, instead of the actual one.
  • Fixed ikvmc to suppress "class not found" warning for a classes that fails to compile for any reason (because we've already given a warning about it).
  • Bug fix. Don't look for main method in excluded classes.
  • Unified the handling of resources and classes in ikvmc.
  • Include all uncompilable .class files (from jars) into the resource jars.
  • Project stub classes into the jar the classes originated from.
  • Fixed ikvmc to prefer the last jar entry if there are duplicate entries. This mirrors the behavior of Java and dynamic mode.
  • Fixed ikvmc to compare extensions case sensitive to find .class files.
  • Fixed ikvmc to use the jar entry name, instead of the class name to load classes.
  • Removed exclude.lst left over from ancient times.
  • Removed META-INF/mailcap.default and META-INF/mimetypes.default resources that are no longer in resources.jar.
  • Changed ikvmc -recurse: option to give a fatal error if it matches no files.

Binaries available here: ikvmbin-7.3.4796.zip

Monday, 18 February 2013 09:05:07 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Monday, 11 February 2013
New Development Snapshot

The recent architectural change makes it possible for the static compiler to use dynamic binding when it encounters a missing type. This in turn makes the dynamic binding support more important, so I've been fixing a lot of bugs in this area.

Currently, ikvmc will still emit code that statically throws a NoClassDefFoundError when accessing missing types, but in the future the default will change to generate dynamic binding code in this case.

Changes:

  • Fixed a bug with Graphics.setComposite(x). This fixes most paint artefacts of Nimbus L&F.
  • Fixed a NPE in constructor of BufferedImage.
  • Fixed TexturePaint bugs.
  • Enable ldc to work for unloadable types in dynamic mode.
  • Changed dynamic bytecode helper methods to be CallerID based instead of trusting the caller to provide the right context type handle.
  • Use sun.launcher.LauncherHelper to make the launcher more consistent with Java.
  • Implemented package access checks in dynamic mode.
  • Fixed reflection handling of unloadable types.
  • Added CodeGenOptions.DisableDynamicBinding to disable dynamic binding, instead of conditional compilation.
  • Bug fix. Dynamic method invocations should unwrap InvocationTargetException.
  • Bug fix. Dynamic instantiation should throw InstantiationError when trying to instantiate an abstract class.
  • Bug fix. Dynamic invokespecial (that isn't a new) should fail verification.
  • Bug fix. Dynamic method lookup should also look in base classes (for non-constructor methods).
  • Bug fix. When dynamically calling a non-constructor instance method, the receiver object should be of the correct type.
  • Added support for setPaint with custom Paint implementation.
  • Added support for dynamic (non-loadable type) annotations in statically compiled code.
  • Have ikvmc record inner classes even if they can't be loaded.
  • Have ikvmc record the outer class even if it is not loadable.
  • Bug fix. If the runtime tries to load a class from an assembly with a custom assembly class loader, the custom assembly class loader should be called. Previously it would only be called if the Java class loader had been instantiated.
  • Bug fix. We should emit miranda methods for non-abstract classes too.
  • Bug fix. Mark "missing" interface methods so that subclasses (that are compiled separately) know that the base class is incomplete.
  • Allow dynamic assemblies injected into assembly class loaders to be (debug dump) saved.
  • Changed the build system to automatically scan all sources files for copyright statements and validate that all GPL licensed files include the Classpath exception.
  • Removed resource compiler dependency from JVM.DLL build.
  • Bug fix. Custom assembly class loader was made visible to other threads while it was still being constructed.
  • Stop considering ACC_SUPER when linking invokespecial (unless compatibility switch is set). This change matches the security change in Java 7u13.
  • Improved ldc MethodType: Use nested type with class constructor to create values with statically know signatures (like ldc MethodHandle).
  • Improved ldc MethodType: Use caching for dynamicallly constructed MethodType values but retry on failure.
  • Fixed dynamic dispatch class loading exception handling to wrap non-java.lang.Error exceptions in NoClassDefFoundError.
  • Added support for dynamic ldc MethodHandle.
  • Added support for dynamically linking the boostrap method of an invokedynamic.
  • IKVM.Reflection: Expose the Name and __ModuleHash for missing external Modules.
  • IKVM.Reflection: Regression fix. The AssemblyNames returned by Module.__GetReferencedAssemblies() didn't have the ContentType set for windowruntime assembly references.
  • IKVM.Reflection: Added new public API Module.__ResolveTypeSpecCustomModifiers() to resolve the (useless) custom modifiers that can be put on TypeSpecs.
  • IKVM.Reflection: Added another overload of the public API Module.__GetSectionInfo() that returns more information about the section.
  • IKVM.Reflection: Bug fix. The OriginalFilename in the version info resource should only include the filename, not the path.

Binaries available here: ikvmbin-7.3.4790.zip

Monday, 11 February 2013 13:02:04 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Friday, 25 January 2013
Fixing a Long Standing Limitation

I finally fixed the limitation that classes dynamically loaded by an assembly class loader can't access the internals of the assembly.

Here's a simple demonstration of the problem. Suppose you have these two classes:

class Other {
  static {
    Test.m();
  }
}

class Test {
  public static void main(String[] args) throws Exception {
    Class.forName("Other");
  }
  static void m() {
    System.out.println("Test.m");
  }
}

Now if you compile Test.class (without Other) into an assembly with a ClassPathAssemblyClassLoader and run this with IKVM.NET 7.2 you'll see something like this:


C:\j>ikvmc Test.class -classloader:ikvm.runtime.ClassPathAssemblyClassLoader
IKVM.NET Compiler version 7.2.4630.6
Copyright (C) 2002-2013 Jeroen Frijters
http://www.ikvm.net/

note IKVMC0001: Found main method in class "Test"
note IKVMC0002: Output file is "Test.exe"

C:\j>Test
Exception in thread "main" java.lang.ExceptionInInitializerError
        at cli.System.Runtime.CompilerServices.RuntimeHelpers._RunClassConstructor(Unknown Source)
        at IKVM.Internal.TypeWrapper.RunClassInit(Unknown Source)
        at IKVM.NativeCode.java.lang.Class.forName0(Unknown Source)
        at java.lang.Class.forName(Class.java:287)
        at Test.main(Test.java:9)
Caused by: cli.System.MethodAccessException: Test.m()
        at Other.<clinit>(Test.java)

The call from Other to Test fails with a MethodAccessException, because the dynamic assembly does not have access to the internal members of the Test.exe assembly.

Long ago I considered fixing this by adding an InternalsVisibleToAttribute to all statically compiled assemblies that allows the runtime generated dynamic assembly access to its internals, but I dismissed this because it would mean that untrusted code could abuse this same assembly identity to access the internals of any ikvmc compiled assemblies.

Forging a StrongNameKeyPair

Yesterday I realized that I could use InternalsVisibleToAttribute without opening a security hole. When the IKVM runtime is running as fully trusted code, it can forge a StrongNameKeyPair to attach to the dynamic assemblies it generates and thereby gain access to the statically compiled assemblies, without giving untrusted code the same access.

The only downside to this approach is that it relies on an implementation detail of the CLR (and Mono). I try to stay away from depending on implementation details, but in this case the gain far outways any risk.

Using It From C#

It also works for your C# assemblies. By adding a custom attribute, you can now enable dynamically loaded Java classes access to your assembly internals.

Here's a C# example:

using System;
using System.Runtime.CompilerServices;
using java.lang;
using java.lang.reflect;

[assembly: InternalsVisibleTo("DemoApp-ikvm-runtime-injected, PublicKey=00240000048000009400000006020000002400005253413100040000010001009D674F3D63B8D7A4C428BD7388341B025C71AA61C6224CD53A12C21330A3159D300051FE2EED154FE30D70673A079E4529D0FD78113DCA771DA8B0C1EF2F77B73651D55645B0A4294F0AF9BF7078432E13D0F46F951D712C2FCF02EB15552C0FE7817FC0AED58E0984F86661BF64D882F29B619899DD264041E7D4992548EB9E")]

interface IFoo {
  Program Instance { get; }
}

class Program : InvocationHandler {
  static void Main() {
    Class c = typeof(Program);
    var cl = c.getClassLoader();
    var interfaces = new Class[] { typeof(IFoo) };
    var handler = new Program();
    var proxy = (IFoo)Proxy.newProxyInstance(cl, interfaces, handler);
    Console.WriteLine(proxy.Instance);
  }
  public object invoke(object obj, Method m, object[] objarr) {
    return this;
  }
}

Here we create a Proxy for the non-public interface IFoo. This is allowed because of the InternalsVisibleToAttribute that gives access to an assembly named DemoApp-ikvm-runtime-injected with the specified public key. The name of the assembly must be the name of the current assembly (in this example "DemoApp") with "-ikvm-runtime-injected" appended. The public key is always the same. The corresponding private key does not exist.

Development Snapshot

Please note that this is a development snapshot, not a release or release candidate. So consider it untested.

Changes:

  • Use InternalsVisibleToAttribute to allow the runtime to dynamically inject classes into statically compiled assemblies.
  • Stop generating proxy helpers for non-public interfaces.
  • Added support for RadialGradientPaint in Graphics.setPaint().

Binaries available here: ikvmbin-7.3.4772.zip

Friday, 25 January 2013 07:51:20 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Wednesday, 16 January 2013
New Development Snapshot

The recent Windows RT jailbreak enabled me to run IKVM on my Surface RT. Out of curiousity I decided to run a part of the test suite. Only three tests failed, one because it relies on specific GC behavior and two because they use JVM.DLL. Previously, IKVM's JVM.DLL assemblies were created with ilasm, because that was the easiest way to create unmanaged exports at the time. However, ilasm doesn't seem to support ARM, so I added this support to IKVM.Reflection and wrote a simple tool to generate unmanaged export assemblies. Of course, supporting unmanaged exports on Windows RT in IKVM.Reflection is not very useful, but it was an interesting exercise to learn a little bit about the ARM instruction set.

I also did some more useful work. I finally got around to changing how ikvmc handles assembly references. It now works like the C# compiler and uses the IKVM.Reflection support for missing assemblies that I added for mcs. This means that it no longer automatically loads assemblies (apart from the IKVM runtime and class library assemblies) and you need to add explicit references for indirectly used assemblies as well. The downside is that this has the potential to break some builds, but the upside is that the compilation process is now more predictable and only loads the actually required assemblies, like a "real" compiler, instead of the System.Reflection heritage that required loading even unused dependencies.

Another change is that ikvmstub jars are now officially deprecated from being used with ikvmc (it will still work for a while, but a warning message is issued).

Finally, it is now illegal to explicitly reference a secondary assembly in a shared class loader group. Previously this caused a warning message, but now this is an error. You need to reference the primary assembly.

Changes:

  • Deduplicate ikvmc messages based on all parameter values, not just the first one.
  • Allow ikvmc messages to be suppressed based on any number of parameter values.
  • Fixed ikvmc to write fully specific -nowarn options to suppress file.
  • Fixed ikvmc to handle missing types.
  • Added ikvmc support for missing assemblies.
  • Officially deprecated using stubs with ikvmc.
  • Bug fix. Don't add duplicate methods to attribute annotation interfaces. The primary cause of this was attributes with virtual properties where we would add the base class property as well as the derived class overridden property.
  • Changed build process to replace ilasm usage to generate JVM.DLL with a custom tool.
  • Bug fix. Local variable analysis for finally blocks was incorrect. Fixes bug #3600788.
  • Changed ikvmc to give an error message (instead of a warning) when referencing a secondary assembly from a shared class loader group).
  • Made ikvmc EditorBrowsableAttribute construction fully symbolic, to avoid having to load System.dll.
  • Disabled automatic assembly loading for ikvmc.
  • Suppress ikvmc warnings after an error has occurred (to avoid obscuring the fact that compilation failed).
  • IKVM.Reflection: Added UniverseOptions.ResolveMissingMembers and deprecated EnableMissingMemberResolution().
  • IKVM.Reflection: Added Universe.ResolvedMissingMember event.
  • IKVM.Reflection: Fixed importing of CLR types with funny names (where we'd previously run into bugs in CLR's Type.Name and Type.Namespace).
  • IKVM.Reflection: Restructured relocation writing to pack all relocation in a page in the same block, as WoA apparently requires this.
  • IKVM.Reflection: Implemented ARM startup stub.
  • IKVM.Reflection: Implemented ARM unmanaged exports.

Binaries available here: ikvmbin-7.3.4764.zip

Wednesday, 16 January 2013 12:24:57 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Tuesday, 15 January 2013
IKVM.NET 7.2 Update 1 Release Candidate 0

An updated release with a couple of bug fixes.

Changes (relative to 7.2):

  • Bug fix. Don't deadlock AppDomain.ProcessExit event handler when the event gets called from another thread than the one initiating exit.
  • Bug fix. Static compiler should not use proxy stubs to implement non-public interfaces in another assembly.
  • Bug fix. Don't add duplicate methods to attribute annotation interfaces.
  • Bug fix. Local variable analysis for finally blocks was incorrect. Fixes bug #3600788.

Binaries available here: ikvmbin-7.2.4630.6.zip

Sources: ikvmsrc-7.2.4630.6.zip, openjdk-7u6-b24-stripped.zip 

Tuesday, 15 January 2013 11:47:09 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]