# Wednesday, October 29, 2014
New Development Snapshot

I've been busy with other things, but there have been enough fixes to warrant a new snapshot.

Changes:

  • Bug fix. When reading a .class resource from an assembly (to attempt to dynamically define it), read all the bytes.
  • Bug fix. Reading/writing java.nio.file attributes of non-existing file should throw NoSuchFileException.
  • Bug fix. Fixed bitmap synchronization in java.awt.image.BufferedImage and com.sun.imageio.plugins.jpeg.JPEGImageWriter.
  • Ignore -Xmn and -XX: Oracle Java specific command line options in ikvm.exe.
  • Enabled pack200 unpacking algorithm.
  • IKVM.Reflection: Bug fix. If custom attribute named argument parsing fails due to missing type, ConstructorArguments should still work.

Binaries available here: ikvmbin-8.0.5415.zip

Wednesday, October 29, 2014 1:24:17 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Wednesday, July 16, 2014
Java Method Overriding Is FUBAR Part 10 of ∞

Yesterday's JDK 7u65 and 8u11 updates changed method overriding yet again and, of course, it is still broken.

Take this example:

package pkg1;

public class A {
  { foo(); }
  void foo() { System.out.println("A.foo"); }
}

package pkg2;

public class B extends pkg1.A {
  { foo(); }
  void foo() { System.out.println("B.foo"); }
}

package pkg1;

public class C extends pkg2.B {
  { foo(); }
  void foo() { System.out.println("C.foo"); }
}

package pkg2;

public class D extends pkg1.C {
  { foo(); }
  void foo() { System.out.println("D.foo"); }
}

public class test {
  public static void main(String[] args) {
    new pkg2.D();
  }
}

Running this with JDK 8u5 yields:

D.foo
D.foo
D.foo
D.foo

Which is, of course, wrong. In yesterday's updates they tried to fix this, but only partially succeeded:

D.foo
D.foo
C.foo
D.foo

The sensible output would be:

C.foo
D.foo
C.foo
D.foo

Wednesday, July 16, 2014 1:27:41 PM (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
Java Security Fixes

In Februari I reported two Java vulnerabilities to Oracle. Yesterday they released the update that fixed them, so here are the descriptions of the two issues.

@java.lang.invoke.LambdaForm.Compiled

Internally, the JDK uses the LambdaForm.Compiled annotation to mark methods that should be skipped in a security stack walk. In JDK 7 it was possible to apply this annotation to untrusted code. Here's an example:

import java.lang.annotation.*;

@Retention(RetentionPolicy.RUNTIME)
@interface java_lang_invoke_LambdaForm$Compiled { }

class test {
  @java_lang_invoke_LambdaForm$Compiled
  public static void main(String[] args) throws Throwable {
    System.out.println(Class.forName("sun.misc.Unsafe"));
  }
}

If you compile and run this with JDK 1.7.0_60 with a security manager, you get the appropriate AccessControlException. However, if you edit test.class to replace java_lang_invoke_LambdaForm with java/lang/invoke/LambdaForm and run it again, you see that the main method is now skipped in the security check and hence is allowed to access a privileged class.

The fix can be seen here.

MaxArityBug

This example demonstrates that the JDK 1.7.0_60 LambdaForm method handle implementation has a type safety bug when dealing with method signatures with the maximum number of parameters.

Wednesday, July 16, 2014 8:53:16 AM (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Sunday, July 13, 2014
Blog Update

I updated my ancient version of dasBlog. Please let me know if something isn't working properly.

Sunday, July 13, 2014 10:18:00 AM (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Friday, July 11, 2014
New Development Snapshot

Completed LambdaMetafactory intrinsification.

Changes:

  • Fixed several LambdaMetafactory.metafactory() intrinsic bug.
  • Implemented LambdaMetafactory.altMetafactory() intrinsic.
  • Include full class name of anonymous classes in stack trace.
  • Added method and field reflection support for lambda classes.

Binaries available here: ikvmbin-8.0.5305.zip

Friday, July 11, 2014 10:08:05 AM (W. Europe Daylight Time, UTC+02:00)  #    Comments [5]
# Wednesday, July 2, 2014
New Development Snapshot

I fixed a whole bunch of weird edge case behaviors and finally did the first part of the lambda performance work.

Here are some rough performance results (times in milliseconds) for this crude microbenchmark:

iteration IKVM 8.0.5274 IKVM 8.0.5296 JDK 1.8.0_05                     ikvmc ngen
1 1040.2 87.6 71.8   1.3 0.4
2 0.9 0.3 0.3   0.3 0.4
3 0.9 0.3 0.3   0.3 0.4

Only LambdaMetafactory.metafactory() is supported at the moment. LambdaMetafactory.altMetafactory() will follow in a future snapshot.

Changes:

  • Removed sun.misc.Unsafe usage from java.util.concurrent.atomic.Striped64.
  • Bug fix. Handle malformed UTF-16 (invalid surrogates) in type/member names and annotations.
  • Added ikvm.exe.manifest file to enable Windows 8.1 behavior (including getting the right version number).
  • Bug fix. Invokedynamic bootstrap method can also be a constructor.
  • Bug fix. Improved handling of incorrect invokedynamic bootstrap methods.
  • Bug fix. Non-canonical modified UTF-8 encodings should throw ClassFormatError.
  • Changed ClassFormatError message for invalid UTF-8 to match JDK.
  • Bug fix. Array classes get the system protection domain, not the element type's.
  • Bug fix. Interfaces can't "override" final methods in Object.
  • Bug fix. Only the bootstrap class loader is allowed to define classes in the java package.
  • Bug fix. Since Java 1.7 class names aren't allowed to have [ and ] characters.
  • Updated ldc error behavior to match Java 8.
  • Bug fix. Calling BasicFileAttributes.size() on a directory didn't work. Thanks to Lucius Junevicus for reporting this.
  • Bug fix. If an invokedynamic bootstrap argument conversion fails, the exception should not be wrapped in BootstrapMethodError.
  • Added intrinsic for invokedynamic using LambdaMetafactory.metafactory().

Binaries available here: ikvmbin-8.0.5296.zip

Wednesday, July 2, 2014 1:43:42 PM (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Friday, June 13, 2014
Malformed UTF-16

While scanning the Java 8 version of the JVM spec for changes, I noticed this new text:

Names of methods, fields, local variables, and formal parameters are stored as unqualified names. An unqualified name must contain at least one Unicode code point and must not contain any of the ASCII characters . ; [ / (that is, period or semicolon or left square bracket or forward slash).

As per usual, the spec and HotSpot implementation don't agree on the details. In particular the text says that a name must contain at least one Unicode code point. At this point it becomes important what a Unicode code point actually is. Suffice it to say that a single UTF-16 code unit in the surrogate range is not a valid Unicode code point and that such names are accepted by HotSpot without any complaint.

The spec, of course, is also incomplete as it does not address what should happen in case of malformed UTF-16 in general. Or maybe it should just be changed to say "... UTF-16 code unit ..." to match the current behavior and not worry about surrogates at all.

On the .NET side, the strings that aren't string literals in the code are stored as UTF-8 and the ECMA CLI spec requires valid UTF-8, but this is only partially enforced which poses its own set of problems.

After discovering all this, I felt compelled to fix IKVM.NET to behave similarly to HotSpot in this regard and added support for escaping malformed UTF-16 strings when writing metadata and unescaping them when reading it back in.

Finally, let's end with a C# example that shows some of the weirdness on the .NET side:

using System;
using System.ComponentModel;

[DisplayName("\uD800")]
class Surrogate {
  static void Main() {
    var attr = (DisplayNameAttribute)typeof(Surrogate)
                 .GetCustomAttributes(false)[0];
    Console.WriteLine(attr.DisplayName == "\uD800");
  }
}
Friday, June 13, 2014 1:43:41 PM (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Tuesday, June 10, 2014
New Development Snapshot

I implemented Type Annotation reflection and fixed a couple of bugs.

Changes:

  • Implemented type annotation reflection.
  • Bug fix. Process exec should support quotes around filename for VFS executables.
  • Bug fix. Process exec should eagerly close the redirected in/outputs when an exception occurs or immediately after the process exits.
  • Bug fix. ProtectionDomain stack walk should ignore "HideFromStackWalk" frames.
  • Include some missing resources.
  • Added security attributes needed for .NET 4 build.

Binaries available here: ikvmbin-8.0.5274.zip
Sources: ikvmsrc-8.0.5274.zip and openjdk-8-b132-stripped.zip

Tuesday, June 10, 2014 10:19:57 AM (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Wednesday, June 4, 2014
New Development Snapshot

More progress on the OpenJDK 8 integration and a couple of unrelated fixes.

I fixed an incompatibility with Mono (and other ECMA compliant tools that parse custom attributes). When I changed package manifest handling in 7.4 to handle reading the manifest from the right jar I unknowingly chose a custom attribute encoding that works on the CLR, but isn't actually ECMA compliant. Mono has since added support for this encoding.

Another somewhat interesting change is that I added an (internal only) feature to make it easier to replace sun.misc.Unsafe.compareAndSet() with Interlocked.CompareExchange. The trigger was the new reflection data caching in java.lang.Class. Here's an example of what it looks like:

@ikvm.internal.InterlockedCompareAndSet("reflectionData")
static native boolean casReflectionData(Class<?> clazz,
                                        SoftReference<ReflectionData<T>> oldData,
                                        SoftReference<ReflectionData<T>> newData);

The ikvmc compiler will generate a method body that uses Interlocked.CompareExchange<T>() to update the reflectionData field in Class. In other places where I previously used Interlocked methods (e.g. java.util.concurrent.AtomicReferenceArray) I implemented the method body in map.xml using MSIL instructions, but here the generic version of CompareExchange is needed and I didn't want to invent a syntax for instantiating a generic method in map.xml.

Oracle is working on adding atomic operations to Java and I suggested this annotation based approach, but it was rejected because "we don't add language semantics through annotations".

Changes:

  • Implemented method parameter reflection.
  • Compile class library with javac -parameters option to get parameter names for abstract and native methods.
  • Added (undocumented) -noparameterreflection option ikvmc to disable emitting method parameter reflection metadata
  • Enabled MethodParameter attribute in runtime stub generator.
  • Support round tripping malformed MethodParameter attribute in stub generator.
  • Fixed PackageListAttribute to be ECMA compliant.
  • Added support for defining abstract methods in remap file.
  • Implemented sun.nio.ch.Net.poll() native method.
  • Replaced sun.misc.Unsafe cas operations in java.lang.Class with Interlocked.CompareExchange().
  • Replaced AtomicReference[Array].getAndSet() implementations with Interlocked.Exchange() based implementations.
  • Fixed Linux build. Thanks to Joan Karadimov for reporting this.
  • Prevent unfocable windows from being activated by mouse click. (Based on patch by Maria Papendieck of pure-systems.)
  • Fixed clipboard access to use the right thread. (Based on patch by Maria Papendieck of pure-systems.)
  • IKVM.Reflection: Added UniverseOptions.DecodeVersionInfoAttributeBlobs to support decoding CustomAttributeBuilder with blob to extract version info fields.

Binaries available here: ikvmbin-8.0.5268.zip
Sources: ikvmsrc-8.0.5268.zip and openjdk-8-b132-stripped.zip

Wednesday, June 4, 2014 9:30:36 AM (W. Europe Daylight Time, UTC+02:00)  #    Comments [4]
# Monday, May 26, 2014
New Development Snapshot

More progress on the OpenJDK 8 integration.

Changes:

  • Integrated Nashorn.
  • Added workaround for bug in .NET 2.0 implementation of ModuleBuilder.GetMethodToken() exposed by Nashorn.
  • Bug fix. A .NET type should always have access to its nested types.
  • Fixed javap build.
  • Bug fix. The MethodHandle code requires the volatile field accessors in Unsafe for volatile field access.
  • Bug fix. Handle more than 8 parameters in invokedynamic.
  • Bug fix. DynamicMethod's ILGenerator doesn't like unbaked array types.
  • Bug fix. MethodHandle.invoke[Exact] methods should have VarArgs flag set.
  • Added workaround for javac MethodHandle.invoke[Exact] detection bug.
  • Moved cldrdata and localedata into separate assemblies.
  • Implemented native methods for HostLocaleProviderAdapterImpl. The system property "java.locale.providers" can now be set to "HOST" to use the .NET locale implementation.

Binaries available here: ikvmbin-8.0.5259.zip
Sources: ikvmsrc-8.0.5259.zip and openjdk-8-b132-stripped.zip

Monday, May 26, 2014 10:22:29 AM (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]