# Friday, 04 November 2011
Java Method Overriding Is FUBAR Part 5 of ∞

One of the bizarre things is that the method overriding behavior changed between Java 6 and Java 7 (and HotSpot preserves the Java 6 behavior for classes that have major class file version < 51), but the spec doesn't mention this at all, where it generally is pretty good about mentioning different behavior based on class file version.

Here's an example that uses a class hierarchy that mixes version 50 and 51 classes to confuse HotSpot JDK 7:

package p1;

public class A
{
  { foo(); }
  public void foo() {
    System.out.println("A.foo");
  }
  public static void main(String[] args) {
    new p4.D();
  }
}

package p2;

public class B extends p1.A
{
  { foo(); }
  void foo() {
    System.out.println("B.foo");
  }
}

package p3;

public class C extends p2.B
{
  { foo(); }
  void foo() {
    System.out.println("C.foo");
  }
}

package p4;

public class D extends p3.C
{
  { foo(); }
  void foo() {
    System.out.println("D.foo");
  }
}

Here's a table view of the structure:

p1   p2   p3   p4
A      
  B    
    C  
      D

After you compile this with javac 7 (again using the trick to first compile without the public modifier on A.foo) then use a hex editor to modify p3/C.class to change the class file version from 51 (33 hex) to 50 (32 hex) at offset 7 in the file.

When you run it you get:

D.foo
D.foo
Exception in thread "main" java.lang.AbstractMethodError: p3.C.foo()V
        at p3.C.(C.java:5)
        at p4.D.(D.java:3)
        at p1.A.main(A.java:10)

(This is a variation of the bug I reported here.)

Friday, 04 November 2011 08:54:44 (W. Europe Standard Time, UTC+01:00)  #    Comments [2]
Java Method Overriding Is FUBAR Part 4 of ∞

Another example of broken JDK 7 behavior:

package p1;

public class A
{
  { foo(); }
  void foo() {
    System.out.println("A.foo");
  }
  public static void main(String[] args) {
    new p3.F();
  }
}

package p2;

public class B extends p1.A
{
  { foo(); }
  void foo() {
    System.out.println("B.foo");
  }
}

package p2;

public class C extends p2.B
{
  { foo(); }
  void foo() {
    System.out.println("C.foo");
  }
}

package p3;

public class D extends p2.C
{
  { foo(); }
  void foo() {
    System.out.println("D.foo");
  }
}

package p1;

public class E extends p3.D
{
 { foo(); }
 void foo() {
   System.out.println("E.foo");
 }
}

package p3;

public class F extends p1.E
{
  { foo(); }
  void foo() {
    System.out.println("F.foo");
  }
}

To make this is a little clearer, here's a table view of the structure:

p1   p2   p3  
A    
  B  
  C  
    D
E    
    F

When you run it you get:

F.foo
C.foo
C.foo
F.foo
F.foo
F.foo

Not only is this not compliant with the spec, it just plain makes no sense.

Friday, 04 November 2011 08:32:31 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
Java Method Overriding Is FUBAR Part 3 of ∞

For another example of weird JDK 7 behavior, start with the code from part 2 and make B.foo final.

When you run that you get:

C.foo
B.foo

So now C.foo suddenly doesn't override B.foo. This also violates the spec, because when a method that would have been overridden is marked final the verifier should fail to load the class. This shows that the verifier and vtable layout code have different ideas about method overriding semantics.

Friday, 04 November 2011 07:33:47 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
Java Method Overriding Is FUBAR Part 2 of ∞

In part 1 of this series I argued that the spec is broken, now I'll show the first example where the reference implementation does not implement the spec.

package p1;

public class A
{
  { foo(); }
  public void foo() {
    System.out.println("A.foo");
  }
  public static void main(String[] args) {
    new p3.C();
  }
}

package p2;

public class B extends p1.A
{
  { foo(); }
  void foo() {
    System.out.println("B.foo");
  }
}

package p3;

public class C extends p2.B
{
  void foo() {
    System.out.println("C.foo");
  }
}

(If you want to compile this with javac you'll need to first compile it with a version of A that does not have a public foo method and then change A and just recompile A.)

Now the question is does C.foo override B.foo? According to the spec it does not, but when you compile this with javac from JDK 7 (because the class file version needs to be 51 to get the new behavior) and run it you get:

C.foo
C.foo

So C.foo overrides both A.foo and B.foo.

Friday, 04 November 2011 07:27:31 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
Java Method Overriding Is FUBAR Part 1 of ∞

The Java Virtual Machine Specification Java SE 7 Edition finally has a section about method overriding (5.4.5). Unfortunately, it does not document the behavior of the Java 7 reference implementation, nor the Java 6 behavior. It also turns out to be a bad design.

Suppose you have a third party widget framework that has a class:

package org.example;

public class Widget
{
}

And your application has a subclass of Widget:

package fubar.tech;

public class MyWidget extends org.example.Widget
{
  void frob() { ... }
}

The frob() method is package private and you don't want external code calling it. Now a new version of the widget framework is released that adds a frob method to Widget:

package org.example;

public class Widget
{
  public void frob() { }
}

Starting with Java 7 (and this particular behavior is defined by the spec) your package private MyWidget frob() will now override Widget.frob() and hence be callable by anyone who happens to have a reference to your object. If MyWidget.frob() does anything security critical, the third party framework has now introduced a vulnerability in your code.

Java already had a poor story for adding virtual methods to public classes, but this change has made it even worse.

Friday, 04 November 2011 07:01:39 (W. Europe Standard Time, UTC+01:00)  #    Comments [7]
# Monday, 31 October 2011
New Development Snapshot

Fixed the remaining known OpenJDK 7 issues, apart from method overriding, which finally made it into the JVM spec (section 5.4.5), but it's not clear yet what the actual behavior should be. It's a little ridiculous, but almost 16 years after the release of Java 1.0 a fundamental part of the JVM is still poorly documented and implemented.

Changes:

  • Added ikvmc -platform:arm and -platform:anycpu32bitpreferred options.
  • Bug fix. Make sure sun.misc.Launcher is initialized before setting a security manager, because Launcher assumes there is no security manager yet.
  • Added ikvmstub -forwarders option to support ".NET Core profile" assemblies.
  • Implemented com.sun.security.auth.module.NTSystem.getCurrent().
  • Fixed Throwable.fillInStackTrace() to respect non-writeable stack trace.
  • Fixed Throwable deserialization issues introduced with OpenJDK 7.
  • Fixed Throwable.getSuppressed() to return a global empty array (OpenJDK tests depend on that).
  • Fixed java.lang.ProcessImpl pipe to flush writes.
  • Fixed java.lang.ProcessImpl.openForAtomicAppend() to actually use atomic append.
  • Fixed java.lang.ProcessImpl to clear out the existing environment variables when an envblock is specified.
  • Implemented java.lang.ProcessImpl redirectErrorStream for case where stdout is not also redirected.
  • Fixed core library build issue. java.lang.AutoCloseable.close() method may be used before it has been linked.
  • Bug fix. When calling a final method on a remapped type we can't call the instance method in the remapped type, but we have to call the instancehelper instead.
  • Starting with class file version 51 the <clinit> methods need to be static.
  • Added java.nio.file security manager checks.
  • Added workaround for another x64 JIT bug.
  • IKVM.Reflection: Added .NET 4.5 metadata enum values.
  • IKVM.Reflection: Added support for PortableExecutableKinds.Preferred32Bit.
  • IKVM.Reflection: Added support for process architecture in assembly flags.

Binaries available here: ikvmbin-7.0.4321.zip

Monday, 31 October 2011 12:01:05 (W. Europe Standard Time, UTC+01:00)  #    Comments [4]
# Monday, 10 October 2011
Windows Runtime (aka WinRT) Thoughts

Confusion

After the Build keynote many people were confused. I think there were three main reasons for that: 1) Metro Style Apps and Windows Runtime APIs were introduced together, 2) marketing had banned the mention of COM and 3) Windows Runtime is a bad name.

As the conference progressed and people had time to ask more questions and attend more sessions things became a bit clearer. If you haven't already done so, please read Miguel's blog entry about WinRT.

Let's dive deeper into the three points of confusion:

  1. Metro Style Apps and Windows Runtime are two orthogonal concepts.
    I expect some people to take issue with this, but the reality is that Metro is a new application model and WinRT is a new way of exposing Windows APIs. They are completely orthogonal. To be clear, most new WinRT APIs are only usuable from Metro Style Apps, but that is simply a consequence of the fact that many new APIs were added to support Metro and WinRT is the obvious way to do it.
    It should be obvious by now, but I'll explicitly mention it. Some WinRT APIs are usable from non-Metro apps and some Win32 APIs are useable from Metro apps.
  2. WinRT is an evolution of COM. IMO, if they had mentioned this during the keynote, there would have been a lot less confusion. However, marketing had apparently found that people don't like COM and so it was decided that this should not be mentioned, of course during subsequent talks it quickly became obvious anyway.
  3. The Runtime in "Windows Runtime" is like the Runtime in "C runtime library", not like the Runtime in "Common Language Runtime". The Windows Runtime is not an execution environment, just a set of APIs and a way of exposing these APIs (and your own APIs using a similar mechanism).

What About .NET?

Like COM, Windows Runtime APIs can be used from different languages (but the language needs to do some work to enable this). Microsoft supports using WinRT APIs from C/C++, Javascript and C#/VB. All of these languages were modified to support this. Not all .NET languages automatically get full support for WinRT, for example there is currently no first class WinRT support in F#.

Some people thought that .NET was going to be replaced by WinRT (probably due to reason 3 above), but this clearly doesn't make sense.

Something of interest from a .NET point of view is that instead of type libraries WinRT uses the .NET (ECMA CLI) metadata format to describe the APIs. I made a couple of minor tweaks to IKVM.Reflection to better support this.

What About IKVM.NET?

I have not yet investigated exactly what is needed for WinRT support in IKVM.NET, but it is very likely that WinRT support will arrive some time before Windows 8 is released.

Note however that running IKVM.NET inside Metro Style Apps will not be supported, because the .NET Metro profile is very limited. Hopefully more about this in a future blog entry.

Recommended Viewing

Lap around the Windows Runtime
Windows Runtime internals: understanding "Hello World"
Ten Tips When Writing a Hybrid Language Metro style Application (this title is very misleading, it really is about the design of WinRT)

Monday, 10 October 2011 11:10:17 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Friday, 07 October 2011
New Development Snapshot

Time for a new snapshot. With the (more or less) completion of java.nio.file the release is getting closer. There are still some minor issues, but the bulk of the work is now complete.

Changes:

  • Fixed ikvmstub to handle annotation defaults.
  • Fixed ikvmc to generate (private) setter for final instance fields that get an access stub to support serialization and reflection.
  • Runtime.exec() fix. When disambiguating executable names, we should only try to append .exe if the filename doesn't already contain a dot.
  • Removed unnecessary link demand from MappedByteBuffer that caused problems when building for .NET 4.0.
  • Fixed MethodHandle constructor invocation issue when running on .NET 4.0.
  • Removed mapping of System.Security.VerificationException to java.lang.VerifyError.
  • Fixed os.name and os.version system properties when running on unknown Windows version (Windows 8).
  • Bug fix. IPInterfaceProperties.GetIPv_Properties() can throw an exception (and does so on Windows 8 for some interfaces).
  • Fixed ikvmc to not open the remap file with write access.
  • Implemented remaining java.nio.file functionality (minus most optional functionality and ACL support).
  • IKVM.Reflection: Implemented ToString() for EventInfo and PropertyInfo.
  • IKVM.Reflection: Copy all assembly flags when adding an assembly reference (WinRT fix).
  • IKVM.Reflection: Added API extension to query and define custom attributes on interfaceimpl records.
  • IKVM.Reflection: Added support for targetting ARM.
  • IKVM.Reflection: Added ILGenerator.__StackHeight property API extension.

Binaries available here: ikvmbin-7.0.4296.zip

Friday, 07 October 2011 12:38:26 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Tuesday, 06 September 2011
New Development Snapshot

Time for a new snapshot.

Changes:

  • Enabled parallel class loading.
  • Fix ClassLoader.findLoadedClass0() to handle null string.
  • Fixed class loader to always check parent class loader for validity.
  • Implemented platform MBean server support (although with very limited information/exposed operations).
  • Fixed race condition in Thread.initContextClassLoader().
  • Updated StringHelper to OpenJDK 7.
  • Fixed field reflection not use "slow path" to get consistent exception behavior (accessing Throwable fields on non-Java exceptions is not supported).
  • Implemented Unsafe.defineClass().
  • Completed implementation of new Unsafe.copyMemory() overload. This fixes several direct ByteBuffer regressions (introduced when we started using OpenJDK 7).
  • Implemented SocketChannel.sendOutOfBandData().
  • Implemented DatagramChannel multicast methods.
  • Fix for #3404229.
  • Bug fix. Don't create a miranda method if the class already has a static method with the same signature.

Binaries available here: ikvmbin-7.0.4266.zip

Tuesday, 06 September 2011 12:07:48 (W. Europe Daylight Time, UTC+02:00)  #    Comments [5]
# Thursday, 01 September 2011
Platform MBean Server

The release notes for IKVM.NET have always said "Not implemented" for java.lang.management and javax.management. This was mostly due to the fact that I don't know very much about this area of Java and it doesn't make a lot of sense to use Java management tools when the equivalent .NET management tools are probably a better fit (at least for VM level operations).

This week, prompted by a question on the ikvm-developers list, I decided to look into improving the situation (a bit). As a result it is now possible to get the platform MBean server and to connect to it with the jconsole application.

To start the server run the following code:

java.lang.System.setProperty("com.sun.management.jmxremote", "true");
java.lang.System.setProperty("com.sun.management.jmxremote.authenticate", "false");
java.lang.System.setProperty("com.sun.management.jmxremote.ssl", "false");
java.lang.System.setProperty("com.sun.management.jmxremote.port", "9999");

sun.management.Agent.startAgent();

Now when you start jconsole in the following way, you can connect to localhost:9999

jconsole -J-Dcom.sun.management.jmxremote.ssl=false

Note that the mechanism that jconsole uses to detect and connect to locally running JDK instances is very platform specific and is not supported. Note also that IKVM does not support "agents" , so you have to start the management agent explicitly by calling it directly.

Limitations

The information (and operations) exposed is pretty limited. I still maintain that using .NET specific management tools is a better solution, but if you have any specific scenario you want to see supported, please let me know and I'll consider it.

Code

If you want to play with it, the binaries are available here: ikvmbin-7.0.4261.zip

Thursday, 01 September 2011 11:16:57 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]