# Tuesday, 10 March 2009
« New Development Snapshot | Main | Defining Delegates in Java »
100% Pure Java

In the past weekend I tried running JRuby 1.2.0RC2's interactive command line jirb on IKVM.NET. It somehow reminded me of an experience I had a little over a decade ago getting an application certified for as 100% Pure Java.

At the time I didn't really see the point of the 100% Pure Java certification, but Sun convinced us (mostly by offering to pay the certification costs), so we went ahead and eventually got certified. As far as I remember we didn't have any major issues, mostly some hardcoded paths that included backslashes and some dependencies on a case insenstive file system (to make it easier to find these issues without having to buy Solaris, I actually patched the C runtime that came with the Windows JDK to make all file access case sensitive). The other problem was that in those days Swing was exceptionally unstable on Solaris, but they eventually conceded that this wasn't our fault ;-)

JRuby

As some may remember, I have been running JRuby tests for a while. This is an older version of JRuby (I don't even know which version) and not used interactively. In any case, running the most recent JRuby revealed a number of issues:

  1. IKVM.NET bug in JNI that caused constructing an object of a dynamically loaded class via JNI to fail.
  2. IKVM.NET regression introduced a couple of days ago.
  3. An infinite recursion bug in the CLR x64 JIT.
  4. JRuby Issue.
  5. JNA Issue (JNA is a library used by JRuby that provides P/Invoke like functionality).

JRuby Issue

The first issue was that when I started jirb, it would not come up with the irb(main):001:0> prompt but simply read lines, echo them back and execute them. After debugging this I found that JRuby uses reflection to access the private handle field in java.io.FileDescriptor (when on Windows). IKVM's FileDescriptor didn't have a handle field, because it uses .NET's System.IO.Stream instead of the native Win32 API.

I solved this by adding a handle property to FileDescriptor (not the actual code):

@ikvm.lang.Property(get = "get_handle")
private long handle;

private long get_handle() {
  if (stream instanceof cli.System.IO.FileStream) {
    cli.System.IO.FileStream fs = (cli.System.IO.FileStream)stream;
    return fs.get_Handle().ToInt64();
  } else if (this == in) {
    return GetStdHandle(-10).ToInt64();
  } else if (this == out) {
    return GetStdHandle(-11).ToInt64();
  } else if (this == err) {
    return GetStdHandle(-12).ToInt64();
  }
  return -1;
}

@DllImportAttribute.Annotation("kernel32")
private static native cli.System.IntPtr GetStdHandle(int nStdHandle);

Now when JRuby uses reflection to get the handle field, it ends up calling the get_handle method and it will get the right handle back. The nice thing about implementing this as a property it that it means the cost is virtually zero when you're not accessing it and it also delays the full trust requiring native method invocation until it is really needed, which means that partial trust code won't be affected.

This solved the first problem, the right prompt now appeared.

JNA Issue

The next problem appeared after I ran jruby.jar through ikvmc and ran the resulting jruby.exe. After exiting I would get an exception:

irb(main):001:0> exit
java.io.IOException: Cannot run program "C:\.virtual-ikvm-home/bin/java": The system cannot find the file specified
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:474)
    at java.lang.Runtime.exec(Runtime.java:610)
    at java.lang.Runtime.exec(Runtime.java:483)
    at com.sun.jna.Native$DeleteNativeLibrary.run(Native.java:761)
    at java.lang.Thread.threadProc(Thread.java:2297)
    at java.lang.Thread$1.Invoke(Thread.java:797)
    at cli.System.Threading.ThreadHelper.ThreadStart_Context(Unknown Source)
    at cli.System.Threading.ExecutionContext.Run(Unknown Source)
    at cli.System.Threading.ThreadHelper.ThreadStart(Unknown Source)
Caused by: java.io.IOException: The system cannot find the file specified
    at java.lang.ProcessImpl.<init>(ProcessImpl.java:126)
    at java.lang.ProcessImpl.start(ProcessImpl.java:54)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:467)
    ... 8 more

This didn't show up when running via ikvm.exe, because I recently added support for this, but when running jruby.exe the ikvm.exe file wasn't available. I debugged this because I was curious why a new JVM was being started at exit. It turns out that JNA extracts a JNI DLL into the temp directory and wants to delete that at exit. While the DLL is loaded by the JVM, it cannot be deleted so in a shutdown hook it fires up a new JVM to delete the temporary file. This is already pretty gross (and not 100% Pure Java :-)), but not the actual issue. Digging deeper I found that this is the alternative code path, because they first attempt to unload the DLL by explicitly calling java.lang.ClassLoader$NativeLibrary.finalize() via reflection on the NativeLibrary object that corresponds to their DLL. This is, of course, exceptionally evil. Nevertheless, I have now implemented support for this in IKVM.

Conclusion

Maybe 100% Pure Java isn't as useless as I thought a decade ago :-)

Tuesday, 10 March 2009 06:53:42 (W. Europe Standard Time, UTC+01:00)  #    Comments [5]
Tuesday, 10 March 2009 14:46:49 (W. Europe Standard Time, UTC+01:00)
If you have suggestions to improve any of these areas, we're certainly willing to accept patches. Generally we only have resources to test against Sun's JVM on a regular basis, which means everything works great for us but may have trouble on non-Sun JVMs. And of course, much of our use of reflection and native libraries is to implement features that would otherwise be impossible. We would certainly rather be "100% Java" but sometimes that doesn't cut it.

You can also specify -Djruby.native.enabled=false to turn off much of the native library stuff.
Monday, 16 March 2009 20:02:58 (W. Europe Standard Time, UTC+01:00)
Having seen lots of code like this in the wild for no good reason whatsoever, I tend to wish the 100% pure program had caught on.

Its amazing how many "Java is write once test everywhere" complaints really boil down to having used some horrendous code that called sun.xxx classes or reflection for no good reason.
Saturday, 21 March 2009 19:29:09 (W. Europe Standard Time, UTC+01:00)
We've started to get complaints on the JRuby mailing list relating to this post. The implication is that because we use some undocumented or native APIs we're no longer able to claim 100% Pure Java.

The truth is that JRuby works just fine with those extra features unavailable. We have people deploying JRuby on VMs that don't suppose the sun.* extensions, with native code disabled, and everything works. We only go above and beyond "100% Pure" when it doesn't provide what we need to implement a feature.

For example, people simply need to be able to manipulate symlinks. There's no way to do this with "100% Pure Java" unless you shell out. So we have code that will either make a native call, or shell out. The latter works, but it's obviously less than ideal.

I think the tradeoffs we've made to support Ruby are valid and acceptable. And as much as "100% Pure Java" is a useful goal, sometimes it's just not enough.

Jess Sightler: We have lots of good reasons for doing what we do. The most obvious is that "100% Pure Java" can't support many features we need. It's easy to throw stones when you've never tried to support symlinks or fstat or signals on the JVM. Perhaps it would be more constructive for you to look at the parts of JRuby that aren't "100% Pure Java" and offer suggestions, rather than blindly claiming we have done horrendous things for no good reason.
Sunday, 22 March 2009 08:51:22 (W. Europe Standard Time, UTC+01:00)
Hi Charles,

First of all, I apologise for accidentally starting this discussion. I can certainly sympathize with the difficulties in implementing one platform on top of another :-)

When I wrote my blog post I wasn't aware of the "100% Pure Java" claim on JRuby's website, I simply used my experience from the past as a way to make for a (hopefully) more interesting blog post. So I didn't intend this as an attack on JRuby at all, but I now realise that it may have come across as such.

Having said that, "100% Pure Java" has (had?) a very precise meaning and claiming that your code is 100% Pure Java but having additional functionality based on depending on implementation details of the JDK was something that was specifically disallowed.

Also, look at it from my (or any other alternative VM implementer's) point of view: When somebody tries to run an application that claims to be 100% Pure Java and it doesn't work (as well) on my VM, they tend to assume that my VM is broken.

From a practical point of view, I would suggest (but as I said before, I can certainly see both sides of the argument) making the "above and beyond" behavior optional (default to -Djruby.native.enabled=false) and allowing people to explicitly enable it to get a better experience.

Regards,
Jeroen
Monday, 23 March 2009 17:40:54 (W. Europe Standard Time, UTC+01:00)
Hi Charles,

I think that you misunderstood my (overly brief) statement. I wasn't aware of your claim to be &quot;100% pure&quot; either, and frankly didn't care one way or the other about your decision to use these features at all. To me it looked like _you_ did it the right way, by knowingly using the non-pure features to enhance functionality while still offering graceful degradation for those who use VMs without those features.

I'm fine with that, and support such behaviour as it enhances the experience without meaningfully hurting portability.

My comment was about other projects which I've seen (and used) which really do these things for no reason at all. They just do them b/c the authors saw some sun.* class and put no thought into it at all. That hurts all of us. :(
Name
E-mail
Home page

I apologize for the lameness of this, but the comment spam was driving me nuts. In order to be able to post a comment, you need to answer a simple question. Hopefully this question is easy enough not to annoy serious commenters, but hard enough to keep the spammers away.

Anti-Spam Question: What method on java.lang.System returns an object's original hashcode (i.e. the one that would be returned by java.lang.Object.hashCode() if it wasn't overridden)? (case is significant)

Answer:  
Comment (HTML not allowed)  

Live Comment Preview