# Thursday, 20 September 2007
IKVM 0.36 rc3

This is probably the last release candiate before the official 0.36 release.

Changes:

  • Fixed non-blocking nio socket send of zero bytes to not return -1.
  • Fixed nio Selector.wakeup() race condition.
  • Added checks for code that uses reflection to call ClassLoader.defineClass() on the assembly class loader for ikvmc compiled classes to throw IllegalAccessError when the class tries to extend a non-public base class (instead of dying with a Critical Failure).
  • Fixed bug in the handling of Java annotations applied in .NET code (via the corresponding ikvmc generated attributes).
  • Significantly improved performance of Class.getModifiers().

Binaries available here: ikvmbin-0.36.0.2.zip.

Sources (+ binaries): ikvm-0.36.0.2.zip

External sources (haven't changed since rc1): classpath-0.95-stripped.zip, openjdk-b13-stripped.zip

Thursday, 20 September 2007 15:55:12 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Monday, 17 September 2007
Two Useful Tools for IKVM.NET

IKVMDoc

Brian Heineman wrote a doclet to convert Javadoc into .NET XML documentation that can be consumed by .NET tools like Visual Studio. This makes the type and member documentation available in the Visual Studio IntelliSense popups. Very nice! There's still room for improvement, especially in the area of converting broken html tags into well formed xml tags. The code is available cvs. I've used it to generate an xml documentation file for IKVM.OpenJDK.ClassLibrary.dll and that can be downloaded here.

In future IKVM releases I plan to include an ikvmc compiled version of IKVMDoc.

jar2ikvmc

Gena Donchyts wrote a handy tool that analyzes a bunch of jars and generates a script to ikvmc compile these jars in the right order and with the required (static) dependencies. The tool and documentation are available here.

Thanks

Thanks to Brian and Gena for providing these great tools!

Monday, 17 September 2007 08:18:25 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Thursday, 13 September 2007
IKVM 0.36 rc2

Release candidate 2.

Changes:

  • Fixed annotation handling to use Enum.valueOf() instead of reflection, to prevent IllegalAccessException when accessing non-public enum.
  • Bootstrap packages are no longer sealed.
  • Packages for ikvmc compiled code are no longer sealed automatically (only when they are sealed in the manifest).
  • Implemented support for enabling/disabling assertions in ikvm.exe.
  • Fixed ikvmstub exception when the resulting jar is empty.
  • Added support for creating proxies for non-public interfaces in ikvmc compiled assemblies.
  • Made method annotation resolution lazy, to deal with annotations that annotate themselves.
  • Fixed bug in finalize method handling that could cause compiler error if base class explicitly called finalize method in derived class.
  • Improved JLS binary compatibility support.
  • Fixed implicit conversion from System.Type to java.lang.Class to return null instead of throwing a NullReferenceException for unrepresentable types.

Binaries available here: ikvmbin-0.36.0.1.zip.

Sources (+ binaries): ikvm-0.36.0.1.zip

External sources (haven't changed since rc1): classpath-0.95-stripped.zip, openjdk-b13-stripped.zip

Thursday, 13 September 2007 11:53:26 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Monday, 03 September 2007
IKVM 0.36 rc1

Release candidate 1 of the first OpenJDK based release.

Compared with the last snapshot I made one small change, I implemented the new 1.6 JNI method.

Binaries available here: ikvmbin-0.36.0.0.zip.

Sources: ikvm-0.36.0.0.zip, classpath-0.95-stripped.zip, openjdk-b13-stripped.zip

If you want to build from source, you need the classpath and openjdk zips as well. The classpath-0.95-stripped.zip file contains the required AWT/Swing sources (from a patched version of GNU Classpath 0.95). The openjdk-b13-stripped.zip contains the required sources and the required generated sources (from running an OpenJDK build on Linux). These two zips need to be unzipped in the same directory as where you unzip ikvm-0.36.0.0.zip. I've tested the build on Windows with .NET 1.1, .NET 2.0 and Mono 1.2.3 and on Linux with Mono 1.2.5.

Monday, 03 September 2007 09:54:34 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Wednesday, 29 August 2007
New Snapshot

One more snapshot before release candidate 1 with a couple of fixes:

  • Windows x64 specific binaries are now included (in the ikvm\bin-x64 directory).
  • Enabled workaround for x64 tail call optimization.
  • Implemented clearing of SoftReferences. They're still not guaranteed to be cleared before an OutOfMemoryError, but the new behavior is hopefully more reasonable than never clearing them.
  • Fixed code generator bug that could cause AbstractMethodError to be thrown when subclassing a .NET type that has an explicit interface method implementation.
  • Integrated IcedTea updates to support PBEwithMD5andDES.

Binaries available here: ikvmbin-openjdk-0.35.2796.zip.

Wednesday, 29 August 2007 08:26:12 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Tuesday, 28 August 2007
MS07-045 bug

We interrupt our regularly scheduled programming for some Microsoft idiocy. If you have a classic ASP website that instantiates .NET classes via COM interop that suddenly stopped working after installing MS07-045 and you're now getting the dreaded 0x8000FFFF Catastrophic failure aka E_UNEXPECTED, you can fix that by allowing Everyone Read access to HKEY_USERS\S-1-5-20\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones or by running MS07-045-patch (.NET 2.0 required) to make the registry permission changes for you (you need admin rights to make the changes or run the executable).

Tuesday, 28 August 2007 11:38:35 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Friday, 24 August 2007
First OpenJDK Snapshot

Several things conspired to make me realise that it would be a good idea to make an official release based on OpenJDK. While it is technically still a hybrid with GNU Classpath (it uses the AWT / Swing implementation from GNU Classpath 0.95), I don't consider this a big problem due to the fact that AWT support in OpenJDK itself still has holes in it as well (due to the "encumbrances") and the fact that IKVM never officially supported AWT / Swing (and still doesn't). This snapshot has been well tested and the disclaimers no longer apply. Please download it, play with it and report any issues you find (either here in the comments, on the mailing list or to me directly). I will hopefully release the first 0.36 release candidate at the end of next week.

In the past I was never able to give a list of the areas of GNU Classpath that worked or not, because I simply was't familiar enough with the large code base to know which parts were of good quality / compatibility and which weren't. With OpenJDK I can at least say where IKVM diverges from the OpenJDK code. So I've made a list of packages / classes that are not fully implemented or not supported.

com.sun.security.auth.module        Not implemented.
java.applet GNU Classpath implementation. Not supported.
java.awt GNU Classpath implementation. Not supported.
java.io.Console Not implemented.
java.lang.instrument Not implemented.
java.lang.management Not implemented.
java.lang.ref.SoftReference Soft references are never cleared.
java.lang.SecurityManager Deprecated methods not implemented:
classDepth(String), inClass(String), classLoaderDepth(), currentLoadedClass(), currentClassLoader(), inClassLoader()
java.net No IPv6 support implemented.
java.net.ProxySelector Getting the default system proxy for a URL is not implemented.
java.security Not supported.
java.text.Bidi GNU Classpath implementation. Not supported.
java.util.zip Partially based on GNU Classpath implementation.
javax.crypto IcedTea implementation. Not supported.
javax.imageio.plugins.jpeg Not implemented.
javax.management Not implemented.
javax.print Not implemented.
javax.script Not implemented.
javax.security Not supported.
javax.smartcardio Not implemented.
javax.sound Not implemented.
javax.swing GNU Classpath implementation. Not supported.
javax.tools Not implemented.
org.ietfs.jgss Not implemented.
sun.jdbc.odbc Not implemented.
sun.net.www.content.audio Audio content handlers not implemented.
sun.net.www.content.image Image content handlers not implemented.

The entire public API is available, so "Not implemented." for javax.print, for example, means that the API is there but there is no backend to provide the actual printing support. "Not supported." means that the code is there and probably works at least somewhat, but that I'm less likely to fix bugs reported in these areas.

In addition to the issues listed above, the extended charset providers are not included (the ones in charsets.jar in the JRE) and enabling assertions is not implemented.

This snapshot is still based on OpenJDK B13, because the closer to 1.6 the better in my opinion. After the 0.36 release, I will start tracking the OpenJDK development version more closely, but I plan on supporting 0.36 for a relatively long time (even after 0.38 and subsequent IKVM releases).

I've updated the Japi results. The scores are obviously very high now, but there are still some differences. Some differences are due to OpenJDK B13 already containing some new 1.7 APIs, some differences are in the AWT / Swing GNU Classpath code and some are Japi bugs (e.g. the java.util.concurrent differences).

Changes:

  • Renamed IKVM.Hybrid.GNU.Classpath.OpenJDK.dll to IKVM.OpenJDK.ClassLibrary.dll and made OpenJDK build the default.
  • Fixed thread creation to use AccessController.doPrivileged() to get ikvm.apartmentstate system property.
  • Fixed AccessController.getStackAccessControlContext() to use "native" method as marker on the stack for privileged operation (because the native method stub may get inlined away).
  • Added default security policy file to VFS.
  • Integrated OpenJDK java.beans package. Included OpenJDK sun.io package. Included OpenJDK sunw packages (for JDK 1.0 compatibility).
  • Don't create TypeWrapper instances for HideFromJava types.
  • Added rmi skeleton classes.
  • Integrated OpenJDK java.nio package.
  • Changed ikvmres protocol handler to be compatible with both GNU Classpath and OpenJDK url parse exception throwing convention.
  • Switched GNU Classpath AWT/Swing back to version 0.95.
  • Copied GNU Classpath version of java.text.Bidi into openjdk directory.
  • Copied and integrated GNU Classpath's pure Java zip support with OpenJDK zip classes.
  • Added GNU Classpath 0.95 compatible versions of awt\font.cs and awt\toolkit.cs.
  • Refactored system properties initialization.
  • Changed ikvm.exe class library version printing to report both GNU Classpath and OpenJDK versions.
  • Added HideFromJava to AnnotationAttribute type member that can potentially use HideFromJava types.
  • Implemented Thread.dumpThreads() and Thread.getThreads().
  • Added a couple more fake native libraries to VFS.
  • Implemented Thread.WaitUntilLastJniThread() (used by JNI method DestroyJavaVM() to wait for all non-daemon threads to end).
  • Implemented Inet4Address.isReachable().
  • Implemented path canonicalization.
  • Create java.lang.ProcessImpl streams via FileDescriptor instead of going through FileChannelImpl.
  • Added support for StructLayoutAttribute annotations.
  • New FileChannelImpl implementation based on OpenJDK.
  • Added map.xml workaround for java.net.DatagramSocket.receive() bug.
  • Implemented DatagramSocketImpl.peek() and peekData().
  • Don't add KeepAlive to constructors of objects that don't have finalizers and extend cli.System.Object.
  • Implemented sun.net.dns.ResolverConfigurationImpl "native" methods.
  • Added map.xml workaround for OpenJDK java.beans.XMLDecoder bug.
  • Fixed sun.nio.ch.DotNetSelectorImpl to make "infinite" blocking select block for Integer.MAX_VALUE instead of returning right away.
  • Fixed sun.nio.ch.Net.writeImpl() to return IOStatus.UNAVAILABLE if the socket is in non-blocking mode and the write failed because of this.
  • Added workaround for .NET 1.1 Directory.Delete() bug.
  • Updated all Java versions to 1.6.

Binaries available here: ikvmbin-openjdk-0.35.2791.zip.

Friday, 24 August 2007 09:42:45 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Thursday, 09 August 2007
New Hybrid Snapshot

I did some stabilization work on the previous snapshot and I finally caved in and made floating point handling more compliant (still not fully compliant with the JVM spec, as that would be way too slow).

Changes:

  • Added support for generating access stubs in public interfaces that extend non-public interfaces.
  • Added workarounds for several OpenJDK class initialization order issues.
  • Fixed build process to include the rmi stubs and ties.
  • Renamed sun.misc.Unsafe instance field to theUnsafe, to facilitate sun.corba.Bridge which accesses the field thru reflection.
  • Changed AtomicBoolean.value field from boolean to int to be serialization compatible with JDK.
  • Fixed ArrayTypeWrapper.Finish().
  • Fixed java.lang.reflect.Array.set() to only unbox primitives when the array is a primitive array.
  • Fixed java.lang.reflect.Array.multiNewArray() to finish the array type before using it.
  • Implemented improved floating point compliance (strictfp is now honored and value set conversion is implemented).

Binaries available here: ikvmbin-hybrid-0.35.2777.zip.

Thursday, 09 August 2007 10:16:51 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Tuesday, 07 August 2007
Floating Point "Redundant" Cast Performance on the CLR

Let's start out with a simple micro benchmark:

using System;
using System.Threading;
class Program
{
public static void Main()
{

int start = Environment.TickCount;
double[] d = new double[1000];
for (int i = 0; i < 1000000; i++)
{
for (int j = 0; j < d.Length; j++)
{
d[j] = (double)(3.0 * d[j]);
}
}
int end = Environment.TickCount;
Console.WriteLine(end - start);
}
}

On my system this takes about 7 seconds when run in optimized mode (i.e. not in the debugger).

Here's the optimized x86 code generated by the 2.0 CLR JIT for the body of the inner loop:

fld   qword ptr [ecx+edx*8+8]      ; d[j]
fmul  dword ptr ds:[007B1230h]     ; * 3.0 
fstp  qword ptr [esp]              ; (double) 
fld   qword ptr [esp]              ; (double) 
fstp  qword ptr [ecx+edx*8+8]      ; d[j] = 

There first thing that jumps out is that the double cast takes two x87 instructions, a store and a load. Part of the reason the cast is expensive is because the value has to leave the FPU and go to main memory and back into the FPU. In this particular case it turns out to be very expensive, because esp happens to be not 8 byte aligned.

Making a seemingly unrelated change can make the micro benchmark much faster, just adding the following two lines at the top of the Main method will make the loop run in about 2.3 seconds on my system:

    double dv = 0.0;
Interlocked.CompareExchange(ref dv, dv, dv);

The reason for this performance improvement becomes clear when we look at the method prologue in the new situation:

push  ebp 
mov   ebp,esp 
and   esp,0FFFFFFF8h 
push  edi 
push  esi 
push  ebx 
sub   esp,14h

This results in an 8 byte aligned esp pointer. As a result the fstp/fld instructions will run much faster. It looks like a "bug" in the JIT that it doesn't align the stack in the first scenario.

Of course, the much more obvious question is: Why does the cast generate code at all, isn't a double already a double?

Before answering this question, let's first look at another minor change to the micro benchmark. Let's remove the Interlocked.CompareExchange() again and change the inner loop body to the following:

    double v = 3.0 * d[j];
d[j] = (double)v;

With this change, the loop now takes just 1 second on my system. When we look at the x86 code generated by the JIT, it becomes obvious why:

fld   qword ptr [ecx+edx*8+8] 
fmul  dword ptr ds:[002A1170h] 
fstp  qword ptr [ecx+edx*8+8]

The redundant fstp/fld instructions are gone.

Back to the question of why the cast isn't always optimized away. The reason for this lies in the fact that the x87 FPU internally uses an extended 80 bit representation for floating point numbers. When you explicitly cast to a double, the ECMA CLI specification requires that this results in a conversion from the internal representation into the IEEE 64 bit representation. Of course, in this scenario we're already storing the value in memory, so this necessarily implies a conversion to the 64 bit representation, making the extra fstp/fld unnecessary.

Finally, in x64 mode all three variations of the benchmark take 1 second on my system. This is because the x64 CLR JIT uses SSE instructions that internally work on the IEEE 64 bit representation of doubles, so the cast is optimized away in all situations here.

For completeness, here's the code generated by the x64 JIT for the inner loop body:

movsd  xmm0,mmword ptr [rcx] 
mulsd  xmm0,mmword ptr [000000C0h] 
movsd  mmword ptr [rcx],xmm0 
Tuesday, 07 August 2007 15:02:08 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Monday, 06 August 2007
IKVM 0.34 Update

I made another 0.34 update, since 0.36 is probably still a ways off.

Changes:

  • Fixed handling of magic “assembly” type for assembly attribute annotations (bug #1721688).
  • LocalVariableTable robustness fix (bug #1765952).
  • Fixed handling of public interfaces extending non-public interfaces in ikvmc.
  • Fixed parameter annotations on redirected contructors.
  • Fixed casting ghost interface arrays (bug #1757889).
  • Fixed JNI NewObject method.
  • Fix to make sure all implemented interface methods on .NET types are public (so that ikvmstub generates jars that javac is happy with).

Files are available here: ikvm-0.34.0.4.zip (source + binaries) and ikvmbin-0.34.0.4.zip (binaries).

Update: I forgot to update the AWT toolkit property's assembly version. Fixed in current zips. Thanks to Ted O'Conner for pointing this out.

Monday, 06 August 2007 10:41:11 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]