# Monday, November 8, 2004
« Japitools & New Snapshot | Main | Comment Spam »
Running Derby on IKVM

At the Colorado Software Summit I attended a presentation on Derby by Dan Debrunner. Derby is the open source version of IBM's Cloudscape database engine. After seeing the presentation I decided to try to run it on IKVM. I tried some simple tests and they all went fine. At the moment Derby doesn't include a test suite yet (IBM is working on releasing their tests), so I didn't do any extensive testing.

Since Derby is a real transactional database engine it uses a transaction log and requires FileDescriptor.sync() to work corectly, so I decided to implement that. I hadn't previously implemented it, because it unfortunately requires platform specific code, so it only works on Windows and on Posix systems (when running on Mono).

I wrote a very simple test app in C# (and the Java equivalent) and did some performance tests (download source):

Operation Sun JDK 1.5 IKVM/.NET 1.1 IKVM/Mono 1.0.2
getConnection 1903 1472 2177
create table 450 634 624
prepareStatement1 344 514 978
1000x insert 1031 387 550
prepareStatement2 124 294 231
1000x select 1038 851 3682

(Times in milliseconds, based on average of three best runs out of four runs.)

All the usual disclaimers about benchmarks apply of course, but it's interesting to see that IKVM on .NET outperforms JDK 1.5 on the queries. Preparing the statements take longs, but that is expected, as IKVM has to convert the generated class files to CIL (and this part of IKVM is not very efficient).

Another interesting aside is that when I ran the test on JDK 1.4.1, the insert operation took about 40 seconds to run. This rather extreme performance bug was fixed in 1.4.2.

One thing that is important to note when running Derby on IKVM is that Derby compiles all queries to bytecode and IKVM doesn't support garbage collecting classes (due to .NET's inability to unload code), so unless your application uses a fixed set of prepared statements that fit in the Derby statement cache, you're going to leak memory.

I made a new snapshot that includes the new sync implementation:

  • Resync'ed with GNU Classpath cvs.
  • Implemented java.io.FileDescriptor.sync() and java.nio.channels.FileChannel.force().
  • Fixed java.io.File to treat file paths as case insensitive on Windows.
  • Made exception helper a little more robust against exceptions happening early in bootstrap.
  • Fixed japi build file to include all the public APIs.
  • Fixed ClassLoaderWrapper.getSystemClassLoader() to bypass ClassLoader.getSystemClassLoader() and access the underlying field directly, to prevent a security check.
  • Changed DotNetTypeWrapper to overrider GetClassLoader to lazily get the system class loader.
  • Fixed ikvmc generated main stub to return 1 if the program exited with an exception.
  • Implemented support for java.security.manager system property (the security manager can now be set from the command line using -Djava.security.manager=<class>).

New snapshots: just the binaries and source plus binaries.

Monday, November 8, 2004 11:51:13 AM (W. Europe Standard Time, UTC+01:00)  #    Comments [8]
Tuesday, November 9, 2004 2:59:54 PM (W. Europe Standard Time, UTC+01:00)
Tuesday, November 16, 2004 9:09:36 PM (W. Europe Standard Time, UTC+01:00)
Sorry for commenting on an older post, but this just occurred to me.

Have you considered a mode that puts each classloader in its own AppDomain so that the classloaders can be garbage collected along with their classes in the usual java manner? I don't know if this is possible because of the different semantics between classloaders and appdomains, and even if possible it would probably perform horribly due to marshalling, but for some applications (like this one) it might be appropriate.
Wednesday, November 17, 2004 9:22:21 AM (W. Europe Standard Time, UTC+01:00)
I have thought about it in the past, but I quickly dismissed the idea. The performance and complexity would be horrible.

Take a byte array, for example, it can only live in one AppDomain so the other AppDomain would have to use a proxy object for the byte array and every load or store from/to the array would have to be marshalled across. That's hopeless.
Wednesday, November 17, 2004 11:50:06 AM (W. Europe Standard Time, UTC+01:00)
Did you try the embedded or the network server? I just did a quick test using the network server and it seems to have a problem: (client is in a standard javavm)
it does look like a gnu classpath bug though...

Exception in thread "DRDAConnThread_3" java.lang.Error: java.io.CharConversionException: Bad input byte: 65
at java.lang.String.<init> (map.xml)
at org.apache.derby.impl.drda.DDMReader.isCmd (DDMReader.java:279)
at org.apache.derby.impl.drda.DRDAConnThread.sessionInitialState (DRDAConnThread.java:519)
at org.apache.derby.impl.drda.DRDAConnThread.run (DRDAConnThread.java:214)
at java.lang.VMThread.run (VMThread.java:38)
at java.lang.VMThread$1.Invoke (VMThread.java:239)
Caused by: java.io.CharConversionException: Bad input byte: 65
at gnu.java.io.decode.DecoderUTF8.convertToChars (DecoderUTF8.java:161)
at gnu.java.io.decode.Decoder.convertToChars (Decoder.java:194)
at java.lang.String.<init> (map.xml)
at org.apache.derby.impl.drda.DDMReader.isCmd (DDMReader.java:280)
at org.apache.derby.impl.drda.DRDAConnThread.sessionInitialState (DRDAConnThread.java:519)
at org.apache.derby.impl.drda.DRDAConnThread.run (DRDAConnThread.java:215)
... 2 more
Philipp Hug
Wednesday, November 17, 2004 1:23:12 PM (W. Europe Standard Time, UTC+01:00)
I only tried the embedded server. You're correct that the exception you're seeing is a GNU Classpath bug, its UTF8 decoder doesn't handle invalid sequences correctly (it throws an exception, instead of simply rendering the incorrect bytes as '?').
Thursday, November 18, 2004 3:35:51 PM (W. Europe Standard Time, UTC+01:00)
Hi, I have a question about the classloader support.
I have created a special classloader (in fact it creates a classloader tree)... Which permits me to use librairies which use other librairies of different version at the same time (it's not very clear so here an example)...

Imagine lib B depends on lib A v1 and lib C depends on lib A v2. unfortunately A v1 and A v2 are not compatible, so you have to choose either to use lib B or lib C but not both. With my classloader you can. But as I've understand here, I will not be able to compile this in .net with ikvmc, as their don't seem to exists a classloading equivalent mechanisme in .net. I ask here, because I have created a web java framework, and I would like to "port" it in .net with ikvm, but I don't know how to adapt this (like also I've a dynamic compilation of .java files and loading just after the compilation, it seems impossible to do with ikvm ?
Wednesday, June 27, 2012 5:27:19 PM (W. Europe Daylight Time, UTC+02:00)
Nice idea - I'll be buggered if I can get it to work on the latest Derby builds.

I assume looking at the stack trace it is something to do with IKVM implementation relating to the Java Heap... which is a bit over my head!


[0]: {org.apache.derby.impl.services.cache.ConcurrentCache.release(ConcurrentCache.java:411)}
[1]: {org.apache.derby.impl.store.raw.data.FileContainer.letGo(FileContainer.java:3521)}
[2]: {org.apache.derby.impl.store.raw.data.BaseContainerHandle.close(BaseContainerHandle.java:423)}
[3]: {org.apache.derby.impl.store.access.heap.HeapConglomerateFactory.readConglomerate(HeapConglomerateFactory.java:286)}
[4]: {org.apache.derby.impl.store.access.RAMAccessManager.conglomCacheFind(RAMAccessManager.java:482)}
[5]: {org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(RAMTransaction.java:394)}
[6]: {org.apache.derby.impl.store.access.RAMTransaction.openScan(RAMTransaction.java:1577)}
[7]: {org.apache.derby.impl.store.access.PropertyConglomerate.openScan(PropertyConglomerate.java:229)}
[8]: {org.apache.derby.impl.store.access.PropertyConglomerate.readDbProperties(PropertyConglomerate.java:712)}
[9]: {org.apache.derby.impl.store.access.PropertyConglomerate.getCachedDbProperties(PropertyConglomerate.java:745)}
[10]: {org.apache.derby.impl.store.access.PropertyConglomerate.getCachedProperty(PropertyConglomerate.java:560)}
[11]: {org.apache.derby.impl.store.access.PropertyConglomerate.getProperty(PropertyConglomerate.java:628)}
[12]: {org.apache.derby.impl.store.access.PC_XenaVersion.upgradeIfNeeded(PC_XenaVersion.java:60)}
[13]: {org.apache.derby.impl.store.access.PropertyConglomerate.<init>(PropertyConglomerate.java:165)}
[14]: {org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:1150)}
[15]: {org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1982)}
Friday, June 29, 2012 11:23:33 AM (W. Europe Daylight Time, UTC+02:00)
Thanks for reporting this. The bug is now fixed in the new release candidate here:
Home page

I apologize for the lameness of this, but the comment spam was driving me nuts. In order to be able to post a comment, you need to answer a simple question. Hopefully this question is easy enough not to annoy serious commenters, but hard enough to keep the spammers away.

Anti-Spam Question: What method on java.lang.System returns an object's original hashcode (i.e. the one that would be returned by java.lang.Object.hashCode() if it wasn't overridden)? (case is significant)

Comment (HTML not allowed)  

Live Comment Preview