# Monday, August 12, 2002
No Title

I started on the ahead-of-time-compiler again, with the goal of precompiling the classpath classes. I've created a zip containing all the binaries needed. Just unzip the file, make sure the directory is in the path and run a Java class:
ikvm <class>

It uses the CLASSPATH environment variable (instead of the earlier IKVM_CLASSPATH), and if that is not set, it assumes the current directory. Zips and jars are not yet supported.

The classpath.dll (from the binaries zip) can also be used from (for example) a C# program (don't forget to add a reference to classpath.dll & ik.vm.net.dll):

class JTest
public static void Main()
java.util.Hashtable h = new java.util.Hashtable();
h.put("foo", "bar");
h.put("fu", "baz");
Updated the source snapshot.
Monday, August 12, 2002 4:41:14 PM (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Friday, August 9, 2002
No Title

I did some refactoring and various small changes. Field & method accessibilty is now enforced (although there still are some issues).

Updated the snapshot.

Friday, August 9, 2002 4:11:29 PM (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Thursday, August 8, 2002
No Title

Stuart commented:

What do you do if you have a Java program that depends on String.hashCode() doing exactly what Java defines it to do (ie, produce consistent output with other VMs) rather than what .NET defines String.GetHashCode() to do? I encountered this in my own code so in theory I can implement (or steal from ClassPath) an implementation of String.hashCode() and use that, but the problem could come up in other scenarios. It's an interesting one, don't you think? :)

I haven't gotten around to implementing it, but the idea is that all virtual calls to Object.toString() are changed to:

  if(o instanceof String) return StringHelper.hashCode(o);
  else return Object.GetHashCode();

One reason I haven't done this yet, is because I think it is totally brain damaged that Sun actually specified the hashcode algorithm for String (and in JLS 1.0 they specified a broken algorithm, which JDK 1.0 didn't implement), another reason is that I haven't figured out how to build this generically (specified in the map.xml file, instead of hardcoded in the VM).

A similar issue also occurs with Object.toString(). Nonvirtual calls to Object.toString() should be redirected to a helper function that returns:

  getClass().getName() + "@" + Integer.toHexString(hashCode())

This isn't really a problem (if you accept the fact that non-Java classes return something different for toString(), which I do), but for arrays it does pose a problem. For arrays to return the proper string when toString() is called on them (virtually through an object reference), all virtual calls to Object.toString() need to be treated like hashCode() above.

btw, how come your example isn't "import System.Reflection"? Does netexp auto-lowercase all namespace names?

At the moment it does. Not because this is the Java convention, but because Java cannot really deal with a namespace System (because of the class java.lang.System).

That seems risky if (as I believe, but don't know for sure) it's possible to have two distinct .NET namespaces with names differing only by case...

This is true, but in practice it seems unlikely you'll ever encounter a problem with this, but the case conversion will definitely be optional in the future.

Thursday, August 8, 2002 10:13:57 AM (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Wednesday, August 7, 2002
No Title

I started work netexp. This is the tool that generates Java stubs for all public types in a .NET assembly, so that Java code can be compiled against .NET code (using any standard Java compiler). The VM understands the stubs and replaces any references to them with the actual .NET types.

It's still in its very early stages, but here is an example that already runs:

import system.*;
import system.reflection.*;
class test
  public static void main(String[] args)
    Type type = Type.GetType("System.Object");
    ConstructorInfo ci = type.GetConstructor(
BindingFlags.Public | BindingFlags.Instance,
       null, Type.EmptyTypes, null);

I updated the snapshot.

Wednesday, August 7, 2002 4:11:58 PM (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Tuesday, August 6, 2002
No Title

I wrote a jasmin class that "uses" all bytecode instructions, to check if I had implemented all of them. It turns out three were missing:

  • jsr_w  -- jump to subroutine using a 32 bit offset (methods have a maximum size of 64KB, so why we need this, I don't know)
  • goto_w -- jump to instruction using a 32 bit offset (see jsr_w)
  • dup2_x2 -- scramble the contents of the stack ;-)

I implemented all three, although I very much doubt that they'll ever be used.

In the process of doing this, I found and fixed a few bugs in the verifier and compiler. I also changed the compiler to generate a "throw VerifyError("...")" when compiling a method that doesn't verify (instead of throwing the exception at compile time). This matches the JVM behaviour better.

Updated snapshot is here.

Tuesday, August 6, 2002 2:09:00 PM (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Friday, August 2, 2002
No Title

Stuart commented:

Btw, I'm just trying out your latest snapshot. I'll add further comments as I get further, but so far I had to make a couple of changes to let me build:

VS.NET treated "if (false)" on line 1732 of vm.cs as creating unreachable code. I changed it to "if (false.Equals(true))" so that it didn't treat it as a compiletime constant.

I'm used to using the if(false) construct, because that's the way to not get the compiler to complain in Java. Anyway, I just commented it out now, it's only there for debugging. So it will go away in the near future, I hope.

And the default compilation options disallowed unsafe code, so I had to change that in order for BigEndianBinaryReader.cs to compile.

Aren't you using my VS.NET project? That should include the proper switches for it to build, at least it builds on my machine ;-)

Okay, I've got it to the point of giving me an intelligent error at runtime, at least. Issues so far:

It doesn't (seem to) honor CLASSPATH, at least when run from a command prompt.

That is correct, for the time being it uses the IKVM_CLASSPATH environment variable. BTW, zips and jars are not supported (and won't be for a while, because I want to write the Zip/Jar class loader in Java. That isn't going to happen until after I've got the ahead-of-time compiler working).

Now the showstopper that I haven't been able to figure out how to get past. As far as I can see the download doesn't include *any* of the GNU Classpath libraries, which means that (as far as I can see) it can't run anything at all. I'm guessing that this is just a simple omission from the zip - is that right?

I don't include the classpath stuff to keep the zip small. I did earlier post a link to the compiled classpath classes that I use. I've created a new snapshot (many small changes) and also posted a zip containing the classpath classes plus a few of my own modifications (the source for those is in the classes directory in ikvm.zip).

Friday, August 2, 2002 10:12:28 AM (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Thursday, August 1, 2002
No Title

Stuart commented:

On most aspects, you've convinced me. Just a few little comments:

wrt "throws Exception" vs "throws Throwable" - there's no difference. Any Java method can throw any Throwable subclass that isn't an Exception subclass, just as any Java method can throw any RuntimeException subclass. Thus "throws Exception" really does let you throw anything.

Actually, no. Try to compile the following:

class test extends Throwable
public static void main(String[] args) throws Exception
throw new test();

It doesn't compile. One of the inellegancies of Java's checked exceptions model is that there are two classes that removed the checkedness, both Error and RuntimeException.

Btw, are you arranging that (most? any?) .NET exceptions end up as subclasses of j.l.Exception, or just direct Throwable subclasses?

Throwable, the most important reason is that I don't want to mess with the class hierarchy, if at all possible.

I agree also that remapping things like InputStream is probably too hard. If you have a generic class/interface/method remapping system, though, I can imagine actually experimenting with Collection vs IEnumerable and indexers on List.

One of the goals of XML remapping system, is to enable exactly these types of experiments. It probably isn't flexible enough to do everything you'd want to do, but in the end we'll get there.

When you get IK.VM working on Mono I might even have a try at that myself, since it shouldn't need any deep magic in the VM itself.

I think it'll still be a while before Reflection.Emit in Mono is at the level where it is complete enough.

Oh, and I like the idea of custom attributes to do "throws" expressions on glue code. I wonder if we could persuade the Mono people to get mcs to actually honor the "throws" attributes - ie, compile C# code with Java's 'checked exceptions' semantics. That would be pretty cool :)

It would be cool, but I don't really like checked exceptions. After programming in Java (almost exclusively) for about five years, I finally decided that the cons outnumber the pros. The idea is compelling, but in practice there are too many problems, but I don't really want to start that war, as it has been waged many times already ;-)

BTW, I just changed the idiv, ldiv, irem & lrem implementations to explicity check for -1, and for a lame little test I wrote, performance seems to be OK (same as JDK 1.1 and slightly faster than JDK 1.4, HotSpot doesn't like microbenchmarks ;-)).

Thursday, August 1, 2002 5:14:02 PM (W. Europe Daylight Time, UTC+02:00)  #    Comments [3]
No Title

Zoltan commented on the div issue:

I believe the reason the CLR throws an exception in this case is that the div CIL opcode is compiled down to the idiv x86 opcode, which raises a hardware exception in case of overflow. A div opcode which does not throw an exception would be much slower, since it would have to be implemented by a series of x86 opcodes.

You're correct. I mistakenly believed that the x86 idiv instruction didn't throw an exception on overflow, but it does (it doesn't in real-mode, which is what I remembered). Still, it would be trivial in the x86 overflow exception handler to detect that it is handling this case and then resume execution after the idiv instruction with the correct results in the registers. This wouldn't make any performance difference for the common case, only the exceptional case would be a little slower.

Stuart commented:

Horrible hacky proposed solution: implement idiv using the broken div, but add an exception handler to every java method that uses idiv that tests for the pathological case and somehow resumes in the right place.

Two problems with this approach: 1) OverflowException can also be caused by trying to allocate a new array with a negative size, so I would somehow have to detect this. 2) Resuming after the exception is very hard (you cannot jump into a try block in .NET, so I'd have to generate lots of code, which would also make to common case slower).

Or just wrap map java (x/y) to try{x/y} catch(OverflowException e) {x}. How much execution overhead is there for a try-catch in the non-exception case?

This is an interesting suggestion. *writing some C# microbenchmarks* Here is the code I cooked up:

 public static void Main(string[] args)
  int d = args.Length + 1;
  int n = (args.Length - 1) * -1;
  int start = Environment.TickCount;
  for(int i = 0; i < 10000000; i++)
    n = n / d;
    n = int.MinValue;
  int end = Environment.TickCount;
  Console.WriteLine(end - start);
  start = Environment.TickCount;
  for(int i = 0; i < 10000000; i++)
   if(d == -1)
    n = n * d;
    n = n / d;
  end = Environment.TickCount;
  Console.WriteLine(end - start);
On my system there is almost no difference between these two methods, so I think I'll go for the explicit test, because of the additional complexity of moving the stack into the try block.
Thursday, August 1, 2002 10:43:40 AM (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
No Title

More very useful comments from Stuart:

I'm not sure what I think about your choice to not remap things like IOException, and to handle it in the pseudo-native code.

The principle of "least surprise" applies here. When Java code specifically calls .NET I/O code, it expect System.IO.IOException (because that is what the documentation says), and vice versa.

With regard to "throws Exception" in the generated JAR files, I'm not sure how I feel about it either. On the one hand, yes, it's certainly awkward for the Java programmer. On the other hand, it's exactly what the CLR really *does*, by implication - any method *can* throw any exception.

Here's a problem case: I could define a Java exception, use IK.VM to compile it to CIL, and then write a C# class that throws it and call that C# class from Java - how would you handle that case with your proposed solution? You can either preserve the "real" Java mapping of the compiled Java exception, OR you can make it a subclass of RuntimeException, but you can't do both. But you really do want to be able to preserve the semantics of a custom-compiled Java exception. Assuming "throws Exception" for all C# code seems to be the only way you *can* deal with this case, because you there's no way you can detect it to specialcase it.

You've conviced me, making all .NET methods declare "throws Throwable" (not Exception), is the only viable solution to this. I'm probably going to support a custom attribute to explicitly declare a throws clause in .NET code for newly written code, to make writing glue code a little less painful.

Yet another couple of semantics-impedance-mismatch issues that I just thought of: Do you plan to allow indexers and foreach to work on java.util.List and java.util.Collection, respectively? Will Collection implement(/extend) IEnumerable with Iterator remapping to IEnumerator? How about the converse ( for (Iterator i = new System.ArrayList().iterator(); i.hasNext(); ) {...} )? Or even, more complicatedly, {Input,Output}Stream and Reader/Writer mapping to .NET equivalents?

No, none of these. It's much easier to writer wrapper classes to handle these conversions than to have the VM handle it.

Will JavaBeans-compatible getFoo()/setFoo() methods remap to a property called foo? Or even to a property called Foo? (even the naming conventions are incompatible...)

Java "properties" are too semantically weak to be useful. There are many methods getXxx() where Xxx isn't a read-only property (Component.getGraphics(), for example) so I'm not going to do anything with them.

You'll notice that I'm posing ever more complicated scenarios - sorry! :)

The feedback is very helpful, thanks!

The reason I'm deliberately making your life difficult is that you seem to have all the easy cases well covered already...

I disagree ;-) The hardest part was figuring out how to do the basic object model mapping (like Sam Ruby said, Object, String, and Exception).

and besides, I'm highly interested to see just how deeply integrated into the .NET platform Java can really get without losing it's soul...

Me too. The priorities are the hard part, in many cases there is a trade off between performance and 100% compatibility, but a compatible JVM is a higher priority than interop with other .NET code (which isn't too say that isn't important).

Thursday, August 1, 2002 10:15:03 AM (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Wednesday, July 31, 2002
What's wrong with this picture?
class DivTest
 public static void Main()
  int i = int.MinValue;
  System.Console.WriteLine(i * -1);
  System.Console.WriteLine(i / -1);

Why does div throw an OverflowException when you're trying to divide MinInt by -1? I'm assuming that since dividing by zero throws a DivideByZeroException, the CLR designers thought it would be nice if div would throw an exception for overflow as well.

This sucks!

Partition III CIL.doc section 3.31 about div says:


Integral operations throw ArithmeticException if the result cannot be represented in the result type. This can happen if value1 is the maximum negative value, and value2 is -1.

Integral operations throw DivideByZeroException if value2 is zero.

Implementation Specific (Microsoft)

On the x86 an OverflowException is thrown when computing (minint div –1).

Why didn't they define a div.ovf (like there are add.ovf, sub.ovf, mul.ovf, etc.) in addition to div (and then make div behave consistently with add, submul)?

Question: What should I do? Implement Java's idiv bytecode using this broken div or compile them into conditional code that checks for MinInt / -1 and treats that specially (and thus slowing down integer division).

BTW, J# uses div but I'm not sure they actually thought about this issue. The following code crashes the J# compiler:

 System.out.println(Integer.MIN_VALUE / - 1);
Wednesday, July 31, 2002 4:14:23 PM (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]