# Tuesday, 06 August 2002
No Title

I wrote a jasmin class that "uses" all bytecode instructions, to check if I had implemented all of them. It turns out three were missing:

  • jsr_w  -- jump to subroutine using a 32 bit offset (methods have a maximum size of 64KB, so why we need this, I don't know)
  • goto_w -- jump to instruction using a 32 bit offset (see jsr_w)
  • dup2_x2 -- scramble the contents of the stack ;-)

I implemented all three, although I very much doubt that they'll ever be used.

In the process of doing this, I found and fixed a few bugs in the verifier and compiler. I also changed the compiler to generate a "throw VerifyError("...")" when compiling a method that doesn't verify (instead of throwing the exception at compile time). This matches the JVM behaviour better.

Updated snapshot is here.

Tuesday, 06 August 2002 14:09:00 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Friday, 02 August 2002
No Title

Stuart commented:

Btw, I'm just trying out your latest snapshot. I'll add further comments as I get further, but so far I had to make a couple of changes to let me build:

VS.NET treated "if (false)" on line 1732 of vm.cs as creating unreachable code. I changed it to "if (false.Equals(true))" so that it didn't treat it as a compiletime constant.

I'm used to using the if(false) construct, because that's the way to not get the compiler to complain in Java. Anyway, I just commented it out now, it's only there for debugging. So it will go away in the near future, I hope.

And the default compilation options disallowed unsafe code, so I had to change that in order for BigEndianBinaryReader.cs to compile.

Aren't you using my VS.NET project? That should include the proper switches for it to build, at least it builds on my machine ;-)

Okay, I've got it to the point of giving me an intelligent error at runtime, at least. Issues so far:

It doesn't (seem to) honor CLASSPATH, at least when run from a command prompt.

That is correct, for the time being it uses the IKVM_CLASSPATH environment variable. BTW, zips and jars are not supported (and won't be for a while, because I want to write the Zip/Jar class loader in Java. That isn't going to happen until after I've got the ahead-of-time compiler working).

Now the showstopper that I haven't been able to figure out how to get past. As far as I can see the download doesn't include *any* of the GNU Classpath libraries, which means that (as far as I can see) it can't run anything at all. I'm guessing that this is just a simple omission from the zip - is that right?

I don't include the classpath stuff to keep the zip small. I did earlier post a link to the compiled classpath classes that I use. I've created a new snapshot (many small changes) and also posted a zip containing the classpath classes plus a few of my own modifications (the source for those is in the classes directory in ikvm.zip).

Friday, 02 August 2002 10:12:28 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Thursday, 01 August 2002
No Title

Stuart commented:

On most aspects, you've convinced me. Just a few little comments:

wrt "throws Exception" vs "throws Throwable" - there's no difference. Any Java method can throw any Throwable subclass that isn't an Exception subclass, just as any Java method can throw any RuntimeException subclass. Thus "throws Exception" really does let you throw anything.

Actually, no. Try to compile the following:

class test extends Throwable
public static void main(String[] args) throws Exception
throw new test();

It doesn't compile. One of the inellegancies of Java's checked exceptions model is that there are two classes that removed the checkedness, both Error and RuntimeException.

Btw, are you arranging that (most? any?) .NET exceptions end up as subclasses of j.l.Exception, or just direct Throwable subclasses?

Throwable, the most important reason is that I don't want to mess with the class hierarchy, if at all possible.

I agree also that remapping things like InputStream is probably too hard. If you have a generic class/interface/method remapping system, though, I can imagine actually experimenting with Collection vs IEnumerable and indexers on List.

One of the goals of XML remapping system, is to enable exactly these types of experiments. It probably isn't flexible enough to do everything you'd want to do, but in the end we'll get there.

When you get IK.VM working on Mono I might even have a try at that myself, since it shouldn't need any deep magic in the VM itself.

I think it'll still be a while before Reflection.Emit in Mono is at the level where it is complete enough.

Oh, and I like the idea of custom attributes to do "throws" expressions on glue code. I wonder if we could persuade the Mono people to get mcs to actually honor the "throws" attributes - ie, compile C# code with Java's 'checked exceptions' semantics. That would be pretty cool :)

It would be cool, but I don't really like checked exceptions. After programming in Java (almost exclusively) for about five years, I finally decided that the cons outnumber the pros. The idea is compelling, but in practice there are too many problems, but I don't really want to start that war, as it has been waged many times already ;-)

BTW, I just changed the idiv, ldiv, irem & lrem implementations to explicity check for -1, and for a lame little test I wrote, performance seems to be OK (same as JDK 1.1 and slightly faster than JDK 1.4, HotSpot doesn't like microbenchmarks ;-)).

Thursday, 01 August 2002 17:14:02 (W. Europe Daylight Time, UTC+02:00)  #    Comments [3]
No Title

Zoltan commented on the div issue:

I believe the reason the CLR throws an exception in this case is that the div CIL opcode is compiled down to the idiv x86 opcode, which raises a hardware exception in case of overflow. A div opcode which does not throw an exception would be much slower, since it would have to be implemented by a series of x86 opcodes.

You're correct. I mistakenly believed that the x86 idiv instruction didn't throw an exception on overflow, but it does (it doesn't in real-mode, which is what I remembered). Still, it would be trivial in the x86 overflow exception handler to detect that it is handling this case and then resume execution after the idiv instruction with the correct results in the registers. This wouldn't make any performance difference for the common case, only the exceptional case would be a little slower.

Stuart commented:

Horrible hacky proposed solution: implement idiv using the broken div, but add an exception handler to every java method that uses idiv that tests for the pathological case and somehow resumes in the right place.

Two problems with this approach: 1) OverflowException can also be caused by trying to allocate a new array with a negative size, so I would somehow have to detect this. 2) Resuming after the exception is very hard (you cannot jump into a try block in .NET, so I'd have to generate lots of code, which would also make to common case slower).

Or just wrap map java (x/y) to try{x/y} catch(OverflowException e) {x}. How much execution overhead is there for a try-catch in the non-exception case?

This is an interesting suggestion. *writing some C# microbenchmarks* Here is the code I cooked up:

 public static void Main(string[] args)
  int d = args.Length + 1;
  int n = (args.Length - 1) * -1;
  int start = Environment.TickCount;
  for(int i = 0; i < 10000000; i++)
    n = n / d;
    n = int.MinValue;
  int end = Environment.TickCount;
  Console.WriteLine(end - start);
  start = Environment.TickCount;
  for(int i = 0; i < 10000000; i++)
   if(d == -1)
    n = n * d;
    n = n / d;
  end = Environment.TickCount;
  Console.WriteLine(end - start);
On my system there is almost no difference between these two methods, so I think I'll go for the explicit test, because of the additional complexity of moving the stack into the try block.
Thursday, 01 August 2002 10:43:40 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
No Title

More very useful comments from Stuart:

I'm not sure what I think about your choice to not remap things like IOException, and to handle it in the pseudo-native code.

The principle of "least surprise" applies here. When Java code specifically calls .NET I/O code, it expect System.IO.IOException (because that is what the documentation says), and vice versa.

With regard to "throws Exception" in the generated JAR files, I'm not sure how I feel about it either. On the one hand, yes, it's certainly awkward for the Java programmer. On the other hand, it's exactly what the CLR really *does*, by implication - any method *can* throw any exception.

Here's a problem case: I could define a Java exception, use IK.VM to compile it to CIL, and then write a C# class that throws it and call that C# class from Java - how would you handle that case with your proposed solution? You can either preserve the "real" Java mapping of the compiled Java exception, OR you can make it a subclass of RuntimeException, but you can't do both. But you really do want to be able to preserve the semantics of a custom-compiled Java exception. Assuming "throws Exception" for all C# code seems to be the only way you *can* deal with this case, because you there's no way you can detect it to specialcase it.

You've conviced me, making all .NET methods declare "throws Throwable" (not Exception), is the only viable solution to this. I'm probably going to support a custom attribute to explicitly declare a throws clause in .NET code for newly written code, to make writing glue code a little less painful.

Yet another couple of semantics-impedance-mismatch issues that I just thought of: Do you plan to allow indexers and foreach to work on java.util.List and java.util.Collection, respectively? Will Collection implement(/extend) IEnumerable with Iterator remapping to IEnumerator? How about the converse ( for (Iterator i = new System.ArrayList().iterator(); i.hasNext(); ) {...} )? Or even, more complicatedly, {Input,Output}Stream and Reader/Writer mapping to .NET equivalents?

No, none of these. It's much easier to writer wrapper classes to handle these conversions than to have the VM handle it.

Will JavaBeans-compatible getFoo()/setFoo() methods remap to a property called foo? Or even to a property called Foo? (even the naming conventions are incompatible...)

Java "properties" are too semantically weak to be useful. There are many methods getXxx() where Xxx isn't a read-only property (Component.getGraphics(), for example) so I'm not going to do anything with them.

You'll notice that I'm posing ever more complicated scenarios - sorry! :)

The feedback is very helpful, thanks!

The reason I'm deliberately making your life difficult is that you seem to have all the easy cases well covered already...

I disagree ;-) The hardest part was figuring out how to do the basic object model mapping (like Sam Ruby said, Object, String, and Exception).

and besides, I'm highly interested to see just how deeply integrated into the .NET platform Java can really get without losing it's soul...

Me too. The priorities are the hard part, in many cases there is a trade off between performance and 100% compatibility, but a compatible JVM is a higher priority than interop with other .NET code (which isn't too say that isn't important).

Thursday, 01 August 2002 10:15:03 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Wednesday, 31 July 2002
What's wrong with this picture?
class DivTest
 public static void Main()
  int i = int.MinValue;
  System.Console.WriteLine(i * -1);
  System.Console.WriteLine(i / -1);

Why does div throw an OverflowException when you're trying to divide MinInt by -1? I'm assuming that since dividing by zero throws a DivideByZeroException, the CLR designers thought it would be nice if div would throw an exception for overflow as well.

This sucks!

Partition III CIL.doc section 3.31 about div says:


Integral operations throw ArithmeticException if the result cannot be represented in the result type. This can happen if value1 is the maximum negative value, and value2 is -1.

Integral operations throw DivideByZeroException if value2 is zero.

Implementation Specific (Microsoft)

On the x86 an OverflowException is thrown when computing (minint div –1).

Why didn't they define a div.ovf (like there are add.ovf, sub.ovf, mul.ovf, etc.) in addition to div (and then make div behave consistently with add, submul)?

Question: What should I do? Implement Java's idiv bytecode using this broken div or compile them into conditional code that checks for MinInt / -1 and treats that specially (and thus slowing down integer division).

BTW, J# uses div but I'm not sure they actually thought about this issue. The following code crashes the J# compiler:

 System.out.println(Integer.MIN_VALUE / - 1);
Wednesday, 31 July 2002 16:14:23 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
No Title

I only just noticed that Stuart commented on the July 25 item:

The natural "con" that I can think of regarding your alternative implementation is that it would make it very difficult for Java code to catch System.Exception. At least it would require that code that wants to catch *any* exception would have to know that it might be running under IK.VM and explicitly catch System.Exception, where a normal Java program might expect to be able to catch Throwable.

I didn't mention this, but I would still map all usages of java.lang.Throwable to System.Exception (except in the case of the base class, when a class is derived from Throwable, it would be derived from Throwable). Sensible interop between .NET and Java exceptions is definitely something I'm aiming for.

The difficulty, it seems to me, is ensuring that you don't just have a good mapping from Java to C#, but that your mapping is fully *reversible*, and makes just as much sense going the other way. Java code expects to know exactly what exceptions are going to happen for a particular method, so you'll need to be remapping things like IOException already - and you'll need some way to ensure that any exception that's going to be thrown by a method where Java doesn't expect it gets mapped to a RuntimeException (or Error or direct Throwable subclass) with the original exception as the cause.

I'm not actually remapping exceptions like IOException, what happens is that the "native" code that implements an I/O function catches the System.IO.IOException and throws a java.io.IOException.

CLR generated exceptions get converted to the Java equivalent when they are first caught. At the moment I don't convert them back, so when .NET code calls Java code it can expect both System.NullReferenceException and java.lang.NullPointerException to be thrown (for example) depending on whether the Java code "caught" (a finally also triggers the conversion) the exception before it was propagated out the calling .NET code. This isn't very elegant, so I expect that in the future I'll be swapping the original exception back when the Java code rethrows the exception.

I suppose that your generated JAR files (described in your previous entry in response to my comment) will have to declare every method as "throws Exception", right?

To be honest, I hadn't even thought about this. My initial thought would be to make it seem (for the Java compiler) as if every .NET exception is derived from RuntimeException to get around this mess. This might cause additional complications, so I'm going to have to think about this a little more. Adding "throws Exception" doesn't appeal to me either, because that would make it very uncomfortable for the Java programmer trying to call .NET code.

Wednesday, 31 July 2002 15:29:13 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Monday, 29 July 2002
No Title

I found an interesting paper on jsr/ret verification. It contains the following example of valid Java code that is not verifiable with the verification algorithm described in the JVM spec:

static int m(boolean x) {
  int y;
  try {
    if(x) return 1;
    y = 2;
  } finally {
    if(x) y = 3;
  return y;
Monday, 29 July 2002 18:12:48 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Saturday, 27 July 2002
No Title

I finally implemented the local variable usage tracking for subroutines in the verifier. In this instance Microsoft certainly did learn from Sun's mistakes. The JVM verifier is ridiculously complex because of two features:

  • jsr & ret instructions (subroutines within a method)
  • untyped local variables

The CLR lacks both of those features. The reason Sun introduced subroutines is because code in a finally block needs to run both in the normal case and in the exception case, there are two ways you can compile this (Java's bytecode has no concept of finally blocks): code duplication or the subroutine construct. Microsoft solved this by explicitly supporting finally blocks in MSIL, a far more elegant solution. Having typed local variables really doesn't have any downside (except that you have slightly more metadata to carry around).

Updated the snapshot.

Saturday, 27 July 2002 10:51:05 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Thursday, 25 July 2002
More java.lang.Throwable

I did some more work on Throwable. In my previous post I forgot to mention that virtual methods in Throwable wheren't overidable yet (because they have no equivalent in System.Exception, I cannot reuse those vtable slots, like I do for System.Object). I implemented a mechanism to support those now. Every direct subclass of Throwable is converted into a .NET subtype of System.Exception and it implements the Throwable$VirtualMethods interface. Each virtual invocation of a virtual method in Throwable is redirected to a static method in the class Throwable$VirtualMethodsHelper that checks if the type implements the Throwable$VirtualMethods interface, if it does, the method is routed there, if it doesn't the method is routed to the C# implementation of the Throwable method in ExceptionHelper.

BTW, the reason that the static helper methods are in a separate class, is because Reflection.Emit (incorrectly) doesn't allow the definition of static methods in interfaces.

The cool part of all this is that all of this virtual method handling is generic and completely based on the remappings defined in map.xml.

I also implemented the 1.4 stack trace handling, so Throwable is now fully functional.

BTW#2, there is also an alternative way I could have handled the virtual methods, by making Throwable a subtype of System.Exception. I haven't figured out the pros and cons of this, so may be I'll do the alternative implementation in a few days to see if that is a more elegant way of doing it.

Updated the snapshot.

Thursday, 25 July 2002 18:16:31 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]