# Monday, 31 March 2003
Sockets
I completed most of the Socket code and fixed a few bugs. Mauve results (with current Classpath CVS code): 172 of 7707 tests failed.

I updated to BlogX rev 20 and this means that the RSS will now contain xhtml:body (for the new entries). Chris says: "So, if you favorite aggregator isn't showing the full text of my blog, send them a mail and get them to support <xhtml:body>!"

I've updated the source and binaries snapshots
Monday, 31 March 2003 17:06:44 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Wednesday, 19 March 2003
System.arraycopy

At first sight, it might seem that System.arraycopy is not a very interesting method, but once you dig into it, it turns out to be an interesting case study.

.NET has a very similar method Array.Copy and one of its overloads has the exact same signature as its Java cousin, but throws slightly different exceptions and there is a tiny difference in behavior. So an initial naïve implementation of System.arraycopy would look as follows:

public static void arraycopy(object src, int srcStart,
       object dest, int destStart, int length) {
    try {
        Array.Copy(src, srcStart, dest, destStart, length);
    } catch(ArgumentNullException) {
        throw new NullReferenceException();
    } catch(ArgumentException) {
        throw new ArrayIndexOutOfBoundsException();
    } catch(ArrayTypeMismatchException) {
        throw new ArrayStoreException();
    } catch(InvalidCastException) {
        throw new ArrayStoreException();
    }
}

This works fairly well, but it has two problems. The first problem is that it is too powerful. Array.Copy allows widening conversions for primitive types. This means, for example, that it is possible to copy a byte array to an int array. Java doesn’t allow this. The second problem is more subtle, Javadocs for System.arraycopy say:

“Otherwise, if any actual component of the source array from position srcPos through srcPos1 cannot be converted to the component type of the destination array by assignment conversion, an ArrayStoreException is thrown. In this case, let k be the smallest nonnegative integer less than length such that src[srcPosk] cannot be converted to the component type of the destination array; when the exception is thrown, source array components from positions srcPos through srcPosk-1 will already have been copied to destination array positions destPos through destPosk-1 and no other positions of the destination array will have been modified.”

In other words, if an ArrayStoreException is thrown, all elements prior to the one that failed are copied. In contrast to this, the .NET documentation for Array.Copy says:

“If this method throws an exception while copying, the state of destinationArray is undefined.”

So to be strictly correct, we cannot rely on Array.Copy to implement copying of reference arrays that might throw an exception during copying. Here is a better implementation of System.arraycopy:

public static void arraycopy(object src, int srcStart,
       object dest, int destStart, int length) {
    // fast path if source and destination are same
    if(src != dest) {
        Type type_src = src.GetType();
        Type type_dst = dest.GetType();
        // fast path if arrays are of identical type
        
if(type_src != type_dst) {
            if(len < 0) {
                throw newArrayIndexOutOfBoundsException();
            }
            try {
                Object[] src1 = (Object[])src;
                Object[] dst1 = (Object[])dest;
                for(; len > 0; len--) {
                    dst1[destStart++] = src1[srcStart++];
                }
            } catch(InvalidCastException) {
                throw JavaException.ArrayStoreException();
            }
        }
    }
    try {
        Array.Copy((Array)src, srcStart,
                   (Array)dest, destStart, length);
    } catch(ArgumentNullException) {
        throw new NullReferenceException();
    } catch(ArgumentException) {
        throw new ArrayIndexOutOfBoundsException();
    } catch(InvalidCastException) {
        throw new ArrayStoreException();
    }
}

I think this is a correct implementation of System.arraycopy. It’s still not perfect though, because it turns out to be really slow. Here is a small benchmark:

class arraycopy {
    public static void main(String[] args){
        test(10, 10000000);
        test(100, 1000000);
        test(1000, 100000);
    }

    private static void test(int size, int count) {
        int[] a = new int[size];
        int[] b = new int[size];
        long start = System.currentTimeMillis();
        for(int i = 0; i < count; i++) {
            System.arraycopy(a, 0, b, 0, a.length);
        }
        long end = System.currentTimeMillis();
        System.out.println(end - start);
    }
}

Here are the results for the JDK 1.1 Symantec JIT, JRE 1.4.1 HotSpot Client VM and IKVM running on .NET 1.0:

Symantec

JRE 1.4.1

IKVM

10

1012

2343

12127

100

340

531

1402

1000

120

310

340

It’s obvious that for small arrays, the overhead is really large. The reason this is so much slower than the Java version is probably because of the cost of the call to GetType combined with the extra checking going on inside Array.Copy. If Sun had only provided overloaded versions of System.arraycopy for each type of primitive array, we wouldn’t need to do so much type checking! It turns out that even without these overloads we can still get the same result (in many cases), because the verifier often knows the types of the arguments. In compiles.cs I added some code to the invokestatic handler to check for invocations of System.arraycopy and if the array arguments are known to be primitive arrays, the call gets redirected to a more efficient arraycopy implementation.

Here is where the call to System.arraycopy in the above benchmark gets redirected to:

public static void arraycopy_primitive_4(Array src,
       int srcStart, Array dest, int destStart, int len) {

    try {

        checked {

            Buffer.BlockCopy(src, srcStart << 2,
                       dest, destStart << 2, len << 2);

            return;

        }

    } catch(ArgumentNullException) {

        throw new NullReferenceException();

    } catch(OverflowException) {

        throw new ArrayIndexOutOfBoundsException();

    } catch(ArgumentException) {

        throw new ArrayIndexOutOfBoundsException();

    }

}

No more calls to GetType and no more casts to Array are required. In addition, since we know that the arrays are both of the same primitive type, we can use the more efficient Buffer.BlockCopy instead of Array.Copy.

Here are the performance results for the new implementation:

Symantec

JRE 1.4.1

IKVM

10

1012

2343

1713

100

340

531

381

1000

120

310

230

Obviously a worthwhile improvement. For completeness, here are the performance results for doing the arraycopy with a simple for loop (i.e. in a modified version of the benchmark that doesn't call System.arraycopy , but instead has simple simple method that copies the array using a for loop):

for(; len > 0; len--)
    src[srcStart++] = dest[destStart++];

Symantec

JRE 1.4.1

IKVM

10

1853

2724

1522

100

1542

2444

1162

1000

1532

2363

1131

Wednesday, 19 March 2003 18:00:14 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Monday, 17 March 2003
Cobol

On the ikvm-developers list, Brian Sullivan reported that he had partial success with running a Cobol application (compiled with PERCobol to a Java jar) under ikvm.

The exception was caused by some missing org.xml.sax.* classes.

In unrelated news, I checked in several changes and made new source and binaries snapshots.

Monday, 17 March 2003 15:20:43 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Thursday, 13 March 2003
Welcome on the other side

Like many before me, I finally grew tired of Radio and so I've migrated to BlogX by Chris Anderson. All I need now is an rss aggregator, preferably ASP.NET based.

Thursday, 13 March 2003 14:28:06 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Monday, 03 March 2003
Hello Mono (2)
C:\>mono --noinline --share-code c:\ikvm\bin\ikvm.exe hello
** Warning **: cannot find C:\cygwin\home\lalo\go-mono\install\
etc\mono\machine.config
Trying to load app config file...
Hello World

The latest ikvm binaries together with Mono 0.21 now run Hello World!

Many thanks to everyone at Ximian, the Mono contributors and especially Zoltan Varga!

The --noinline and --share-code options are needed to work around a bug in the current Mono JIT that cause it to call the class constructors to eagerly.

I've updated the source and binaries snapshots.

Monday, 03 March 2003 11:57:22 (W. Europe Standard Time, UTC+01:00)  #    Comments [2]
# Friday, 28 February 2003
Undefined behavio[u]r

While fixing bugs in ikvm to get more Mauve tests working (BTW, current results: 224 of 7584 tests failed), I ran across a small but interesting difference between Java and C# (and the underlying bytecodes) in converting floating point numbers to integers.

Java code:

class Test {
  public static void main(String[] args) {
    float f = Integer.MIN_VALUE;
    f -= 5;
    System.out.println((
int)(f + 0.5f));
  }
}

Java output: -2147483648 (= Integer.MIN_VALUE)

C# code:

class Test {
  static void Main() {
    float f = int.MinValue;
    f -= 5;
    System.Console.WriteLine((
int)(f + 0.5f));
  }
}

C# output (release build): 2147483644
C# output (debug build): -2147483647

It turns out that the CIL instruction conv.i4 returns an undefined value when the float value lies outside of the range representable by an 32 bit integer, unlike the JVM's f2i instruction which is defined to return either Integer.MIN_VALUE or Integer.MAX_VALUE in that case.

If you want your .NET code to run consistently, use the conv.ovf.i4 instruction that checks for overflow. In C# this can be done by using a checked block:

class Test {
  static void Main() {
    float f = int.MinValue;
    f -= 5;
    checked {
      System.Console.WriteLine((
int)(f + 0.5f));
    }
  }
}

Now, instead of returning an undefined value, the cast throws a System.OverflowException.

The JVM designers felt it was very important not to have any undefined or implementation defined behavior. One of the lessons learned from C and C++ is that whenever there is undefined or implementation defined behavior, code will be written that depends on the behavior of a particular platform/compiler. The JVM designers wanted to removed this source of portability problems.

However, they payed a price in performance for this. A well known example is the case of floating point performance (on x86), which was later "fixed" by relaxing the floating point specification and introducing strictfp. As Morpheus would say: "Welcome to the real world!" (Don Box claims everything in computing can be understood by watching The Matrix enough times).

Let's examine the performance of Java's f2i compared with .NET's conv.i4. Please note that the usual disclaimer wrt (micro) benchmarking applies.

Here is the loop I used:

float f = SOME_VALUE;
for(int i = 0; i < 10000000; i++) {
  int j = (int)f;
  f = j;
}

Timings:

 

 

SOME_VALUE

 

 

0

+Infinity

Sun JDK 1.1 (Symantec JIT)

Float

600 ms

2000 ms

Double*

800 ms

2300 ms

Sun J2RE 1.4.1 (Hotspot Client VM)

Float

600 ms

3800 ms

Double*

1100 ms

2600 ms

.NET 1.0

Float

500 ms

500 ms

Double*

800 ms

800 ms

.NET 1.0

(checked)

Float

800 ms

n/a

Double*

1200 ms

n/a

* For the double test, the value was cast to a long instead of an int.

Let's look at the code that the Symantec JIT uses to convert a float to an int:

01F543F0  ftst             
01F543F2  fldcw       word ptr ds:[1F5FB30h]
01F543F8  push        eax 
01F543F9  fistp       dword ptr [esp]
01F543FC  fldcw       word ptr ds:[1F5FB34h]
01F54402  pop         eax 
01F54403  cmp         eax,80000000h
01F54408  je          01F5440B
01F5440A  ret             
01F5440B  push        eax 
01F5440C  fnstsw      ax  
01F5440E  sahf            
01F5440F  pop         eax 
01F54410  jp          01F54416
01F54412  adc         eax,0FFFFFFFFh
01F54415  ret             
01F54416  xor         eax,eax
01F54418  ret             

When the JIT compiles an f2i bytecode, it emits a call to this function. I've never written any x87 code, so I'm going to have to make this up as I go. Let's look at each individual instruction:

01F543F0  ftst 

No idea what the purpose of this is.

01F543F2  fldcw       word ptr ds:[1F5FB30h] 

Presumably this is used to mask the invalid-arithmetic-operand exception (#IA) that is generated by FISTP when it encounters a value that cannot be represented as a 32 bit integer.

01F543F8 push eax 

Make room on the stack for the integer.

01F543F9 fistp dword ptr [esp] 

Convert value on the floating stack to integer and store on the regular stack in slot we just created.

01F543FC fldcw word ptr ds:[1F5FB34h] 

Restore the FPU control word to its normal Java setting.

01F54402 pop eax 

Load the integer we just created into EAX.

01F54403 cmp eax,80000000h

Is it the integer indefinite value? When the #IA exception is masked FISTP returns the integer indefinite value for floats that cannot be represented as a 32 bit integer.

01F54408 je 01F5440B 

If it was the integer indefinite value, continue executing at 01F5440B.

01F5440A ret 

If not, return.

01F5440B push eax 

Save the integer indefinite value.

01F5440C fnstsw ax 

Load the FPU status flags in AX.

01F5440E sahf 

Load AH into the CPU status flags.

01F5440F pop eax 

Recover the integer indefinite value.

01F54410 jp 01F54416 

If the parity flag is set, the original float was a NaN, so we jump to the code that clears EAX and returns.

01F54412 adc eax,0FFFFFFFFh 

If the carry flag is set, leave EAX unchanged (i.e. add zero) otherwise add -1. This is a clever way of turning 0x80000000 into either 0x80000000 or 0x7FFFFFFF.

01F54415 ret 

Return to the caller.

01F54416 xor eax,eax 
01F54418 ret

Set EAX to zero and return to the caller.

After analyzing this code, it's kind of surprising that the "exceptional" case (when the float lies outside of the representable range) is so much slower.

Conclusion: As usual there is no conclusion, but hopefully we learned something today ;-)

Friday, 28 February 2003 11:56:00 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Saturday, 15 February 2003
Mauve

Zoltan has been working on running the Mauve testsuite on IKVM.NET running on Mono  and I've been doing the same on MS .NET.

Current status on MS .NET: 298 of 7338 tests failed
Current status on Mono: 143 of 3996 tests failed

Thanks to Mark for getting me started with Mauve and thanks to Zoltan for his excellent work on getting ikvm running on Mono.

Saturday, 15 February 2003 19:10:50 (W. Europe Standard Time, UTC+01:00)  #    Comments [2]
# Wednesday, 12 February 2003
Getting Eclipse to run

In the comments of the previous item John asked for specific instructions to get Eclipse running.

  1. Download the most recent IKVM binaries (I just updated them).
  2. Download Eclipse. I use build M3.
  3. Unzip both zip files. Here I will assume that both are unzipped in the root of C:\ (the ikvm zip creates an ikvm directory and all the Eclipse files end up in an eclipse directory)
  4. Download eclipse.bat, save it in the eclipse directory.
  5. Open a Command Prompt window and cd into the eclipse directory and run eclipse.bat

This should do the trick. If you have any problems, please let me know.

 

Wednesday, 12 February 2003 18:19:47 (W. Europe Standard Time, UTC+01:00)  #    Comments [5]
Finally back

I came across a class file that was the equivalent of the following source:

class Test
{
    public static final int FOO = 1;

    static {
        FOO = 1;
    }
}

This isn't legal Java, but the class file equivalent is. The FOO field has a ConstValue attribute with the value 1 and then there is code in the static initializer to set the value again. The code in the static initializer isn't needed and the Java compilers I've seen so far don't emit it.

Anyway, IKVM handles assignments to (non-blank) final fields by just ignoring the assignment, but my code generator emitted a nop instruction, instead of a pop (because it should consume the value on the stack). Fixed.

GNU Classpath is about to release version 0.05, so I got their code from cvs and updated my native methods to work with the latest code (the only changes required were for Object[In][Out]putStream, because Mark cleaned those up to use less native code, a nice improvement!). There was still one remaining issue with compiling the classpath code with ikvmc, I had to comment out a line of code in java/nio/charset/Charset.java:

  public final ByteBuffer encode (String str)
  {
    return encode (CharBuffer.wrap (str));
  }

CharBuffer.wrap takes an CharSequence as its argument, but my java.lang.String doesn't implement CharSequence (yet). It occurred to me that since it is legal for any reference type to be passed where an interface is expected (see here) this code was legal as well (even if String doesn't implement CharSequence), so I added support to the compiler to insert casts when the arguments on the stack do not match with the expected method arguments (but only for interface references).

Finally, there is still one patch required to Classpath, because new File("c://") hangs:

RCS file: /cvsroot/classpath/classpath/java/io/File.java,v
retrieving revision 1.21
diff -r1.21 File.java
334,335c334
< if (!PlatformHelper.isRootDirectory(path))
< while (PlatformHelper.endWithSeparator(path))
---
> while (!PlatformHelper.isRootDirectory(path) &&
> PlatformHelper.endWithSeparator(path))

You wouldn't expect this to be a common occurrence, but it turns out that this exact path is constructed by the code that computes the current directory, so if you use ikvm to run a Java application in the root directory of a drive it hangs (without this patch).

 

Wednesday, 12 February 2003 13:46:52 (W. Europe Standard Time, UTC+01:00)  #    Comments [2]
# Friday, 17 January 2003
Hello Mono

Zoltan Varga wrote on the ikvm-developers list:

   After lots of hacking, I managed to get IKVM to run HelloWorld under mono. This involved lots of changes/bugfixes to IKVM, the mono runtime and the class libs. I intend to submit a big patch to the mono mailing list shortly with the changes.

Great news!

Unrelated to the above, I checked in a bunch of changes and updated the source and binary snapshots. I'll be out of the country for two weeks, so there probably won't be much activity.

Friday, 17 January 2003 22:04:28 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]