# Thursday, 24 May 2012
New Development Snapshot

OpenJDK 7u4 has been integrated. Next stop RC.

Changes:

  • Integrated OpenJDK 7u4 b22.
  • Fixed threading bug in sun.font.PhysicalStrike (bug #3524226).
  • Fixed regression introduced in 7.0 that caused the manifest to be ignored when creating the java.lang.Package objects.
  • Updated boot class package property that still referred to Sun Microsystems instead of Oracle Corporation.

Binaries available here: ikvmbin-7.1.4527.zip

OpenJDK 7u4 b22 stripped sources: openjdk-7u4-stripped.zip

Thursday, 24 May 2012 11:04:15 (W. Europe Daylight Time, UTC+02:00)  #    Comments [1]
# Friday, 11 May 2012
Using MethodRental.SwapMethodBody to do Method Level JIT Compilation

IKVM.NET has always had a class granularity JIT. Whenever a type is first "used" the CLR fires the AppDomain.TypeResolve event and at that point the IKVM.NET runtime compiles the Java bytecode to CIL for all of the methods in the class.

I don't recall my exact thought process, but I assume that when I started on IKVM.NET I looked at MethodRental.SwapMethodBody and was scared away by the lack of documentation and the fact that it requires full trust and manually constructing a method body blob.

Later on, I focussed more on static compilation and didn't care too deeply about dynamic performance. So I never revisited this decision. However, recently I have been thinking about dynamic performance, triggered by potential invokedynamic optimizations.

To get reacquainted with MethodRental.SwapMethodBody I wrote a small program that dynamically creates the following class:

class Frob {
public Frob(int i) { Console.WriteLine(i); } public static void M(int i) { Console.WriteLine(); Console.WriteLine(new Frob(i)); } }

The code is available here: MethodRentalDemo.cs

When the constructor and the M method are first created, the method body is defined, using MethodBuilder.CreateMethodBody, as a simple trampoline that calls Program.JIT to just-in-time generate the actual CIL for the method.

Here's the managed JIT trampoline CIL code for Frob.M:

ldtoken    method void Frob::M(int32))
call       void Program::JIT(valuetype System.RuntimeMethodHandle)
jmp        void Frob::M(int32)

The jmp instruction is interesting, it transfers control to a method that takes the same arguments as the current method and passes the current argument values. Here it is used to jump to a new version the same method, where the method body has been replaced with the actual CIL code.

The native code that is generated for the trampoline is:

  x86 x64
 
push   ebp 
mov    ebp,esp 
sub    esp,8 
xor    eax,eax 
mov    dword ptr [ebp-8],eax 
mov    dword ptr [ebp-4],ecx 
lea    ecx,[ebp-8] 
mov    edx,6231A0h 
call   680065F0 
lea    eax,[ebp-8] 
push   dword ptr [eax] 
call   FFE39B70 
mov    ecx,dword ptr [ebp-4] 
mov    esp,ebp 
pop    ebp 
jmp    dword ptr ds:[006231A8h] 
 
push   rbx 
sub    rsp,20h 
mov    ebx,ecx 
lea    rcx,[00257330h] 
call   FFFFFFFFF35036D0 
mov    rcx,rax 
call   00000000001C84C0 
mov    ecx,ebx 
lea    rax,[00247D80h] 
add    rsp,20h 
pop    rbx 
jmp    rax
add    rsp,20h 
pop    rbx 
ret

(Note that the x64 JIT generates three unreachable instructions at the end. Shown in gray.)

When this code is invoked, it loads the method handle and calls the Program.JIT method. After that returns, it invokes the new method body that the JIT method installed using MethodRental.SwapMethodBody.

When run on the CLR* this all appears to work as you'd hope, but unfortunately that's no guarantee that it will work in the real world. Googling (and experience) suggests that there aren't many** users of MethodRental.SwapMethodBody, so it is quite possible that there are some showstoppers lurking somewhere.

* It does not work on Mono at the moment.
** I found one reference to it in a paper on RubySharp from 2004.

Friday, 11 May 2012 12:24:32 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Friday, 04 May 2012
New Development Snapshot

This should hopefully be the last snapshot based on OpenJDK 7 FCS as 7u4 has shipped and the bundle should be available soon. After that I'll start work on integrating 7u4 and working towards the IKVM 7.1 release.

Changes:

  • Bug fix. When adding certificates to virtual cacerts file make sure that the aliases that are generated from the certificate subject are unique.
  • Bug fix. If a class file UTF-8 string ends with an incomplete multi byte char, we should throw the appropriate exception.
  • IKVM.Reflection: Added ModuleBuilder.__GetAssemblyToken() API.
  • IKVM.Reflection: Removed Mono CryptoConvert code and use .NET 2.0 API instead.
  • IKVM.Reflection: Rewrote StrongNameKeyPair to remove dependency on System.Reflection.StrongNameKeyPair.
  • IKVM.Reflection: Improved assembly title and description.
  • IKVM.Reflection: Metadata is now sorted with a stable sort algorithm.

Binaries available here: ikvmbin-7.1.4507.zip

Friday, 04 May 2012 08:58:49 (W. Europe Daylight Time, UTC+02:00)  #    Comments [2]
# Monday, 23 April 2012
IKDASM - IL Disassembler Example for IKVM.Reflection

A while ago I wrote an IL disassembler to test IKVM.Reflection. Both as a correctness test and also to test if the API surface exposes enough of the underlying information.

I thought it would make a good IKVM.Reflection example (although the code won't win any awards, it's a bit of a hack). One nice feature is that it tries really hard to emit the same output file as the .NET ildasm to make comparison easier. There's even a command line option to match a specific ildasm version (2.0, 4.0 or 4.5) and its quirks.

The binaries are available in ikdasm-v0.1-binaries.zip and the Visual Studio 2010 solution in ikdasm-v0.1.zip.

Note that in its current form the ildasm compatibility mode only works on Windows, because it needs to P/Invoke _gcvt in msvcrt.dll to make sure the floating point numbers match the ildasm output.

Monday, 23 April 2012 16:54:40 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Thursday, 19 April 2012
New Development Snapshot

At Lang.NEXT I met someone who was interested in using IKVM.Reflection and after he started porting his code, he ran into some missing functionality in IKVM.Reflection, so I've made some improvements there.

I've also fixed the remaining known issues with access stubs and "unloadable" (missing types) custom modifiers.

Changes:

  • Fix for recently introduced bug (with access stub rewrite). Bug #3512589.
  • Changed build process to fall back to the NAnt task, if we can't find the resource compiler.
  • Made WinForms message loop thread creation lazy to hopefully allow more applications to run without message loop thread. This is a (partial) workaround for bug #3515033.
  • Changed ikvmc to read input files after processing all the options (to make -nowarn: and -warnaserror: options that follow the file names work for warnings produced during input file reading).
  • Added support for type 2 access stubs for constructors.
  • Bug fix. When an unloadable type is used in a method signature that overrides a method (or implements an interface method), the custom modifier must be the same as the base class or an override stub must be generated.
  • Added partial implementation of ThreadMXBean.
  • IKVM.Reflection: Bug fix. When writing an assembly that has a .netmodule, the TypeDefId field in the ExportedType in the manifest module should contain a TypeDef token instead of an index.
  • IKVM.Reflection: Bug fix. When exporting a nested type (via AssemblyBuilder.__AddTypeForwarder()), we should also set the namespace (in practice it is unlikely for a nested type to have a namespace, but is is possible).
  • IKVM.Reflection: Corrected a couple of method parameter names in Assembly.
  • IKVM.Reflection: Added Assembly.GetType(string,bool,bool) method.
  • IKVM.Reflection: Added support for case-insensitive type and member lookup.
  • IKVM.Reflection: Implemented case insensitive lookup in Type.GetInterface().
  • IKVM.Reflection: Moved GetEvents(), GetFields(), GetConstructors(), GetNestedTypes() and GetProperties() to a common implementation that fixes a number of bugs.
  • IKVM.Reflection: Fixed GetMethods() to properly filter out base class methods that have been overridden.
  • IKVM.Reflection: Moved member lookup by name to a common implementation that fixes a number of bugs and adds IgnoreCase support.
  • IKVM.Reflection: Added MemberInfo.ReflectedType.
  • IKVM.Reflection: Added Binder support for method and property lookup.
  • IKVM.Reflection: Bug fix. ParameterBuilder.Position should return the 1-based position passed in to DefineParameter, not the 0-based ParameterInfo.Position.
  • IKVM.Reflection: Changed Type.__ContainsMissingType to return true for generic type parameters that have constraints that return true for __ContainsMissingType.
  • IKVM.Reflection: Bug fix. It should be possible to import a function pointer type into a ModuleBuilder.
  • IKVM.Reflection: Implemented Assembly.ToString().
  • IKVM.Reflection: Added [Flags] attribute to ResourceLocation enum.
  • IKVM.Reflection: Added support for reading/querying manifest resources that are forwarded to another assembly.
  • IKVM.Reflection: Bug fix. Module.GetManifestResourceStream() should return null (instead of throwing FileNotFoundException) for non-existing resource names.

Binaries available here: ikvmbin-7.1.4491.zip

Thursday, 19 April 2012 09:26:43 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Thursday, 12 April 2012
MS12-025

This patch Tuesday Microsoft released MS12-025 that fixes approximately a zillion vulnerabilities in System.Drawing.dll.

Here's what they fixed (multiple instances of each issue):

  • Added security demands to unsafe methods.
  • Wrap handles in SafeHandle instead of using IntPtr.
  • Use checked to guard against integer overflow when calculating how many bytes to AllocHGlobal.

It's a little embarassing to have so many vulnerabilities in this old code. Luckily, silently running .NET code in the browser is becoming a thing of the past.

Thursday, 12 April 2012 12:33:30 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Friday, 06 April 2012
Lang.NEXT

I'm back home after a really great Lang.NEXT conference.

The video of my talk is available on Channel 9. The slides are here.

Friday, 06 April 2012 22:24:37 (W. Europe Daylight Time, UTC+02:00)  #    Comments [0]
# Monday, 26 March 2012
New Development Snapshot

Enough changes to warrant a new development snapshot.

Changes:

  • Fixes and improvements to Windows version of JFileChooser.
  • Bug fix. When writing a direct ByteBuffer to a non-blocking socket and the write fails because there is no kernel buffer available, we should not advance the ByteBuffer position.
  • Bug fix. Non-public property getter/setter methods should be ignored when we create properties to hide properties inherited from shadow types. This fixes a build break with .NET 4.5 beta which introduces a protected setter for Exception.HResult.
  • Bug fix. Custom attribute properties that don't have a public getter and setter should not be exposed as annotation properties.
  • Added ikvmc -win32manifest: option.
  • Added ikvmc -filealign: option.
  • Added ikvmc -highentropyva option (to enable high entropy ASLR in 64 bit processes on Windows 8).
  • Added support for custom paper format to Win32 print service.
  • Changed ikvmstub to create a missing assembly when a dependency is not found and only complain about it when it is actually needed.
  • Added explicit -help and -? options to ikvmc.
  • Added ikvmc -nologo option.
  • Changed ikvmc to print copyright when compiling (unless -nolog is specified).
  • Cleaned up to ikvmc help message.
  • Lots of ikvmc error handling clean up. All errors now have an IKVMCnnnn error code.
  • Added support to ikvmc to automatically set the full source path in the debugging info if the source file lives next to the .class file.
  • Improved BufferedImage.setRGB().
  • IKVM.Reflection: Added AssemblyBuilder.__DefineManifestResource() API to add a Win32 manifest resource.
  • IKVM.Reflection: Various win32 resource related methods on AssemblyBuilder now throw ArgumentException if a conflicting resource has already been defined.
  • IKVM.Reflection: Marked ModuleBuilder.__SetStackReserve() obsolete and made ModuleBuilder.__StackReserve property writeable to be consistent with __ImageBase property.
  • IKVM.Reflection: New API. Made ModuleBuilder.__FileAlignment writeable.
  • IKVM.Reflection: New API. Added Module.__DllCharacteristics and ModuleBuilder.__DllCharacteristics properties to get and set image DLL characteristics flags.

Binaries available here: ikvmbin-7.1.4468.zip

Monday, 26 March 2012 08:53:15 (W. Europe Daylight Time, UTC+02:00)  #    Comments [10]
# Friday, 16 March 2012
IKVM.Reflection: Inspecting an Assembly Without Loading Dependencies

One of the advantages of IKVM.Reflection over System.Reflection is that it is much easier to load an assembly without having to load its dependencies. This functionality is used by Mono's C# compiler to load referenced assemblies without having to (potentially) load the transitive closure of dependencies of those assemblies.

Here's a simple example that loads an assembly and prints its types and members:

using System;
using IKVM.Reflection;

class Program {
  const BindingFlags AllDeclared =
    BindingFlags.Public |
    BindingFlags.NonPublic |
    BindingFlags.Instance |
    BindingFlags.Static |
    BindingFlags.DeclaredOnly;

  static void Main(string[] args) {
    var universe = new Universe();
    universe.AssemblyResolve += AssemblyResolve;
    var assembly = universe.LoadFile(args[0]);
    foreach (var type in assembly.GetTypes()) {
      Console.WriteLine(type.FullName);
      WriteMembers(type.GetFields(AllDeclared));
      WriteMembers(type.GetProperties(AllDeclared));
      WriteMembers(type.GetEvents(AllDeclared));
      WriteMembers(type.GetConstructors(AllDeclared));
      WriteMembers(type.GetMethods(AllDeclared));
    }
  }

  static void WriteMembers(MemberInfo[] members) {
    foreach (var member in members)
      Console.WriteLine(" {0}", member);
  }

  static Assembly AssemblyResolve(object sender, IKVM.Reflection.ResolveEventArgs args) {
    return ((Universe)sender).CreateMissingAssembly(args.Name);
  }
}

When you don't handle the Universe.AssemblyResolve event (or non of the handlers return an assembly), IKVM.Reflection will fall back to using System.Reflection.Assembly.ReflectionOnlyLoad() and then use the Location property of the returned assembly and load the assembly from there.

If you do handle the Universe.AssemblyResolve event, you can either find the assembly somewhere and load it, or use Universe.CreateMissingAssembly() to create a placeholder Assembly that will automatically resolve all types and members that it is expected to contain (based on references from loaded assemblies that are inspected).

When you're reflecting over the members you can encounter types and members from the missing assemblies and you should be prepared to handle these because many operations are not allowed on them (since there is no type or member definition, a lot of information simply isn't available). You can detect these missing types or members by looking at the MemberInfo.__IsMissing property.

Friday, 16 March 2012 13:26:07 (W. Europe Standard Time, UTC+01:00)  #    Comments [2]
# Thursday, 15 March 2012
Lesser Known CLR Custom Attributes -- UnsafeValueType

In a comment to the previous post about CLR Custom Attributes I listed some other custom attributes that the CLR recognizes (by name). Some of them I previously thought were compiler only custom attributes, so I decided to investigate them.

System.Runtime.CompilerServices.UnsafeValueTypeAttribute

The documentation for this attribute, somewhat uncharacteristically, actually explains what it does, but I decided to try it out.

Here's an example that demonstrates what it does:

using System;
using System.Runtime.CompilerServices;

//[UnsafeValueType]
struct Foo {
  public int field;
}

class Program {
  [MethodImpl(MethodImplOptions.NoOptimization)]
  static void Main() {
    int i = 1234;
    Foo foo = new Foo();
    Corrupt(ref foo, ref i);
    Console.WriteLine(i);
  }

  [MethodImpl(MethodImplOptions.NoInlining)]
  unsafe static void Corrupt(ref Foo foo, ref int unused) {
    fixed (int* p = &foo.field) {
      *(long*)p = 4567L << 32;
    }
  }
}

When you run this it prints out 4567 and terminates successfully. However, when you uncomment the //[UnsafeValueType] line and then run it again, you'll see that it prints out 1234 and crashes and if you attach a debugger you see that it crashes with error code STATUS_STACK_BUFFER_OVERRUN because the CLR inserted a canary on the stack after the unsafe value type.

As the documentation indicates, both the C++ and C# compiler use this attribute. The C++ compiler uses it to implement /GS for managed code and the C# compiler automatically applies it to the value types that it creates to represent fixed size buffers.

Thursday, 15 March 2012 09:42:27 (W. Europe Standard Time, UTC+01:00)  #    Comments [0]