A Fun Windows Hack (Part Two)

If you recall from last time, I showed you the in-memory layout of a COM object. It looked like this:

I had asked you if you could think of a way to emulate this in REALbasic.

If you had said "with MemoryBlocks, Modules and the AddressOf operator", then you were absolutely correct.

The ClassPtr is the COM object itself. It's simply a MemoryBlock whose first four bytes points to a vTable of interface functions. Since the caller has already defined the order of the functions, you're able to take advantage of this and lay the function pointers out in memory in the proper location. Any data following the vTable in the ClassPtr is pure gravy -- you can put anything in there you'd like.

When one of the methods is called from the vTable, it is passed in a "this" (or me) pointer. This pointer is to the ClassPtr itself. In this manner, each of the interface method gets a data pointer (as its first parameter) to the class instance itself.

Now that we've got all the information we need, let's see some code!

First, let's create an instance of our COM object. Let's assume (for clarity's sake) that our COM object only implements the IUnknown interface. So we're responsible for adding the following methods (in this order) to satisfy the contract:
[rbcode]
QueryInterface( refid as Integer, ByRef out as Integer ) as Integer
AddRef() as Integer
Release() as Integer
[/rbcode]
So the first thing we need to do is initialize our COM object
[rbcode]
// First, create the memory block for the COM object.
// It's going to be 4 bytes long since it only holds a Ptr
// to the vTable
dim comObject as new MemoryBlock( 4 )

// Now we're going to make the COM object's vTable
// This needs four bytes for each method that we need
// to expose.
dim vTable as new MemoryBlock( 4 * 3 )
[/rbcode]

Now comes the trickier part. We need to add the functions to the vTable itself. Here's where the new AddressOf operator comes in handy. We're going to stick our "callback" methods into a module so that we can use the AddressOf operator to get a function pointer to them. We can then stuff that address into the vTable MemoryBlock.
[rbcode]
// The next thing we need to do is construct the
// vTable for the COM object. We do this by using
// the AddressOf operator and pass in methods declared
// within this module.

// Here's the IUnknown interface that every COM object
// must implement
vTable.Ptr( 0 ) = AddressOf QueryInterface
vTable.Ptr( 4 ) = AddressOf AddRef
vTable.Ptr( 8 ) = AddressOf Release

// Now let's set the COM object's vTable
comObject.Ptr( 0 ) = vTable
[/rbcode]
Don't worry about the structure of the methods themselves -- I'll be getting to that in a little bit. First, we have other things to work on. Since we're using module methods, we're fairly restricted. That means we can't really have multiple instances of our COM object. That's no good -- so let's remove that limitation.

We're going to use a Dictionary to map an RB class to one of our MemoryBlock-made COM objects. Remember, we're given a "this" pointer which is equal to our MemoryBlock's base address. We can use this knowledge to our advantage by storing the address of our comObject MemoryBlock into the Dictionary as the key, and the class instance we want to hook into as the Value. Like this:
[rbcode]
// And set the COM object's extra information so
// that we can locate the RB object we're interested
// in.
dim comObjectPtr as new MemoryBlock( 4 )
comObjectPtr.Ptr( 0 ) = comObject

// Add the class to our object map
mObjectMap.Value( comObjectPtr.Long( 0 ) ) = c
[/rbcode]
At this point, the comObject variable is ready to be handed out to any Win32 API needing an interface instance. Our COM object is complete! So let's turn our attention to the module methods I was talking about earlier.

The first parameter to any of our COM interface-driven methods is always an Integer representing the C++ "this" pointer. Basically, that integer is the base address of our MemoryBlock. We mapped the base pointer to an actual class instance though, so we've got a way to go from our COM object to our RB object easily. Let's assume that we're working with a class named Class1 and it also is implementing our IUnknown interface (which you should model after the methods described above).

All of the module methods we were using the AddressOf operator on are going to look very similar. They're essentially nothing more than proxy methods which forward the request on to the proper RB interface implementor. We're going to pull a class out of the object map, do some sanity checking on it, and forward the method call. Here's how the AddRef method will look:
[rbcode]
Private Function AddRef(this as Integer) As Integer
// Find the object in our map
dim mappedClass as Object = mObjectMap.Lookup( this, nil )
if mappedClass <> nil and mappedClass IsA IUnknown then
return IUnknown( mappedClass ).AddRef
end if

return 0
End Function
[/rbcode]
As you can see, the proxy methods are quite simple to implement. The true guts of the implementation need to happen within your RB class (the RB interface implementor). But once you've got those implemented, then you're set! Voila! Instant COM interface gratification.

As you can see, we've covered the entire process from start to finish. You simply layout the COM object in memory using a MemoryBlock, map the MemoryBlock to an RB class instance -- and this class instance is where you'd put your implementation.

I've created a sample project with a partially working code sample. I say partially working because I didn't want to muck the entire thing up with too much detail. So instead of putting in the "interesting" guts part of the class, I mostly put in Break statements. So when you run the application and press the PushButton, you'll break into the debugger in one of the OS-called COM methods. I made the project using RB2005r2, but it should be possible to use it (perhaps with minor modifications) in earlier versions of REALbasic as well.