Why BinaryStreams Rock

So you may be wondering to yourself -- just why should I love REALbasic's BinaryStream class? I mean, it's kinda cool that I can open a file up in binary mode and read/write data using it. But honestly, just what's so cool about it? You may also be named Will Leshner and be wondering about a Stream interface. ;-)

If you're in either of the two categories, you'll really enjoy this post, because it's all about a little-known new trick with RB 2005.

You can create BinaryStreams in code with non-file backing and use them. That's right -- a stream object that works completely in memory without a file behind it, but with the entire BinaryStream API that you already know and love.

Let's see how it works.
[rbcode]
Dim bs as new BinaryStream( new MemoryBlock( 0 ) )
[/rbcode]

Phew! That was hard. Hopefully I didn't lose anyone with the complex code snippet. ;-) So how does it work and why is it awesome?

What the code snippet is doing is making use of the 0-length MemoryBlock feature in RB 2005, which basically is a chunk of reserved memory that stores no data. Then it passes the MemoryBlock off to the BinaryStream constructor. This tells the BinaryStream "use this chunk of memory for all your operations." The BinaryStream then uses the MemoryBlock under the hood to do all its reading, writing and other functions. The memory under the hood is automatically managed for you so that you don't have to worry about writing out of bounds or things like that (and it's done so in an efficient manner as well). So let's see another example of how this works.
[rbcode]
Dim bs as new BinaryStream( new MemoryBlock( 0 ) )

bs.WriteLong( 12 )
bs.WriteShort( 0 )

bs.Position = 0

MsgBox "Size = " + Str( bs.Length ) + ", First 4 bytes = " + Str( bs.ReadLong )
[/rbcode]
As you can see, the BinaryStream behaves just like you'd expect it to. It has random access just like a file would, it keeps track of the size used, can be written to or read from, etc. Pretty cool, eh?

But wait, there's more! You don't have to give the BinaryStream constructor a 0-length MemoryBlock. You can give it one that's already "filled out" if you'd prefer. Let's say you get a chunk of data back from TCPSocket.ReadAll and you want to use the BinaryStream class to handle it. Your code would look something like this:
[rbcode]
Sub TCPSocket1.DataAvailable()
Dim mb as MemoryBlock = me.ReadAll
if mb = nil then return

Dim bs as new BinaryStream( mb )

if bs.Length >= 4 then MsgBox Str( bs.ReadLong )
End Sub
[/rbcode]
You can shove the data from the ReadAll operation into the MemoryBlock, and then pass it along to the BinaryStream to be used for regular stream operations. But that seems kind of silly since you're sticking it into a MemoryBlock first, and then passing it along to the BinaryStream class. I bet you're looking for an easier way! Well, there is. You can also use a String for the BinaryStream backing if you'd like.
[rbcode]
Sub TCPSocket1.DataAvailable()
Dim bs as new BinaryStream( me.ReadAll )

if bs.Length >= 4 then MsgBox Str( bs.ReadLong )
End Sub
[/rbcode]
There's one thing to keep in mind though when you use the String version of the BinaryStream constructor, which is the fact that RB strings are still immutable objects, and because of this, you cannot use any of the Write operations.

Some of you may be wondering why this is helpful -- I mean, you can just use a MemoryBlock, right? I use this all the time when dealing with network and serial protocols because I hate trying to keep track of my current position in the MemoryBlock. Using the BinaryStream means that I don't have to worry about that -- I just write the data out and the stream handles the rest. Same with reading data in from an outside source; it just makes life much easier.

One last cool thing to note -- the contents of the BinaryStream backing are actually changed. So you can use this with declares if you'd like. Check it out:
[rbcode]
Dim mb as new MemoryBlock( 0 )
Dim bs as new BinaryStream( mb )

bs.WriteLong( 12345 )
bs.Close

MsgBox Str( mb.Long( 0 ) )
[/rbcode]
Calling BinaryStream.Close is a safegaurd against the BinaryStream modifying the contents of the backing buffer. Once you've called Close, the backing cannot be changed any more by the BinaryStream class.

So how does it work under the hood? The BinaryStream class is a thin wrapper around an implementor (this is known as the bridge pattern, btw). When you open up a file as a binary stream, the implementor is a DataFile class. So calling it calls thru to the appropriate OS file system calls to deal with the file. However, when you use a MemoryBlock, there's a different implementor (I forget what I named it, I think MemoryBlockMechanism or something like that) that calls thru to all the MemoryBlock function calls. And for Strings, there's a third implementor which calls thru to the string function calls. Because of the bridge pattern, we could also add other types of backing if we saw a good reason to do so.

When using the MemoryBlock backing, we have to do some magic to deal with the size of the MemoryBlock. We have three counters -- one that keeps track of the current write position, one that keeps track of the "true size" of what's being used, and the one that keeps track of how much data we can possibly store. When writing data out, we increment the write position and the true size. When the write position goes beyond our possible size counter, we grow the buffer (by using a sliding memory scheme, much like the one sockets use internally for their buffers). But when you call BinaryStream.Length, we only report back the size you're currently using -- not the size you can possibly use before we grow the buffers again. Because of this buffer sliding scheme, it's wise to pick a rough estimate of the end-size of the memory you need. This causes the framework to spend less time managing memory and provides for a more efficient application.

While this isn't quite the Stream interface Will is wanting, it's still a pretty neat trick that will probably be overlooked by many people when RB 2005 is released. It's a handy trick to put in your toolbox of programming goodies. Enjoy!