failed assertion in commonruntime.cpp:XXXX

Someone called ObjectIsa, passing in an ObjectDefinition that we could not lookup. The object we're looking for is: YYYYY

For instance, you may run into this assertion on the Mac if you have a menubar for the application which does not contain an item whose super is QuitMenuItem. In that case, it'll say the object we're looking for is QuitMenuItem.

Thankfully, these assertions are trivial to work around -- but remember, you should always file a bug report so that we can fix these assertions for you! But in the mean time, you don't want to wait for us to fix them. Whenever you run into one of these assertions, you can fix it with the following block of code:

dim o as Object

if false and _

  o IsA XXXX then

end if

So why do these assertions happen in the first place? Well, for the past few years, we've had a technology to transition from C++ object definitions to REALbasic object definitions. This technology is called the object definition converter, and it was responsible for converting from one object model into the other. However, the down side to having this was that it made adding new functionality to the object model very painful -- you had to remember to hit up this strange converter thing and force it to behave properly. So for the past year, we've been trying to ween our internal technologies from relying on this converter -- and finally, we've "removed" it. It's this removal that is causing pain for people in rare instances (and by rare, we've only found two cases that have been overlooked).

(I put removed in quotes above because it's not quite true. The converter does still exist, and will have to exist for as long as we have a C++ plugin SDK, since the plugin object model still needs to be converted to the REALbasic object model.)

So why do you get those assertions in the first place? Simple -- the linker is doing its job and dead stripping out references to things your project doesn't use. So if your project doesn't make use of a TCPSocket, then there's no sense to carrying around data about the TCPSocket at runtime. However, the C++ code might make use of the TCPSocket class -- and unless we tell the linker to please not dead strip TCPSocket somehow, the C++ code will throw that assertion. The code above accomplished that nicely because of the IsA clause. The code doesn't actually perform any work, but it does cause the compiler to take note of the dependency, which in turn solves the problem.

Plugin authors have to be extra careful of this problem. If your plugin relies on an object internally (via calls to REALnewInstance, for example), then you have to design your plugin in such a way that it alerts the IDE of the internal reliance. The way you do this is by using calls to REALGetClassRef within your PluginEntry method. By doing this, the IDE can be sure to generate code (like above) to ensure that the linker doesn't dead strip the class. For instance, let's say your plugin makes use of the MD5Digest class internally. You'd do something like this:

static REALclassRef md5DigestRef = NULL;


void PluginEntry( void )

{

  md5DigestRef = REALGetClassRef( "MD5Digest" );

}

void SomeMethod( void )

{

  REALobject obj = REALnewInstance( md5DigestRef );

}

In closing, you probably won't ever run into one of these object definition converter assertions. But if you do, be certain to report it (telling us how you get the assertion is especially helpful), and work around it with the code found at the top of the post.

Interface Test

  Sub Foo()

End Interface


Interface Awesome

  Sub Bar()

End Interface

Interface Sweet

Aggregates Test, Awesome

  Sub Blah()

End Interface

Basically, when a class implements an interface, it must implement all of the methods from the interface, as well as methods from any aggregated interfaces. When a class implements an interface, then calling IsA on the class will return true for that interface as well as any aggregated interfaces. From our example above, a class implementing Sweet, then IsA Sweet, IsA Test and IsA Awesome are all true (since it implements all three of the interfaces).

This allows you the ability to combine interfaces in creative ways! This is important because interfaces don't necessarily relate to one another, but there are times when you need something that IsA multiple different interfaces. Or, when you want to merge functionality together for interfaces.

For instance, let's say that you want to have an item which can be iterated over. A common interface might be:

Interface Iterator

  Function Current() as Variant

  Function MoveNext() as Boolean

  Sub Reset()

End Interface

Interface Disposable

  Sub Dispose()

End Interface

You might be wondering "so why not just put the Dispose method right onto the Iterator interface?" That's a perfectly legitimate way to accomplish the same goal -- however, Iterator and Disposable aren't the same conceptually. So it doesn't make sense to force the two interfaces together in such a fashion. It doesn't hurt, but it just doesn't help either. Let's pick an example where it would hurt!

Let's say you have a method that is going to generically take a parameter which represents something that can read and write to/from a source. In this case, you want something that's both Readable and Writeable. You could just declare the method like this:

Sub DoSomethingAwesome( foo as Readable )

  if foo IsA Writeable then

    // Do reading and writing

  end if

End Sub

Interface ReadableAndWriteable

Aggregates Readable, Writeable

End Interface


Sub DoSomethingAwesome( foo as ReadableAndWriteable )

End Sub

The last thing I want to cover is the naming of the feature. Aggregates might seem like a very strange term to use, but in actuality, it really describes the feature better than anything else. The term "aggregate" means "to sum up" (essentially), which is what happens here. You are taking all of the interfaces and summing them up into a single uber-interface. The term "Inherits" would be a possible candidate, but it doesn't fit very well. Inherits assumes that the interfaces relate to one another, which isn't correct (look at the Readable and Writeable, for example). What's more, REALbasic only allows single inheritance and you can aggregate multiple interfaces together.

I could go on for pages about how awesome it makes our internal processes, and how much time it saves me, etc. But you probably don't care too much about that. Instead, I want to tell you about how awesome it is for you!

We've integrated bug reporting into the IDE to make it trivial for you to enter bug reports about some of the most common error situations you run into. What's more, you have to do almost no work to take advantage of it!

Whenever a bug happens while you're working in the IDE, you'll be presented with a bug reporting dialog that looks like this:

This dialog allows you to enter in some extra information, such as what you were doing when the crash happened, and your email address. It also gives you a link that you can click on to see the report contents, which look something like this:

All you have to do is click Send -- that sends a bug report to us. So why is this so awesome? Well, for starters, it's a significant improvement over the old way you'd have to report the bug. In the old system, you'd have to type out the stack trace yourself (or take a screen shot), and enter in the entire bug report manually. Now you just click a button. But it gets better! All of the identical reports are automatically merged together in FogBugz so that we can see how many people are affected by the same issues. And, even better than that, we can provide instant feedback to users who run into the same issue!

Let's say you have an unhandled exception that fires any time you open up the icon editor. I can enter in a text message within FogBugz that says "We're aware of the issue with the icon editor -- you can work around it by doing X, Y and Z. We hope to have this issue fixed by 2008rX. Thank you for your patience!" Then, whenever someone hits that same bug with the icon editor, they're presented with my friendly message. Annnnnnnd... once the bug is fixed, I can give users a different message, and tell FogBugz to ignore any new reports about the crash. If you'd like to see this particular feature in action, there's a report that is fairly easy to reproduce which demonstrates it (but only if you're using the Mac IDE and have a functional network connection):

1) Make a new project
2) Go into the Menu Editor, and navigate to the File->Quit menu item
3) Change its super from QuitMenuItem to just regular MenuItem
4) Launch the application in the debugger

When you launch the application in the debugger, the IDE should kick the error reporter window up. If you hit the Send button, you should get a response back from the server telling you what the issue might be, and how to resolve it.

So a few things to note about the automated reporter, just for your own education. If you enter an email address into the optional field, then we can communicate directly with you. For instance, when the bug is fixed, you'll get an email letting you know about it. So if you're interested in this sort of communication (which can be very helpful if we need to ask you for more information), then please enter an email address. In terms of what information is sent to our servers: it's all there in the report window. We collect information about whatever we can (and that's subject to change as we determine what helps us to track down bugs) and send it back. Currently, that information is: what plugins are installed, a stack trace, and user-entered data.

Finally, the error reporter pops up for a handful of different error situations. We currently have automatic error reporting for unhandled exception within the IDE, IDE failed assertions, compiler failed assertions, and any framework failed assertions that occur while debugging your application. If any new error reporting situations arise, we will probably add them, because automated reporting saves everyone a ton of time!

So yeah, don't mind me when I'm all giddy about error reporting. But I can assure you, this type of feature was basically impossible with the old feedback system. The switch to FogBugz has made it possible, and it's already saved us a ton of time within the betas program. As an example, there was an assertion in the compiler with the new GetTypeInfo operator -- a user had auto-reported it via the feedback system, and it immediately had enough information for me to go in and resolve the bug. Anyone else who ran into it immediately was told how to work around the bug temporarily, and a fix was forthcoming within a release. Very, very slick!

First and foremost, this wasn't a decision that was made lightly. Trust me when I say, if there was a reasonable alternative, we'd have taken it. But there aren't reasonable alternatives, which is why the decision was made. That being said, let's get some background history on the topic.

In ancient versions of REALbasic, the compiler used to "bake in" plugin shared libraries for the final executable. They would reside as extra pieces of data, tacked onto the end of the executable. When the application would launch, the runtime would take all of the DLLs and write them out to disk. Then the application would load the DLLs from disk, and continue executing. However, this turned out to be problematic in several cases.

For starters, it caused DLL hell. If you had two REALbasic applications that were going to run at the same time, you could run into monster amounts of troubles when the second application tried to write out its DLLs. If the names were the same, the application couldn't write out its DLL, so it would try to use the one already on the disk. However, this doesn't work if the two applications were built with different versions of REALbasic! What would end up happening is that the second application would crash sometimes. So plugin authors had to name their DLLs with some version information so that applications didn't run into naming conflicts. This worked ok, so long as everyone remembered to do that (and trust me, not everyone remembered).

Another major problem was with 3rd party anti-virus software. It would see an application carrying what essentially amounted to an executable payload. Even though we weren't doing anything unscrupulous, some AV software would still flag the application as being a trojan. This would understandably scare people away from using that REALbasic application!

Finally, we had issues with permissions, disk space, and just plain old misunderstandings of the functionality. If the application couldn't write the DLL out to disk due to lack of permissions, or because the disk was full, it would simply fail to launch. This happened with a relative amount of frequency, especially as more and more people started using REALbasic on server OSes like Windows 2000. Also, some users had hard requirements that stated they couldn't modify the user's machine at all -- their application had to run from a CD or thumb drive, and be entirely self-sufficient. This simply couldn't be avoided with the scheme with the way things were.

So after a while, we realized that writing the DLLs out to disk wasn't a reasonable solution. It had way too many drawbacks, and cost a lot in customer support for us as well as for our customers (who had to deal with these issues with their end users). So we came up with another, more clever solution: don't write the DLLs out to disk at all!

We rewrote the OS DLL loader functionality -- since the DLLs were already in memory, why not just leave them there, since that's where they'll end up? So we would still push the plugins into the executable at compile time, and at runtime, we would have the framework do all of the same monkey-work that the OS loader would usually do so that we could just use the DLL already in memory. It was a very ingenious idea because it solved a lot of problems. We didn't have to worry about DLL hell anymore since every application runs in its own memory space. We didn't have to worry about running out of disk space or permissions issues because the application was in memory already. We also didn't have to worry about cases where the user couldn't modify the hard drive since the DLLs were never being written out. The AV software problem was significantly lessened as well (though it didn't disappear totally). However, this solution turned out to be untenable, though we didn't realize it at the time.

You see, the operating system simply does not allow for DLLs to be loaded by anything other than the OS loader. Our solution was very creative, but it had problems. Some of the problems we could solve (like thread local storage). However, the fact still remains that there is a major class of problems we cannot solve: application compatibility shims. Microsoft's OS loader handles these shims internally, and there is absolutely, positively, no way for us to mimic the functionality. We must use the system loader to load DLLs in order for them to be loaded properly.

So how did we stumble into this problem, and why am I using such strong language about it?

For starters, ever since we implemented the memory loader, we saw a lot of bug reports simply go away. No more complaints about multiple RB applications unable to run at the same time, etc. However, there was a slow trickle of unrelated, unexplainable crashes on Windows. The hallmark of these reports was that they usually said something like "I don't understand this. I have XXXX installs of my application, but on six machines the app totally fails. I can't figure out why." Here's a classic example of a user's problem on the forums. This user actually did an amazing amount of research into the problem, and is one of the few cases that can be classified as "obvious." I've seen it posited that this doesn't affect very many people. That's not true. While it's very difficult for me to put an exact number on it due to the nature of the problem, I can attest that it's a non-trivial percentage of our users.

We would get this trickle of reports, and since no one could reproduce the issue, we would just close the reports as not reproducible. I mean, NO ONE could reproduce these issues... Then one day, I found a way to reproduce the issue! I had a particular machine in my house that couldn't run the REALbasic IDE. Whenever I would try to compile any applications, I would get a strange error dialog that said the C runtime floating point library wasn't loaded. Then the compiler would work, I'd get my application and life was fine. Until I went to quit the IDE when I'd get another message dialog that said thread locale storage couldn't be initialized -- and the IDE would crash in a ball of flames. Very, very weird stuff considering the fact that the CRT floating point library certainly should have been loaded, and we don't use TLS anywhere!

So I pulled out my debugger, and went to town on it. After three days, I was still absolutely no closer to figuring it out. What was neat about the problem was: if I tried to debug the plugin directly, the problems went away. But if I just attached to the IDE with a kernel debugger, the crashes would happen -- and were valid crashes! Honestly, the floating point library really wasn't loaded... it was the strangest thing I had ever seen. I let the problem sit for a few weeks while I worked on other things, and just avoided that machine. Then one night, I realized what a difference was: the way I was debugging the plugin! You see, REALbasic has always had this debugging technique where the application would prefer to load a DLL from disk if it was named right, and was directly next to the executable. In that case, we were using the system loader to load the plugin up. So by using this plugin debugging technique, I was using the system loader. But when I just used the kernel debugger, I didn't have the DLL next to the IDE, so we used the memory loader. This turned me on to the fact that the problem was something to do with the memory loader -- I just assumed we had a bug in there, and went to work trying to fix it.

I spent another day working on the memory loader, and nothing made sense. We didn't have any TLS specified in the spawn compiler DLL, and so that failure eluded me entirely. But what's more, the CRT was statically linked into the DLL, so there was no import record for floating point to load! The CRT takes care of that for you, supposedly. Problems with plugins causing applications to crash is a very serious thing, and I was starting to realize how large the problem was after looking at the number of "not reproducible" bugs and how they all pertained to a plugin at some point. So finally, I decided to use one of my support tickets with Microsoft to see if they could help track the issue down.

I don't know how many of you have ever had a reason to use Microsoft's developer support program, but it is a pretty incredible experience. They basically assign an engineer to you who looks at your source code, and debugs it for you. Since I had narrowed the issue down to the memory loader, that's where I had the engineer start and he was immediately perplexed -- it's not every day that you run into a hand-written loader! So he escalated the issue up the food chain until it finally landed with the DLL loader team. These are the engineers at Microsoft responsible for actually writing and maintaining the code that loads DLLs from disk into memory. They loved our source code, but pointed out the problem: we have no way to load application compatibility shims. It turns out that the CRT from Microsoft has an app compat shim that my particular machine was forcing to be loaded. Since the memory loader had no idea about this shim, all hell broke loose and that was the trouble, in a nutshell.

Ok, so the memory loader is impossible to use. Period. When the people responsible for loading DLLs into memory say "you have to use the system because of X, Y and Z", that pretty much seals the deal. So the next step was to discuss options with Microsoft to see what we could do. You all know the end result by now, but there were other options discussed.

Of course, the first one was "what if we just keep using the memory loader?" The answer to that was, "your applications will become more and more problematic." You see, many times Microsoft comes out with a hotfix that includes more compatibility shims. They have an ever-growing database of times when a shim needs to be loaded. And if the shim isn't loaded, the bugs can be spectacular. Sometimes they're subtle crashes at any point during the application's session. Sometimes it's a crash when the app launches, or a big hairy dialog that makes no sense at various times, etc. What's more, something as simple as installing a hotfix could break an application, but only one one particular hardware configuration! Basically, keeping the memory loader would put a huge burden on our users by having crashes that are reasonably out of their expertise to track down and resolve. That then puts a burden on us in terms of bad publicity ("RB applications are buggy"), and man-hours (tracking down bugs related to plugins). And the problem will only get worse and worse as time goes on. Basically, keeping the memory loader is a very bad idea.

The other option discussed was by having the plugin format change in various ways. For instance, we could require plugin authors to rewrite their plugins as static libraries that the REALbasic compiler would link in to the executable. In that case, there would be no need for a DLL at all, ever. This has some major problems too. For starters, it would require plugin authors to rewrite their plugins. Never mind the amazing amount of work that would be for some, what would be done about the plugins that no longer have an author supporting them? But a bigger problem is the fact that static libraries have custom file formats that are compiler specific. That means we'd either have to restrict plugin authors to just one or two compilers (and versions of the compilers too!), or we'd have to spend *years* writing linkers that break constantly. It's a noble idea, but it's also basically impossible to get right. Even if we tried to statically link in the DLLs without requiring the authors to rewrite a thing would run into compatibility issues.

So in the end, the discussion from Microsoft's end really boiled down to: just load the DLLs from disk like every other application on Windows does.

Now you understand why we had to make the decision to write plugins out to disk. Our options were to go back to the old way with DLL hell, AV problems, etc. Or to just spit the DLLs out to a folder (which is what we already do on the Mac, though those are packaged up more cleanly by OS functionality via bundles). Since the whole point to this issue was to reduce bugs, we really only had one option to pick from.

We had toyed with the idea of providing users with an option to turn this functionality on and off, but eventually decided against it (strongly). You see, at the root of it all, what we were discussing was an option that basically read like this: "Would you like your applications to randomly crash in unavoidable ways? Yes or No" That's not an option we are comfortable giving to our users. When something is a preference, the expectation is that it's supported whether the user says "yes" or "no." And this wouldn't be supported. So now we're talking about more than just a simple option -- we'd have to explain to the user the ramifications, pitfalls, etc. But most users won't read that far, they just figure "it works well enough I guess" and pick one, regardless of how strongly worded the cautions might be. So in the end, the decision was strongly against providing an option because we don't want there to ever be a choice between stable and unstable. The whole point to REALbasic is that "it just works."

The nice thing is: REALbasic still does just work. Yes, if you use plugins it no longer generates a single-file executable. Instead, it generates a single folder that you can zip up and send to your friends. It's also no more confusing to the majority of people since the application is still blindingly apparent (it is the only file in the folder). So if you are not interested in writing an installer, your distribution is only slightly modified. What's more, because of the folder layout we eventually went with, the vast majority of applications will require no modifications. You can still run multiple REALbasic applications from the same directory, etc. This change will affect your build processes, and that is the unfortunate cost of making applications on Windows more stable. If you're creating a professional-grade application on Windows, then you have an installer that places the executable into the Program Files directory. This is a directory that most users never see, much less monkey with. In fact, users can't monkey with it unless they have admin rights to their machine (in which case, they hopefully know enough not to remove random files, right?). So while this certainly is a change, it's also not one likely to be felt by your end users in many circumstances.

In closing, I understand that this sucks. Having a single-file executable distribution was a really neat idea that set REALbasic apart. Changing that is not a decision that we made lightly! But it is undeniable that we value stability, and this is a case where we needed to ensure that REALbasic does the right thing to provide the most stable applications possible. Single-file executables are neat, but not neat enough to justify the pain they were causing many of our users.

if &hFFFFFFFF > 0 then

  break

else

  break

end if

When you run this code, you'll get results that seem a little strange at first blush. You get into the else clause -- so the compiler is generating code that says a large unsigned number is less than 0. How's that possible??

Well, what the compiler does whenever it gets any binary operator (+, -, =, <, etc) is first compute a common type between the operands. And for backwards compatibility reasons, the common type between signed and unsigned is signed. This harks back to the days when REALbasic only had signed 32-bit integer values. So in essence, this causes the compiler to treat the large unsigned value as being a signed value whenever the high bit is set. Any value >= 0x80000000 will be treated as a negative, thus turning the if statement into this: if -x > 0 then, which will always evaluate to false.

So what should the compiler do in this situation? If it promotes everything to unsigned, then you run into an equally undesirable situation with code like this:

if &h0 > -1 then

  break

else

  break

end if

Keep in mind that the behavior has nothing to do with constants either -- you can get the same behavior using just variables as well. Maybe the compiler could try to do something sensible with constants, but the problem still remains any way you slice it. But let's explore this a bit: should the compiler try to "be smarter" when it comes to constant comparisons? Given the following code:

dim u as UInt32 = &hFFFFFFFF

if u > 0 then

  break

else

  break

end if

dim i as Int32 = -1

if i < &h0 then

  break

else

  break

end if

This tells me that computing the common types is clearly the wrong place to try to place more logic. No matter what we do, there are very trivial cases where the behavior will be undesirable.

The only other alternative is to handle constants as a special case during comparison operations. If either of the operands is a constant, then we could do some special processing that essentially ignores the common type computation. However, what would that special processing be? Convert the constant to the variable's sign? That might work, but what about this code:

dim u as UInt32 = &h80000000

if u > -1 then

end if

So basically, this is a really difficult question to answer. As much as I dislike saying it: I think the original behavior is not a bug. It's just the misfortune that comes about from mixing signs. Ideally, the compiler would warn you about this situation -- and that may come about in a future release of the product. But for the time being, you should watch out for cases where you mix signs, and understand that the compiler will always convert both values to being signed if any of the operands in a binary operation if either operand is signed. You can avoid this situation by being explicit and telling the compiler how to handle the operator: use typecasting. If you want an unsigned comparison, typecast using UInt32( 0 ), or use a constant value that is implicitly unsigned (like &h0).

The concept of a pair is something that comes up frequently in programming. As REALbasic programmers, you're undoubtedly familiar with the Dictionary class. That's just a fancy way to store pairs of information -- keys and values. However, it doesn't afford you an easy way to describe a single key/value pair as its own entity. That's what makes the new Pair class a very interesting concept. Let's start out on the simple side of things and say you want to represent a key/value pair in your code. To create a pair the easy way, you can do this:

dim p as Pair = 1 : 2

What this gives you is a single class whose Left value is 1, and Right value is 2. It's as easy as that! So let's take a more real-world example. Let's pick on serialization -- you want to represent a property, and its value in a generic fashion.

  dim ti as Introspection.TypeInfo = Introspection.GetType( self )

  dim props() as Pair

  

  for each prop as Introspection.PropertyInfo in ti.GetProperties

    props.Append( prop.Name : prop.Value( self ) )

  next prop

  dim out as TextOutputStream = SpecialFolder.Desktop.Child( "Test.csv" ).CreateTextFile

  for each p as Pair in props

    try

      out.Write( p.Left.StringValue + "=" + p.Right.StringValue + ", " )

    catch err as TypeMismatchException

      // Just ignore the problem

    end try

  next p

  out.Close

But wait, there's more! It's not just a singly-linked list -- it can also represent a binary tree. Because there's a left and a right property, you can use the Pair class as a node in a tree with two possible children.

Hopefully this gives you a few ideas as to how you can make use of the new Pair class, and it's corresponding operator (the colon) to clean up some of your projects. It's not the most ground-breaking feature ever, but it is a neat little tool to make use of.

ByRef and ByVal are two modifiers used with parameter list declarations that allow you to specify the way the arguments are treated by the compiler. On the face of things, when you pass something "ByVal", you are passing it "by value", and when you pass something "ByRef", you are passing it "by reference." Of course, this is slightly muddied by the fact that some things can only exist as a value type, and so they cannot be passed ByRef, and other things only exist as a reference type and so they cannot be passed ByVal.

So how do these things work? Let's talk about ByVal first, since that's a bit easier to understand. When you pass something ByVal (which is the default in REALbasic, when you don't specify any modifiers), what really happens is that you get a temporary variable in the method being called. So any modifications to the parameter are really happening on the temporary and don't affect the original. Let's say you have a method signature:

Sub SomeAwesomeMethod( foo as String, bar as Integer )

Now, when you try to call that method

SomeAwesomeMethod( someProperty, 12 )

You can imagine what really happens is this:

// SomeAwesomeMethod( someStringProperty, 12 )

Push someStringProperty's value, which is "Rock on!"

Push 12

Call SomeAwesomeMethod


//Sub SomeAwesomeMethod( arg1 as String, arg2 as Integer )

Dim bar as Integer = 12

Dim foo as String = "Rock on!"

// Do the work

You see, the arguments in the parameter list are actually just local variables that are temps. So if you make changes to bar or foo, then those changes are entirely local to the SomeAwesomeMethod method.

Let's contrast that with ByRef, which passes things by reference. When you pass something by reference, what you're really doing is saying "hey compiler! You can find the value for that thing over here." So the compiler doesn't pass the actual data -- it passes a reference to where the data can be found (those of you coming from C/C++ know this concept -- it's just pointers!). So modifying the example above to be:

Sub SomeAwesomeMethod( ByRef foo as String, ByRef bar as Integer )

Now we're passing in the parameters by reference. In this case, we have to call the method with things that have storage somewhere (like properties, or local variables -- not constants):

Dim numberTwelve as Integer = 12
SomeAwesomeMethod( someStringProperty, numberTwelve )

You can imagine that what the compiler does instead is:

// Dim numberTwelve as Integer = 12
AllocStackSpace( 4 bytes )
Move 12 into allocated space (at address 0x1234)

// SomeAwesomeMethod( someStringProperty, numberTwelve )

Push someStringProperty's addres (let's say 0x5678)

Push 0x1234 (the stack address where the numberTwelve variable lives)

Call SomeAwesomeMethod

// Sub SomeAwesomeMethod( ByRef foo as String, ByRef bar as Integer )

Dim bar as Ptr = 0x1234 (reference to numberTwelve)

Dim foo as Ptr = 0x5678 (reference to someStringProperty)

// Do the work

dim foo as Integer = 55

DoSomething( foo )

MsgBox Str( foo )


Sub DoSomething( ByRef bar as Integer )

  MsgBox Str( bar )

  bar = bar + 10

End Sub

// Dim foo as Integer = 55

AllocStackSpace( 4 bytes )

Move 55 into allocated space (at address 0x1234)


// DoSomething( foo)

Push 0x1234

Call DoSomething

// MsgBox Str( foo )

MsgBox Str( foo )  // How this works isn't important

// DoSomething( ByRef bar as Integer )

dim barTemp as Ptr = address where bar lives, 0x1234

// MsgBox Str( bar )

MsgBox Str( barTemp.Int32( 0 ) )  // How this works isn't important

// bar = bar + 10

barTemp.Int32( 0 ) = barTemp.Int32( 0 ) + 10

// End Sub

So how does this relate to things which are already references? In that case, the reference parameter (such as an object or an array) is passed by reference automatically, since it's already a reference in the first place!

An interesting thing to understand are the compiler errors (or lack thereof) for these modifiers. If you try to pass a constant as a ByRef, you'll get an error from the compiler. This is because the contract for the method cannot be satisfied. A constant has no storage location in memory, so the caller attempting to modify the argument has no reference to modify. Hence the reason you get a compile error -- it's impossible to do!

But in the case of passing a reference type ByVal, you don't get a compile error (or even a warning): why is that? Because the alternative sucks worse! REALbasic defaults to using the ByVal calling convention. If you had to be explicit, you'd have very verbose method parameter lists. But what's more, the concept makes no sense, but in such a way that there's a reasonable alternative. If you pass something which can only ever be as a reference, it's reasonable to assume that passing it as a reference is a sensible default behavior, right?

The final compile error seems to cause a few people some confusion -- why can you pass "SomeClassProperty", but not "Self.SomeClassProperty"? The explanation is: we want to keep things very, very simple. You cannot pass expressions as ByRef because there are some cases where it'd make sense and work, and other cases where it wouldn't work. So we make it very easy on you: no expressions! And you can consider anything with a dot in it as being an expression, even if it may not make sense. This makes using ByRef significantly more simple to use because you don't have to think "hmm, will this work? What would the compiler do?" Instead, just keep in mind: no dots, no constants!

So to make a long story longer, here are the take-home points for you to remember about ByRef and ByVal:

• ByRef allows the method being called to modify the parameter such that the caller can see the modifications.


• ByVal allows the method to modify the parameters in any way it wants without fear of the caller seeing the modifications. Except if the parameter is a reference type!!


• It's worth repeating: objects and arrays are reference types, so there's no such thing as ByVal for them -- the method can modify these types, and the caller will notice the changes.


• REALbasic defaults to ByVal passing, whereas VB6 and earlies defaulted to ByRef.


• You cannot pass a constant or an expression ByRef. Only locals and simple properties can be passed ByRef.

Phew! That was a whole lot of prattling on about something that most people don't give a second thought to. Sorry about the meandering and incoherent explanations. If I've only muddied the waters for you, then please ask questions and I'll do my best to provide better answers for them. If I've cleared some things up for you, then it was a great use of not napping (which is what I'm going to go do now).

Nathan Regener is our new engineering manager. He's in charge of the builds, and so that makes him in charge of all the little pieces and parts that go into the builds as well. His job consists of making sure programmers are getting stuff done on time (he's my boss, basically), as well as making sure QA is testing everything, our doc writer is explaining it properly, our icon designer is making pretty pictures, etc. On top of all of that, he gets to work late nights trying to script the build process down to just a single shell script. :-P

David Alter is our awesome new Mac guy -- he's the one helping us to make our transition into Cocoa while continuing to support our current Mac offerings. Since he started working for us, he's been spending most of his time learning our massive framework and how it all fits together. It's quite a daunting task, but he's been making awesome progress at it! But now he's starting to work more towards how Cocoa will fit into the framework, and how to best move forward with it.

Norman Palardy is a long-time REALbasic user that I think almost everyone in the community knows. He's our most recent addition to the engineering staff, and he's a very welcome face. Aside from considering him to be a friend, he's also a great programmer with years of database and RB experience under his belt. He's going to be working mainly on the IDE, but I wouldn't be too shocked to see him helping out with some framework features as well.

Kyle Findlay is our most-excellent leader of testers. He's only been with us a short period of time, but has already had a huge impact on our internal testing procedures. He's basically in charge of reviewing all the bug reports that come in, verifying and categorizing them, as well as making sure our internal testing needs are met. So, for instance, when I'm adding some new compiler feature -- Kyle's job is to ensure that it's being tested adequately before it ever reaches the public in an alpha. He brings many years of testing experience to the table, and we're happy to have him on board!

Finally (but certainly not least), we've brought on Thom McGrath to lead our internal IT needs. He's another long-time REALbasic user that you're probably familiar with (he writes RB Developer articles, as well as posts frequently on the NUG). He's going to be in charge of making sure our internal development needs are met -- for instance, making sure the forums are running smoothly, watching over our order system, etc. All of the fun tasks you'd expect out of an IT guru.

While you certainly don't have to... I'd still like you to join me in welcoming all of our latest additions. Everyone plays a vital role within REAL Software and I'm tickled pink to have them on board!

That being said, we're still looking for at least one more person. If you're interested, please send your resume to hr@realsoftware.com

So what do these sundry titles actually mean?

As lead compiler architect, I am in charge of language direction, design and implementation. New compiler features, compiler-related bug fixes, etc are all things which eventually land on my plate. Before you start to fret that I'm going to totally murder the REALbasic language, let me put your mind at ease. I have no intentions of destroying the language. ;-) I certainly have plans and ideas, but many of them come from years of discussion with Mars. For the immediate future, most of my linguistic plans basically revolve around bringing previously planned language features to light.

I'm very excited about this change of scenery with regards to my daily programming topics. I've always wanted to work on the compiler, and have spent many months training for this transition with Mars. In fact, I've been working solely on the compiler for about the last six to eight months and have several shipping features under my belt (along with dozens of fixes and a few refactorings to boot). Both Mars and I are very confident in my ability to step into the role and perform to everyone's expectations.

The additional title of lead programmer is a somewhat new concept internally. We've always had the concept of "person we look to for technical answers", as does any team of programmers. However, we've not had a defined role for the lead programmer to fill. Recently, we've realized that we need a person in charge of the technical day-to-day aspects of running the programming staff. I'm very happy and excited to be that person.

In addition to my daily programming duties, I now have some extra duties. Some of them are technical in nature, as you would expect. Others are managerial in nature, which is a very interesting new area of growth for me.

One of the new managerial duties I have is to ensure my staff is adequate. We're currently looking to hire programmers with strong REALbasic, OOP, and database skills. So if you think you have what it takes to work on the REALbasic IDE with a team of passionate and dedicated people, please send me your resume and an example of your best code.

All-in-all, I'm very excited about the changes in my professional career. The technical and personal challenges these changes pose are certainly welcome! I know I can rise to meet them and help make REALbasic even better (boy, don't I just *sound* like a manager now!!).

  dim d as Double = -1.0

  dim u as UInt32 = d

  

  MsgBox Str( u )

double d = -1.0;

unsigned long u = (unsigned long)d;

printf( "%d\n", u );

When you disassemble the source code, and trace your way through it, the problem will become clear -- the code generator is constraining the results within the range 0..MAX_UINT. What's bizarre about that is: it's not needed, and it's the cause of the issue. Any negative values end up getting mapped to zero, which is silly because the conversion is to UInt.

The reason REALbasic demonstrates the same issue is because we make use of an intrinsic instead of doing the conversion ourselves. We use intrinsics to do conversions as a way to save on space (it takes less space to call a function than it does to do the conversion), and so our conversions suffer from the same problems as the XCode application.

So watch out for this one, as it can be particularly painful. Thankfully, it only affects conversions from double to uint32 -- other conversions all appear to be fine in quick tests. We've filed a bug report with Apple, but since Radar appears to be a closed system, no one gets to see how the report progresses. :-P

With my blog, it's more of a "push" model -- if I want everyone to know about personal stuff, such as my wedding, then I can push the content out to my blog and anyone can see it. But Facebook is different in that I tend to use it to keep in touch with personal friends, bitch about problems (both work and personal), etc. So the question becomes: do I want to allow RB users to view my Facebook profile?

The answer is: no, I don't want that. Believe it or not, but I have to watch what I say on my personal blog. ;-) Even though it's personal, I still represent REAL Software and that can affect people's purchasing decisions. So I don't spout off about the idiot neighbor and their damnedable barking mutt from Hell (often), and tend to discuss more professional matters. (Note: no one from work censors my blog, so my blog isn't just a front for REAL Software.) But I certainly make use of Facebook for that sort of thing since I don't have to worry about offending anyone, coming across unprofessionally, etc. And the lack of worry is what makes Facebook interesting to me, and I don't want to lose that.

So! If you happen to "friend" me on Facebook, don't be too upset with me if I decline the invitation. It's not that I don't like you, because honestly, I do! It's mostly a matter of keeping my personal life separated from my work life. Thanks for understanding!

One
Two
Three

Enjoy!

1) If you have to ask, you probably don't need it
2) When you need it, you'll know

Very deep, eh? ;-) Well, I'm going to take a stab at giving a general and specific response. But I have to admit that I also agree with #1 and #2 above. I'm simply explaining away the corollary question of: "but if I won't need this, why did RS add it?"

Let me start with the last question first: why did we add Introspection support? The short answer is: because we needed it. Of course, it certainly helps that it was the single most-requested item in the feedback system! But at the end of the day, it turns out that we required it in order to provide more powerful features to the user. So now that we have that question answered, I can focus on: why do you need introspection?

I've given examples previously about serialization and unserialization, among other things. And those are very valid uses for introspection. But the more general answer is: because it provides a powerful mechanism that allows programmers to accomplish things they couldn't otherwise do previously (as easily). That's the main reason *any* new compiler feature comes along. Compilers are meant to take the repetitive tasks and abstract them away so that the programmer can focus on accomplishing their goals.

For instance -- why have a for loop, or while loop? After all, they can be accomplished by just using a counter/controller and a goto! The reason is: because it's easier, cleaner and more maintainable!

The same is true of any compiler feature, and introspection is no different. You could serialize and unserialize things before -- you just had to write the code to accomplish them. Same for cloning, or any of the other introspection-friendly features people discuss. The difference is, there's now less code to write.

Introspection gives you a generic way to interrogate an object for its data, and manipulate the object's state. This is a powerful new technique -- but it's also dangerous. So most people tell you "omg, zounds! don't use it!" and they're partially right. Introspection can be a very fragile way to accomplish the same goals that could be accomplished without it. For instance, calling someObject.SomeMethod is less fragile than looking up a method named "SomeMethod" on an instance of someObject and calling Invoke on it. Why? Well, for starters -- just calling the method takes less code! But it's also less fragile because you can rename SomeMethod and get a compile error if you've forgotten to rename its usage everywhere. But with introspection, all of the work happens at runtime -- no compile errors! And errors that happen at runtime are just a fancy name for bugs.

So if you can do all of these things before, and this is so scary and bad to use -- why would RS (or anyone) ever want to make use of it? Because sometimes there's no way around generic programming when you're designing a toolkit.

Let's say you want to have a generic base class that performs a bunch of work on behalf of a subclass. The base class does not have access to the subclass' properties -- so if the base class wants to assign to those properties, it has to come up with some mechanism to have the subclass do the assignment. This gets pretty complicated very quickly. It's also a terrible example to try to talk out in this fashion, so let me get a bit more specific with a real world example:

Let's say that we (REAL Software) want to expose an object model that represents a database to our users. We will generate a class that represents a table, and inside of that class are a bunch of properties that represent the columns. However, we don't want to shove a whole bunch of source code into these generates classes. If we do that, then it suddenly becomes very difficult (if not impossible) to push bug fixes out to users with any of that source code's logic. Instead, we want the logic to live in a base class that we control and that isn't exposed. By doing this, we can fix the bug in one place (if any crop up, of course) and everyone benefits. This also makes the classes we generate a lot cleaner, and less scary.

Now comes the tricky part -- these classes have properties in them that represent the data. So we need a generic way to map the data from the database query onto the concrete class from within the base class. This is where introspection comes in handy -- the base class can do something like this:

dim ti as Introspection.TypeInfo = Introspection.GetType( self )

dim props() as Introspection.PropertyInfo = ti.GetProperties

for each prop as Introspection.PropertyInfo in props

  if prop.Name = "The Database Column Name" then

    prop.Value( self ) = "The Database Column Value"

    exit

  end if

next prop

This is just one of the many uses (or abuses, depending on your school of thought) that you can find for Introspection. There are plenty of other reasons to use it, of course. But if you're pondering why we implemented the feature, hopefully this adds another example to the pile already given. But I still stand by the majority's opinions that you probably won't need introspection in your day to day life -- but when you do need it, it's very hard to live without it.

Oh, and by the way -- I appreciate that you might want to point out all sorts of solutions to the example I gave. You can refrain from doing that though. I'm aware there's always more ways to skin a cat (trust me, I have multiple cats in the house!), and this was just an example. :-)

These thumbnails are a live view of what's happening in the window, and are quite a neat way to visually see what's happening "behind the scenes" so to speak. However, they have some very unfortunate drawbacks which are really annoying.

For starters -- you can only display a thumbnail for a top-level window. This totally destroys them for the use I had in mind, as for other great places to use this. The task bar is nothing more than a fancy name for a tab panel, and so it stands to reason that most places you'd use a tab panel would be a great place to use DWM thumbnails. For instance, in the REALbasic IDE, I was going to have a thumbnail appear when you hover the mouse over a tab in the tabs bar. This way you get a more visual understanding of what's being edited in a particular tab. But since it requires a top-level window to work... that's just not going to happen.

Now... I do not understand why this limitation is required. It seems like a fairly contrived limitation. However, I also have only a cursory understanding of how the thumbnails work internally -- so it could be that there's a valid reason for the limitation. It's just unfortunate, because it totally killed a really fun feature for the IDE.

And, just in case you want to use thumbnails in your own applications -- here's another tidbit of information. DWM will only work with a *visible* window. If the window is minimized, then DWM will cache the previous contents of the window and display that. It's interesting to note though, that if DWM doesn't have the opportunity to cache the window's image, you get a stock "blank" window thumbnail. This is the same thumbnail you will get if the window the window is outright hidden. However, if the window is simply off-screen, then it will display just fine (but it will also show up in the Alt+Tab list and accept focus/input, so it's not very hidden!).

Regardless, I've got some rudimentary DWM support in REALbasic code. It's nothing too fancy, but it gets the job done.

Structure DwmThumbnailProperties

  flags as Integer

  destination as Rect

  source as Rect

  opacity as UInt8

  visible as Integer

  sourceClientAreaOnly as Integer

End Structure

 

Structure Rect

  left as Integer

  top as Integer

  right as Integer

  bottom as Integer

End Structure

 

Soft Declare Function DwmRegisterThumbnail Lib "DwmApi" ( dest as Integer, source as Integer, ByRef id as Integer ) as Integer

Soft Declare Function DwmUpdateThumbnailProperties Lib "DwmApi" ( id as Integer, ByRef properties as DwmThumbnailProperties ) as Integer

Soft Declare Sub DwmUnregisterThumbnail Lib "DwmApi" ( id as Integer )


dim id as Integer

 

if DwmRegisterThumbnail( self.Handle, sourceWindow.Handle, id ) = 0 then

    dim properties as DwmThumbnailProperties

    

    properties.flags = &h8 + &h1 + &h4 + &h10

    properties.destination.top = 22

    properties.destination.right = self.Width

    properties.destination.bottom = 22 + self.Height

    properties.opacity = 255

    properties.visible = 1

    properties.sourceClientAreaOnly = 0

    call DwmUpdateThumbnailProperties( id, properties )

end if

Enjoy!

For starters, you'd have to be able to create an object instance. This sounds simple, but it happens to have some interesting quirks. Plugin authors have always had access to an API call REALnewInstance. This took a string, or a REALclassRef, and would return back to you a new instance of the object. However, in practice, this is a terrible idea for several reasons. For starters, if you pass in a string, the compiler has no way to know that the plugin relies on a given class. This means that a plugin can call REALnewInstance( "Dictionary" ), but the linker dead-strips out the Dictionary class if the user doesn't make use of one within their project. Since most authors don't check to see if a call to REALnewInstance fails -- this basically leads to very hard to reproduce bugs with plugins. So we introduced the REALclassRef version of the API as a way for the user to alert REALbasic that various classes are required. In the PluginEntry function, the user call REALGetClassRef as a way to get a class reference, which is the part that alerts REALbasic to the use of various classes (at compile time, hence the reason it needs to happen within PluginEntry). Problems solved, right?

Wrong... that only solves the dead-stripping problem. There's one more annoying issue that most people don't realize -- the object returned isn't fully initialized because no constructors have been called! The plugin author is required to look for a constructor method and invoke it manually! This is also something a lot of people forget to do, and can cause a lot of obscure issues.

So if we're going to design a way to create an object instance in REALbasic, we want to solve these kinds of issues. So there should be no way to create an object via introspection that causes dead-stripping issues, and it must always create a fully-constructed object.

Well, for starters, let's say that there's no way to get an object instance via string -- that solves the dead-stripping problem by requiring the user to always use actual class references within their project. But we'll come back to this in a moment...

To solve the half-constructed problem is actually pretty easy -- have a way to get a list of constructor methods from a given TypeInfo object. Calling any of these constructor methods via Invoke always returns a new instance of the actual object, and calls the constructor automatically. In this way, it is impossible to get a half-constructed object since the only way to create an object is by calling its constructor. This of course makes the assumption that all objects have a constructor method, but that's easily abstracted away by returning a "fake" constructor method that takes no params -- it just creates the object instance, and doesn't need to call any actual constructor method since none exists.

Ok, so now we know how to make object instances. However, this is kind of silly too -- the only way to get a TypeInfo object is via an object instance itself! That means you'd have to do something like this:

dim ti as Introspection.TypeInfo = Introspection.GetType( new Class1 )

dim c as Class1 = ti.ConstructorMethods( 0 ).Invoke

So let's make this a bit more interesting -- let's define an operator which returns a TypeInfo object without requiring an object instance. Some may be tempted to say "aha, now we can make a TypeInfo from a string and create objects that way!" Please, don't give in to that temptation (yet). That still runs into the dead-stripping problem, since the linker may or may not know about the class type. Remember, the following code would be perfectly legal:

dim v as Variant = MagicalTypeInfoOperator( "Class1" ).ConstructorMethods( 0 ).Invoke

dim v as Variant = MagicalTypeInfoOperator( Class1 ).ConstructorMethods( 0 ).Invoke

Well, hold on now! This doesn't exactly solve the original problem -- how do we unserialize an object from disk? In that case, we don't have an actual class reference handy, so all this work has been for nothing! Hah, not really... I assure you. ;-) But we do have to make some allowances, to be sure.

When you're making a disk format, or a wire format, 99% of the time you do not want the external source to make arbitrary object instances. That would be a major security hole, after all. Imagine if I were able to write out a FolderItem instance and start calling .Delete on it, all via the wire. Oops, your hard drive is gone. ;-) Generally, you know what object instances are legal to unserialize for your particular needs. Given that as a premise, the solution to the problem becomes quite easy -- wrap a dictionary with some extra functionality.

We'll define one method called: Sub LegalToCreate( ti as Introspection.TypeInfo ), and another method called: Function CreateInstance( name as String ) as Variant. The first method is the way we set up the list of items that are legal to create, and the second method is the way we create instances by name. The functions are quite simple:

Sub LegalToCreate( ti as Introspection.TypeInfo )

  mMap.Value( ti.Name ) = ti

End Sub


Function CreateInstance( name as String ) as Variant

  dim ti as Introspection.TypeInfo = mMap.Lookup( name, nil )

  if ti <> nil then return ti.ConstructorMethods( 0 ).Invoke

End Function

LegalToCreate( MagicalTypeInfoOperator( Class1 ) )

dim v as Variant = CreateInstance( "Class1" )

Of course, all of this is purely academic right now since the constructor methods and magical type info operator do not exist. However, it's neat to think about the things you could do if they did exist, right?