Well I'll be Dipped

This is a bug that's bothered me for a while because it seemed so random to me: the mask for a TrayItem was sometimes ignored, even though you could draw that same picture to a Canvas and it would display fine.

So it turns out that the issue was with a call to the Win32 API CreateIconIndirect. This handy function allows you to create an icon (or a mouse cursor) by specifying the bitmaps for the image and the mask. Since the underlying OS support for tray icons deals with an HICON, I needed to convert a REALbasic picture object into an HICON object and this was the API to do it.

However, I failed to notice the remarks on MSDN. Specifically: "Because the system may temporarily select the bitmaps in a device context, the hbmMask and hbmColor members of the ICONINFO structure should not already be selected into a device context. The application must continue to manage the original bitmaps and delete them when they are no longer necessary."

The reason the mask was being ignored was because it was already selected into a device context. But the image itself wasn't! It turns out that we cache the device contexts under the assumption that if we used it once, we'll probably use it again sometime soon. Since the mask was the last item drawn into picture being passed to the TrayItem class, it was selected into a cached device context.

What makes this bug tricky is that most Win32 APIs don't care if a bitmap is already selected into a device context. Most APIs just assume the state is stable and use the bitmap handle anyway. However, this API is sort of an odd-man-out. It never occurred to me that this might be the issue, and it wasn't until I went back to double-check the ocumentation that I noticed the problem.

Mucking with the graphics caching system we have certainly wasn't something I was interested in doing. It's archaic, has its tendrils all over the place, and I was fearful of drastically slowing down all drawing just to fix the TrayItem class. Then I realized something else -- I can be in charge of the bitmaps myself.

Instead of using the bitmaps in the picture passed in, I simply clone the bitmaps and handle them myself. Since they're always small (certainly less than 32x32 in the normal case), this isn't a very lengthy operation. And it solves the issue in a very localized way that doesn't involve mucking with the caching system.

So ta da, bug solved. Just took me a while to track down what was the issue in the first place and come up with a reasonable solution to it. However, I was supposed to be done working over an hour ago. :-P