Remote Debugger Stub Post Mortem (part one)

As you may (or may not) know, the remote debugger stub was completely rewritten for RB 2005. I figure it'd be kind of fun for you to know why that is, as well as how it went.

The old remote debugger stub worked alright, but it had some design flaws and implementation issues that just couldn't be worked around. Before I get into what these issues are, here's a little information about how the old stub worked when actually remotely debugging (so I'll leave out the auto-discovery details).

1. The user would say 'Run Remotely'. Along with the menu command was a chunk of information (stored in the tag) which had the IP address of the remote machine.


2. A new class was instansiated which would communicate via UDP with the remote stub at the other location. It would first send a packet to make sure there was a stub there. So if you ever got stuck trying to find the remote stub, that's the stage you were stuck at.


3. When the remote stub would respond, the IDE would ask the stub what platform it was running on. The stub would then respond with an integer constant for the platform.


4. The IDE would then ask the stub what file format to build. This is because you could run either a Classic or Carbon app on OS 9. The thought was that we'd support Mach-O as well eventually.


5. Finally, the IDE would ask the stub for what TCP port to connect on when doing the file transfer.


6. Once the IDE knew the platform and link format, it was finally able to start the build. The UDP communicator class was torn down and the compiler started running. And by compiler, I really mean renderer.


7. The renderer would take all the project information and render it into source code to feed off to the compiler. Part of the rendered information was an IP address for the debug application to use to connect back to the IDE with. Once rendering was done, the compiler was fed the source code and would spit out an application.


8. If the app built succesfully, then a TCP connection was established with the remote debugger stub on the port the stub supplied earlier. As soon as the connection was established, the IDE would send over a header. This header had information in it like the file name and size.


9. Once the header was sent, the IDE would then start sending the file over. Once the file was sent, it would wait.


10. On the stub side, it would store the header information in memory (on a per-instance basis) and then read all data past the header directly into the file. Once the stub had received N bytes of data (where N is the file size gotten in the header), it would terminate the connection and launch the application from disk.


11. After the connection was terminated, the IDE would start listening on port 13897.


12. Once the application was launched, it would connect back (via the string baked in during rendering) to the IDE and the debug session was started.

As you can imagine, this process was an alright first-pass, but it left a lot to be desired for the following reasons:

• Since the protocol did most of its communication via UDP, it was a very volatile situation (especially when run over wireless routers). You couldn't always find the stub, and sometimes packets would be lost so the entire process would come to a screeching halt.


• Sending the file size over and putting the burden on the stub for determining when the file was completely downloaded was a bad idea. For example, I believe the reason you can't debug to OS 9 is because the stub never fully figures out the file size properly even though the entire file was sent. Because the stub doesn't terminate the connection, the IDE never starts listening and the process halts. Furthermore, if the network cable gets unplugged then the IDE starts listening and throws no errors (but the remote machine does balk since the entire file wasn't downloaded).


• There's no mechanism for sending anything other than files across. So debugging a Mach-O application or Carbon bundle is impossible.


• There was no versioning or handshaking so the entire protocol couldn't be modified without breaking things horribly.


• The entire process took too much effort -- you had to know to open up 2 UDP ports and 2 TCP ports. 1 UDP port for each direction (IDE to stub and stub to IDE were different ports). 1 TCP for the file transfer. 1 TCP for the debug session.


• There was no security involved -- you couldn't lock a stub down aside from saying that it wasn't a public stub (meaning it couldn't be auto-discovered).


• There was no multi-homing support for the stub. It just picked an interface and went with it. So if you had two network interfaces; one for the internal LAN and one for the WAN, you were screwed.

So yes, the remote debugger stub was functional much of the time. It was also a lot easier to configure than any other remote debugger stub I've ever used since it was automatically discovered, and the stub list sort of managed itself. However, it still sucked IMHO.

So that's why it needed rewriting. I'll discuss more about the rewrite tomorrow.