Remote Debugger Stub Post Mortem (part three and a half)

TCP Communication Protocol for new Remote Debugger Stub
Version 1.0.2
by Aaron Ballman
Latest: Mar 18 2005
Original: May 14 2004

Problem Set:
The IDE needs a way to communicate with an entity on a remote machine to do simple (one-way) file transfers for the purposes of remote debugging as well as remote deployment. The IDE needs to be able to send over individual files or folders. It also needs to be able to direct when the remote entity should remove the files (if at all), or whether to launch files (and how the files are to be launched). For the purposes of this document, the remote entity will be called the "stub".

--------------------------------------------------------------------------------

Assertions:
The IDE is responsible for establishing the TCP connection with the stub.
The IDE is responsible for terminating the connection with the stub except in extreme circumstances.
All binary data is transferred in Big-Endian form
All data in the packets that is marked as being reserved can be safely ignored and is not considered malicious.
After the Initial Handshake, any unknown Command IDs can be safely ignored and are not considered malicious.
All string data is sent as UTF8.

--------------------------------------------------------------------------------

Process:
Once a connection has been established, the stub will wait for a HELLO packet. Once it receives this packet, it will send a response packet (called a OLLEH packet) back to the IDE. These two steps are called the "Initial Handshake". If the packet does not arrive in a timely manner, the stub should assume that the connection is malicious and terminate it.

To complete the Initial Handshake, both the IDE and the stub need to check to make sure they can work with the version information provided. If the stub cannot work with the IDE due to the version provided, then stub then needs to send an SOS signal back to the IDE. If the IDE cannot work with the stub, then it should alert the user and terminate the connection. Given that no SOS packet is sent to the IDE from the stub, and the IDE does not terminate the connection, then the handshake is complete.

Part of the handshake process is determining which protocol version to use. When establishing a connection, the IDE will send over the protocol version it is using. The stub will then send back what protocol version it is using. Both the IDE and the stub are expected to use the lowest common protocol version when communicating. It is acceptable that some features may be disabled due to this. Protocol version numbers start at 1 and are always in integer values. Each time the protocol changes, the version is incremented by 1, with a possible 65535 protocol changes.[1]

Once the handshake is completed, the stub is then ready to process commands that come from the IDE. If the handshake has not completed and any unknown data (read: any data that is not part of the hand shaking protocol) is received, that data should be considered malicious and the stub should terminate the connection.

After the Initial Handshake, the IDE will send a query to the stub and then immediately goes into a wait loop to wait for the response (usually). If the response does not arrive in a timely manner, then the IDE can assume there was an error, display a message to the user asking whether they would like to continue waiting, and possibly terminate the connection.

When the stub receives a query, it should process it and send the response back to the IDE. The stub cannot assume any order to the queries it receives as they can arrive in any order.

The IDE needs to know what file system the stub is running so that it can build the proper executable to send. When the IDE wishes to know the platform to build for, it will send the stub a PLATFORM-REQUEST packet. The stub will then send back a PLATFORM-RESPONSE packet after determining the platform information.

The IDE knows the size of the file that it is about to send, along with all of the information about the file, such as the file's name, whether it's compressed, etc. The stub needs to know this information in order to write the proper file out, whether to unpack the file, etc. First, the IDE will send a CREATEFILE packet to the stub. The stub will then attempt to create the file based on the information the IDE provided. The stub then sends a FILECREATION packet back to the IDE to let the IDE know it was able to create the file (or not). This packet will also contain a file identification number that the stub creates. Even if the file was not written properly, this code still identifies the file. The IDE will then send a FILEINFO packet to the stub which the stub will then store for later processing. This FILEINFO packet signifies the start of the file transfer and contains information about the file size, whether the file is compressed or not, etc. The stub will then receive a series of FILEPART packets that contain the file identification number, and binary data to put into the file.

Once the stub receives notice that the file is coming, the stub should have already opened a file on the local machine for writing. The file will be sent in chunks by the IDE. Each chunk is called a FILEPART, and as each FILEPART message is received, the data portion of the message will be written to the file. When the IDE is finished sending the file, the last FILEPART packet will have a nil data portion. This signfies that the stub can close the file because the transfer is complete.

Once the stub has closed the file, it should report back to the IDE that the file has been written out successfully. It does this with a FILEWRITTEN packet. This packet will contain status information based on calculations it does. For example, the stub will ensure that the file size the IDE reported is the same as the actual size of the file written out. After sending this packet out, the stub will add the file to a list of files to be deleted in the future if it was told to do so by the IDE. It will also unpack the file if the file was reported as being compressed (replacing the original with the unpacked version).

Finally, when the IDE wants the stub to launch an application, it will tell the stub which file to launch. It tells the stub to launch the file by sending a FILELAUNCH packet and passes in the file identification code corresponding to the proper file. When the stub gets this packet, it should attempt to launch the file. One piece of information the IDE will send with the FILELAUNCH packet is whether the stub should attempt to launch the file from within a terminal. If the file should be launched within a terminal, the stub will have to take the appropriate actions on the local machine before launching the application. If a terminal window cannot be launched, or the application itself cannot be launched, then the stub will send an SOS packet to the IDE.

When the IDE is finished communicating with the stub, it will terminate the connection with a one-way handshake. The IDE will send the stub a SEEYA packet and then terminate the connection. If the stub gets a SEEYA packet and the connection does not terminate within a reasonable amount of time, the stub should terminate the connection. Any data that the stub receives after a SEEYA packet is to be ignored. If the connection is terminated before receiving a SEEYA packet, then the stub should clean up all of the files for that session.

Miscellaneous:
The IDE needs a way to tell the stub to delete a specific file. This action won't be used very often since the stub will usually be in charge of removing files when it gets the chance. To remove a file, the IDE sends the stub a FILEREMOVE packet. The stub does not respond to this packet.

The IDE may need to have the stub create folders. When this happens, the IDE will send the stub a CREATEFOLDER packet. This packet contains the name of the folder to create along with a path specifier. If the path specifier is blank, then the folder is assumed to be relative to the base directory. There is a flag that denotes whether the path specifier is relative or absolute. When the stub creates the folder (or if it fails to create it), the stub will send back a FILECREATION packet to let the IDE know whether the operation succeeded or not.

At any point in time, either the stub or the IDE may issue a CHALLENGE packet. When this packet has been issued, the other side must respond with a RESPONSE packet. If any packets arrive after the CHALLENGE packet but before the RESPONSE packet, they must be stored for later processing after the RESPONSE packet comes in. If the RESPONSE packet does not match the expectations, then the connection is to be considered malicious and should be terminated immediately (and all files cleaned up). This challenge-response system will be described later.

The default port for both automatic discovery and all other communications (both TCP and UDP) is 44555. The IDE is not required to automatically discover stubs when the stubs use a non-standard port assignment. However, the IDE must provide a way for the user to enter an IP (and optionally, a port) to locate the stub directly.[1]

The multicast group address for automatic discovery is 226.34.56.76 This address cannot be changed by the user.[1]

--------------------------------------------------------------------------------

Challenge/Response System:
Both the stub and the IDE will implement a challenge response system. Either system can initiate a challenge, and the other system must respond immediately to the challenge. The CHALLENGE packet's payload will consist of a single encrypted string. The encryption method must be known by both systems and can be any reasonable form of encryption that uses a key to do the encryption. The challenge string will be an auto-incrementing counter[3], followed by a space, followed by the challenger's IP address, followed by a space, followed by the respondant's IP address. This string will then be encrypted using the user's password. For example:

123EFA 10.10.10.117 10.10.10.116

would be the plain-text challenge string. The responder will then decrypt this string. If it cannot locate its IP address and the remote machine's IP address in the decrypted message, then the challege fails and the connection is terminated immediately. Also, if the serial number does not match the next in the sequence then the connection is terminated[3]. If it can locate the proper IP addresses, then it will send a response string in the form: serial number[3], space, responder's IP address, space, challenger's IP address. So, in response to the above challenge, the responder would return the following string (encrypted with the user's password):

123EFB 10.10.10.116 10.10.10.117

The challenger can then verify that the IP addresses have come back swapped from what they were originally sent out as. If the IP addresses do not match the swapped version of the originals, or the serial number does not match the next in the sequence[3], then the challenge failed and the connection must be terminated immediately.

The benefit to this system is that each side can verify a chunk of known information without having to share any unknown data with the remote machine. By sending IP addresses across, each side can verify that they are talking to who they think they are talking to. And since the data is sent over encrypted with the user's password in each case, the system is secure. The inclusion of the serial numbers ensure that the encrypted strings are constantly changing. Because the encryption string is reasonable, small changes to the decrypted string should result in large changes to the encrypted string, which in turn protects the scheme against spoofing the response packet.[3]

If either side cannot decode the challenge string (not including the auto-incrementing serial number[3]), they can optionally prompt the user to enter a password to retry the decoding. All of this happens locally and does not affect the challenge/response system protocol.[1]

The initial serial number does not have to be determined dynamically. It can be a constant value that is known to both the stub and the IDE (much like the encryption scheme must be known to both parties).[3]

The local IP address written out and checked against in the challenge/response system is determined during the initial handshake. This is done so that stubs and IDEs can communicate even if there is a NAT device between them.[2]

--------------------------------------------------------------------------------

File Paths:
Any packet that sends file paths can send the path in one of two ways: relative or absolute mode. Relative mode works relative to where the stub is located on the user's machine. So, for example, if you don't specify any path information, then the file or folder will appear in the same folder as the stub itself. If you specify a relative path and use the string "My Folder", then the file or folder will appear inside a folder named "My Folder" which is next to the stub.

Because each OS has a different path separator, and the stub and IDE can run on different platforms, path separators can be a problem. To get around this issue, all path parts must be separated by a null character [Chr( 0 )], which is designated in this document as \0. So, if you wanted to specify a relative path that contains a few folders, then the string would look like this:

Folder One\0Folder Two

The stub is responsible for converting the \0 into the proper path separator for the platform that it is on.

All relative paths are relative to the stub's location and can only specify directories "below" or "inside" of the folder the stub is in. This means that you cannot specify the stub's parent directory with relative paths. This is due to the fact that the stub can reside anywhere and may not have a parent path.

Absolute paths are to be avoided like the plague. They follow the same path separator rule as decribed above and should only be used in very special circumstances due to the major differences in file system structures.

-------------------------------------------------------------------------------

Packet Descriptions:
CHALLENGE Packet:
Command ID: 0x00CA1133
Data = Variable length string of challenge data

CREATEFILE Packet:
Command ID: 0xC4EAF11E
Data = 4-bytes for the string's length
4-bytes reserved for future use
Variable length string for the files's name
1-byte specifying whether the path is relative or absolute (0x0 if relative, 0x1 if absolute)
7-bytes reserved for future use
4-bytes for the string's length
4-bytes reserved for future use
Variable length string for the path specifier

CREATEFOLDER Packet:
Command ID: 0xC4EAF0DE
Data = 4-bytes for the string's length
4-bytes reserved for future use
Variable length string for the folder's name
1-byte specifying whether the path is relative or absolute (0x0 if relative, 0x1 if absolute)
7-bytes reserved for future use
4-bytes for the string's length
4-bytes reserved for future use
Variable length string for the path specifier

FILECREATION Packet:
Command ID: 0xFC4EA710
Data = 4-bytes of status indication code. This code corresponds to the error code of the folder item the stub attempts to create.
4-bytes for the file identification number

FILEINFO Packet:
Command ID: 0x0F1131F0
Data = 4-bytes for the file identification number
8-bytes (double precision) designating the file size
1-byte designating whether the file is compressed or not (0x0 if false, 0x1 if true)
1-byte designating whether the stub should remove the file when it gets the chance or not (0x0 if false, 0x1 if true)
6-bytes reserved for future use

FILELAUNCH Packet:
Command ID: 0x0F1131AC
Data = 4-bytes for the file identification number
1-byte specifying whether to launch from a terminal (0x0 for no terminal, 0x1 for a terminal)
7-bytes of reserved for future use

FILEPART Packet:
Command ID = 0xF1139A47
Data = 4-bytes for the file identification number
A binary string containing the next chunk of the file to be written out. If this string is nil, then the file is to be closed.

FILEREMOVE Packet:
Command ID: 0xF1134303
Data = 4-bytes for the file identification number

FILEWRITTEN Packet:
Command ID: 0xF113D073
Data = 4-bytes for the file identification number
4-bytes of status indication code.

HELLO Packet:
Command ID = 0x0017E110
Data = 2-bytes major IDE version number
2-bytes minor IDE version number
2-bytes bug IDE version number
1-byte release code
1-byte candidate number
2-bytes of protocol version information [1]
2-bytes of reserved for future use [1]
4-bytes of integer remote IP address information [2]

Example Data (as hex): 0017E110 000500005 00020303 00010000 0A0A0A74
(as string): 0x17E110 5.5.2fc3, Protocol Version 1, 10.10.10.116

OLLEH Packet:
Command ID = 0x011E7100
Data = 2-bytes major stub version number
2-bytes minor stub version number
2-bytes bug stub version number
1-byte release code
1-byte candidate number
2-bytes of protocol version information [1]
2-bytes of reserved for future use [1]
4-bytes of integer remote IP address information [2]

PLATFORM-REQUEST Packet:
Command ID = 0x091A4E57
Data = No data will be sent with this request. Any data that is sent can be safely ignored by the stub and will not be treated as malicious.

PLATFORM-RESPONSE Packet:
Command ID = 0x91A4BAC8
Data = 2-bytes describing the platform (0x1 = Windows, 0x2 = Macintosh, 0x3 = Linux)
1-byte describing the system architecture (0x1 = x86, 0x2 = PPC)
1-byte describing the file format (0x1 = PE32, 0x2 = PEF, 0x3 = Mach-O, 0x4 = ELF)
4-bytes for the length of the string
4-bytes reserved for future use
Variable length string describing the platform

Example Data (as hex): 91A4BAC8 00020203 0000000D 000006000 "Mac OS 10.3.3"
(as string): 0x91A4BAC8 Macintosh, PPC, Mach-O, 13 bytes long, Mac OS 10.3.3

RESPONSE Packet:
Command ID = 0x04E59053
Data = Variable length string of response data

SEEYA Packet:
Command ID: 0x0000533A
Data = No data will be sent with this request. Any data that is sent can be safely ignored by the stub and will not be treated as malicious.

SOS Packet:
Command ID = 0x00000505
Data = 4-bytes for the length of the string
4-bytes reserved for future use
Variable length string that to be displayed to the user.

--------------------------------------------------------------------------------

Example Successful Exchange Between the IDE and a Stub for Remote Debug:
IDE->Stub HELLO, 6.0.0a1, Protocol Version 1, Stub IP address
Stub->IDE OLLEH, 2.0.0a1, Protocol Version 1, IDE IP address
IDE->Stub PLATFORM-REQUEST
Stub->IDE PLATFORM-RESPONSE, Windows, x86, PE32, Windows XP Professional
IDE->Stub CREATEFILE, "My Application.exe"
Stub->IDE CHALLENGE, Serial # - Stub IP Address - IDE IP Address
IDE->Stub RESPONSE, Serial # + 1 - IDE IP Address - Stub IP Address
Stub->IDE FILECREATION, No Error, File ID: 0
IDE->Stub FILEINFO, File ID: 0, 1024K, Not Compressed, Delete When Finished
IDE->Stub FILEPART, File ID: 0, Some File Data
IDE->Stub FILEPART, File ID: 0, More File Data
IDE->Stub FILEPART, File ID: 0, Last of the File Data
IDE->Stub FILEPART, File ID: 0, nil
Stub->IDE FILEWRITTEN, File ID: 0, No Error
IDE->Stub FILELAUNCH, File ID: 0, No Terminal
Stub->IDE CHALLENGE, Stub IP Address - IDE IP Address
IDE->Stub RESPONSE, IDE IP Address - Stub IP Address
IDE->Stub SEEYA

Example Unsuccessful Exchange Between the IDE and a Stub for Remote Debug 1:
IDE->Stub HELLO, 9.5.0a1, Protocol Version 1, Stub IP Address
Stub->IDE OLLEH, 2.0.0a1, Protocol Version 1, IDE IP Address
IDE Terminates Connection, Stub Cleans Up

Example Unsuccessful Exchange Between the IDE and a Stub for Remote Debug 2:
IDE->Stub HELLO, 6.0.0a1, Protocol Version 1, Stub IP Address
Stub->IDE OLLEH, 2.0.0a1, Protocol Version 1, IDE IP Address
IDE->Stub PLATFORM-REQUEST
Stub->IDE PLATFORM-RESPONSE, Windows, x86, PE32, Windows XP Professional
IDE->Stub CREATEFILE, "My Application.exe"
Stub->IDE FILECREATION, 104, File ID: 0
IDE Terminates Connection, Stub Cleans Up

Example Unsuccessful Exchange Between the IDE and a Stub for Remote Debug 3:
IDE->Stub HELLO, 6.0.0a1, Protocol Version 1, Stub IP Address
Stub->IDE OLLEH, 2.0.0a1, Protocol Version 1, IDE IP Address
IDE->Stub PLATFORM-REQUEST
Stub->IDE PLATFORM-RESPONSE, Windows, x86, PE32, Windows XP Professional
IDE->Stub CREATEFILE, "My Application.exe"
Stub->IDE CHALLENGE, Serial # - Stub IP Address - IDE IP Address
IDE->Stub RESPONSE, Some Malformed String
Stub Terminates Connection, Stub Cleans Up

Example of the IDE Sending Multiple Files to the Stub:
IDE->Stub HELLO, 6.0.0a1, Protocol Version 1, Stub IP Address
Stub->IDE OLLEH, 2.0.0a1, Protocol Version 1, IDE IP Address
IDE->Stub PLATFORM-REQUEST
Stub->IDE PLATFORM-RESPONSE, Windows, x86, PE32, Windows XP Professional
IDE->Stub CREATEFOLDER, "My Folder"
Stub->IDE CHALLENGE, Serial # - Stub IP Address - IDE IP Address
IDE->Stub RESPONSE, Serial # + 1 - IDE IP Address - Stub IP Address
Stub->IDE FILECREATION, No Error, File ID: 0
IDE->Stub CREATEFILE, "My File 1"
Stub->IDE CHALLENGE, Serial # + 2 - Stub IP Address - IDE IP Address
IDE->Stub RESPONSE, Serial # + 3 - IDE IP Address - Stub IP Address
Stub->IDE FILECREATION, No Error, File ID: 1
IDE->Stub CREATEFILE, "My File 2"
Stub->IDE CHALLENGE, Serial # + 4 - Stub IP Address - IDE IP Address
IDE->Stub RESPONSE, Serial # + 5 - IDE IP Address - Stub IP Address
Stub->IDE FILECREATION, No Error, File ID: 2
IDE->Stub FILEINFO, File ID: 1, 1024K, Not Compressed
IDE->Stub FILEINFO, File ID: 2, 2048K, Not Compressed
IDE->Stub FILEPART, File ID: 1, Some File Data
IDE->Stub FILEPART, File ID: 2, Some File Data
IDE->Stub FILEPART, File ID: 1, More File Data
IDE->Stub FILEPART, File ID: 1, Last of the File Data
IDE->Stub FILEPART, File ID: 2, More File Data
IDE->Stub FILEPART, File ID: 1, nil
Stub->IDE FILEWRITTEN, File ID: 1, No Error
IDE->Stub FILEPART, File ID: 2, More File Data
IDE->Stub FILEPART, File ID: 2, Even More File Data
IDE->Stub FILEPART, File ID: 2, Even More File Data
IDE->Stub FILEPART, File ID: 2, Last of the File Data
IDE->Stub FILEPART, File ID: 2, nil
Stub->IDE FILEWRITTEN, File ID: 2, No Error
IDE->Stub SEEYA

--------------------------------------------------------------------------------
Modifications:
[1] Mar 17 2005 Aaron Ballman Added protocol information to the handshake process. Also fleshed out port and auto-discovery information.

[2] Mar 18 2005 Aaron Ballman Modified the initial handshake and challenge/response system to allow for communications across NAT devices (thanks to Brad Hutchings).

[3] Mar 18 2005 Aaron Ballman Modified the challenge/response system to have an auto-incrementing serial number to increase the security of the protocol (thanks to Tim Jones).