When I originally wrote EasyTCPSocket and EasyUDPSocket, they were mostly just for internal projects that I worked on. Basically some helper classes that I used for stupid little side projects. Both classes were very simple to use, but I never really intended to release them. However, I was soon convinced that they should be put into the product and stuck them in.
Now that I'm older and wiser, I've come to hate the underlying protocol that they use. It's extremely basic, and IMHO, quite worthless for its intended purpose. Here's why:
The intended audience for those classes are people who've never really done any network programming in their life. For example, they don't understand endian issues pertaining to networking. But the classes really don't do enough to help the user out! They only take them part of the way there, but then leave them scratching their head wondering "why isn't this working?" And if there's anything I've learned, it's that when a user sees something not working, the product is automatically called buggy. :-P But that's a digression. Here's an example of where a user can get caught -- they send a UTF8 string from one Computer A to Computer B. Computer B doesn't set the encoding, so when they stick the string into an EditField, it comes out garbled. Neither of the Easy classes deal with encodings! And what's worse... the protocol was designed sloppily, so the support can't be added! :: hangs head in shame ::
So I've been wanting to redesign the entire protocol for those two classes for a while now. But the problem is, if I do that, I'll end up breaking compatibility with older programs, which is unacceptable. So how do you take a crappy protocol that was never intended to be extended and make it backwards compatible? Why hackery of course. :-)
I figured out that I can stuff a magic value into the length field of the current protocol. If the newer application reads a non-magic number, it can assume that it needs to fallback on the old protocol. Also, it can assume it needs to send all messages out in the old protocol format. If it does read the magic value, then it can assume it's using the newer protocol and continue to use that (yay!) There's one place where this scheme breaks down (which may be obvious to the astute reader): What happens with EasyUDPSocket if the first packet is sent from the newer protocol to a program running the older protocol? Well... we have a problem then. I haven't quite worked it out fully yet -- but this is the closest I've come to being able to redesign the protocol in a backwards compatible fashion. Hopefully I'm able to figure out the last mile so that I can reimplement something I'm not happy with.
Or you could just remake the classes...call them HardTCPSocket and HardUDPSocket, and then bundle them along with the Easy classes
I would second that... Maybe call them QuikTCPSocket and QuikUDPSocket and focus on the fact that they do more "heavy lifting" than the UDP/TCP Sockets. Easy and Hard are subjective anyway.
While I agree the classes need to be recreated, making a second set of classes that do the same thing but with a better underlying protocol just isn't acceptable. Better to just tell users they're going to take a one-time hit and break their code than two put cruft of that nature into the product.
I'm just trying to avoid the breakage and do it in a way that's backwards compatible. It may be the sort of thing where I put in "fixes" to the current code base which don't cause breakage but make the new classes backwards compatible. Let the "Fixed" classes sit in circulation for a release or two and then roll in the new improved classes that are backwards compatible.
You're the boss! I was thinking along the same lines though. Except that the new classes would get out of the gate sooner and the old classes could be "advertised" as being a development deadend for a couple of releases before killing them off. Just a temporary redundancy to maintain backward compatability for at least a few previous releases.
like..Microsoft had Winsock in VB 6.0...and alot of apps are written in that, then in .NET, they don't have winsock, but the System.Net.Sockets classes.....broke alot of stuff. Alot of programmers that I have talked to were kind of mad, but sockets in .net give you more flexibility, but yet they are more complex that just using a Winsock control and settings some properties. The one thing with sockets is that Microsoft gave like one example for the longest time, and it was something very complex. Most people just want something easy...like connect to a telnet, or ftp using sockets. Or listen for syslogs, etc. I think if you give examples of something like that with newer, more complex classes, people will be fine with it, as long as there is a good reason for switching to the newer stuff.
Not to commandeer Aarons site for my own evil purposes but I have long thought that a book could be written and marketable based almost soley on examples. RB already has a verbose tome of explanatory logic with just a few examples (although a bit dated). What we don't have is a tome of examples with just a modest amount of explanation. I recall waaaaaay back in the 80's buying these "cookbooks" of code to write little games and such. They did say "how" they worked, but after a while (debugging due to poor copying) I figured out quite a bit... Right now I would kill (or at least maim) for some quality RBscript examples!
@Steve -- True, and if you saw just how many people jumped ship from Microsoft because VB.NET so completely broke old projects from previous versions of VB, you'd understand why breaking people's code is not an alternative. It's something we try to avoid whenever possible because it only causes headaches. Of course, sometimes it's inevitable -- but we always try to make it as painless as possible (like with the scheme I described).
@Joe -- From what I hear, there are a few RB books in the works. When I looked into writing a networking book, the problem I ran into was getting someone to publish. From what I understand, this is a common problem. So if you want to see RB books, I'd suggest contacting O'Reilly and telling them about it (and get everyone you know to do the same). If they see a market for the book to sell in, they'll publish.
I don't know how the protocol is under the hood (and don't have the time to look with Ethereal), but shouldn't you have thought of future compatibility earlier? Like have the client send a protocol version string to the server and vice versa?
-- SirG3
Yup, should have, but didn't. What's there works well enough for what people are using it for -- but I would still like the option to change things in the future. But que sera sera; it'll get fixed over the course of a few releases.
I don't think adding a duplicate set of classes would be particularly horrible. That's what we did with the REALdatabase, for pretty much the same reasons as the ones you're facing here. Just write new classes the way you want them, then rename the old classes "EasyTCPSocketOldProtocol" and "EasyUDPSocketOldProtocol". This gives anyone who cares about backwards compatibility an option, but lets most people move forward with future-proof code.
Also, it might be worth passing the new protocol around for a design review, so other people have a chance to help catch this kind of problem...
I plan on doing the same thing for the new classes as I did with the new remote debugger stub. Put together a pseduo-RFC draft and send it around for everyone to approve of and *then* work on the classes. I forgot about the fact that we did this with the old vs new database... Hmm.. well, since there's precedent, I don't see it being quite as bad as I did before.
Protocol work is fun, I hate doing GUIs. So, you talking sending the draft around to normal people, or just the RS employees? I probably couldn't help much, but it'd be fun to look at :-).
-- SirG3
Initially, probably just internally. But it's entirely possible that I'll end up talking about it publicly at some point (probably as either a whitepaper, an article, or just informally on my blog). But I must admit, I am slightly hesitant to openly hand out the protocol because I am sure people will try to abuse it somehow... Thoughts?
Abuse how though? It'd be nice for the protocol to be key/value based and be able to send dictionaries back and forth :-). I've actually been working on a protocol like that, I can write it up / send you the packet parser code if you'd like.
-- SirG3
Abuse like assuming the protocol will never change, so they try to write their own plugins that try the communications in a poor fashion (and break when the protocol gets updated two years down the line).
I've got the protocol I want in mind already, and partially sketched out on paper. Just no time to actually work it up into anything formal because of RB 2005.
If you clearly state that there's a version sent in the protocol and they should not attempt to handle newer protocols :-/. I don't see why that wouldn't work.
-- SirG3
I'm not unconditionally opposed to exposing the underlying protocol -- it's just something that I think requires a fair amount of thought before doing. The problem you run into with certain users is that by giving them information, they immediately seek a way to exploit it. And even when you tell them not to do something, they go ahead and do it, and we're stuck supporting whatever weird mechanism they came up with to misuse the product until the end of time. I'd like to avoid that if possible. ;-) But again, I'm not strictly against exposing the protocol. I can see it as being a good thing because it would allow a user to write a plugin that interacts with a REALbasic application using the easy networking classes without forcing them to reverse engineer the protocol. So in some regards it helps users to write smart code....
Well, you could always post the protocol pseudonymously as a "reverse engineering job". Then you could respond as yourself and say "yep, that looks like it, but don't count on it not changing". Then you could switch back to your alter ego and dis the protocol, starting a minor flame war. When you're finally bored, you can then demand that it be taken off-list.
LoL, nice Charles. :-P Don't tempt me.
Late on the scene, but...
Anyone who is clever enough to find a problem with the protocol is probably smart enough to reverse-engineer the thing too. Might as well post it somewhere publicly in an RFC-like fashion and make our lives easier. You seem to have many conflicts with yourself over whether or not to give low-level access to the user.
A work-around to your problem: Add an "OldProtocolError" event to the new class which can be used to send one last packet and close the connection. For instance, for a chat room, the event could send a message to the client saying there is a new version available.
Honestly, the developer gets what he paid for (in the proverbial sense; if I'm going to pay $100 for a socket it better damn well work). If they can't come up with their own protocol, they rely on the fact that your protocol works. I guess they put all their eggs in one basket, eh? They will recuperate. Not to mention, no commercial entity would release shareware with an "EasyTCPSocket"...
@Ryan -- Better late than never! I'm not certain I agree with your assumption that people clever enough to find issues are smart enough to reverse engineer. But I am leaning towards agreeing with making the protocol public. The more I think about it, the less I see wrong with the idea.