Writing a stream protocol: Message size field or Message delimiter?
delimiter, message, protocols, stream, terminator
Solution
I agree with sigjuice. If you have a size field, it's not necessary to add and end-of-message delimiter -- however, it's a good idea. Having both makes things much more robust and easier to debug.
Consider using the standard netstring format, which includes both a size field and also a end-of-string character. Because it has a size field, it's OK for the end-of-string character to be used inside the message.
Problem
I am about to write a message protocol going over a TCP stream. The receiver needs to know where the message boundaries are. I can either send 1) fixed length messages, 2) size fields so the receiver knows how big the message is, or 3) a unique message terminator (I guess this can't be used anywhere else in the message). I won't use #1 for efficiency reasons. I like #2 but is it possible for the stream to get out of sync? I don't like idea #3 because it means receiver can't know the size of the message ahead of time and also requires that the terminator doesn't appear elsewhere in the message. With #2, if it's possible to get out of sync, can I add a terminator or am I guaranteed to never get out of sync as long as the sender program is correct in what it sends? Is it necessary to do #2 AND #3? Please let me know. Thanks, jbu