[standards-jig] JNG Ramblings.

Matthias Wimmer m at tthias.net
Fri Aug 9 17:23:25 UTC 2002

Hi Mike!

Mike Lin wrote:

>>- It's much easier to debug a non-binary protocol. Not just because you 
>>can telnet, it's also that that you can easily tcpdump the network 
>>traffic and see what has been exchanged. If there has been send wrong 
>>data you see it at the first look - wrong bits, wrong packet length 
>>numbers and all that things are much harder so recognize in a dump.
>The protocol I laid out is perfectly readable in ethereal. There are a
>few extra bytes spread around that are meant to help the machine, and
>everything else is XML.
For me ethereal is often a much to big program. Most of the time I just 
capture the traffic with tcpdump and pipe it through the strings command.

>You only have to get framing right once, and you'll know something is
>wrong if you don't.
But you have to implement it first. On Wednesday I visited a friend that 
is a Flash programmer. He wanted me to show him how to use Jabber 
because I wan't to use Jabber for some projects. That was cool, we had 
written a Flash Jabber client with presences, roster and all what you 
have in a basic client within less then one your. We could just use the 
XMLSocket class that is a standard part of Flash. - If we would had to 
implement the framing first, it would have been impossible to get a 
working client that fast.

What is the Flash support in the Jabber server for? We havn't needed 
something special.

>>- A binary protocol has less redundancy, this means it's less tollerant 
>>on errors and it's less robust.
>If you emit bad XML in Jabber, your stream is over, and the server
>disconnects you. With a framing protocol, the possibility exists to
>recover from that error; so I would argue it is more robust at the
>transport level.
This is what the actual server does ... but it's not a protocol requirement.

>Binary framing simply leaves less room to screw up at the framing level,
>which is a good thing, because you don't want to screw up - at all - at
>the framing level.
Why do you think that it leaves less room to screw up?

>>- "Packet sizes" in binary protocols are always limited as size fields 
>>have limited size, this limits the extensibility.
>So does your CPU. You're going to load that number into a CPU register
>at some point no matter how it's represented on the wire, so it is a
>good thing from every perspective to have bounded packet sizes. It's
>easier on everyone.
No, that's not true. While I can't change a protocol that is used ... 
CPUs are exchanged all the time, register size and performance increase 
all the time. Also I can program a CPU to use numbers at a precision 
only limited by the memory I have.
But the point is: I don't care if the CPU registers are limited because 
I don't transmit the size of a packet. The packet is just limited by an 
"end of packet" marker (a closing XML tag in the case of Jabber).

>>- In non-binary protocols there are more ways to extend the protocol 
>>when new features are added.
>The entire protocol isn't binary, only the message framing is.
Yes, but what you call the "wire protocol" is binary. And I can imagine 
that this layer of the protocol could be extended at one time in the 
future too. Look at the payload identifier ... 7 bit ... 128 possible 
values. That isn't much. My mime.types file has already more entries 
today. I remember the fidonet (-> http://www.fidonet.org/ - something 
like the original usenet) and its (binary) protocols. Many fields in 
these protocols are designed as 8 bit and everybody thought that this 
will be enough for all time ... but they got to this limit and had to do 
bad tricks to ship around the problems. (Also think of the 640 KB, 1MB, 
4MB, 512 MB, 3 GB, 4 GB boundaries in the memory architecture of PCs ... 
when they were introducted everybody thought that nobody will every need 
that much memory. - And the black magic like the A20 gate you use to 
solve these problems.)

>Everything that needs to be extensible is done in XML. I'm using binary
>framing to achieve a specific, measured goal of greatly improving the
>ease and efficiency with which a PDU framer can be implemented. I never
>intend to use it for anything more than that, because we have XML
>everywhere else.
The efficiency is the only point where I think a binary protocol has an 
advantage. But I think to care that much about the computing power you 
need to run the protocol is "thinking of the past". - Nearly nobody 
still writes programms in assembler code just because they are faster. 
It's much more important be fast in writing your programs and to get 
them (as) error free (as possible).
Also I don't think that parsing XML is the most CPU consuming part of a 
Jabber server. I expect that there is much more computing power needed 
just to pass data between the different functions, routing information 
and all that stuff.

>>- When programmers deal with binary protocols they tend to use fixed 
>>size buffers ... this causes buffer overflows. My impression is that 
>>binary protocols are more often vulnerable.
>Buffer overflows are a result of bad programmers and bad programming
>languages, not binary protocols. Static buffers are a boon for
>performance. Not checking the bound, when your language doesn't do it
>for you, is just incompetence.
Sure ... programmers using static buffers should be brain washed ... but 
they exist ... if they wouldn't, we would have much less security holes 
in existing software. - But the question is what supports them more to 
do this ... binary or text.

>>- Programmers have to deal with endianness, this can be a really hard 
>>work in some languages especially if you want to write portable code.
>If it doesn't have ntohl, your language needs AND, OR, SHL, and SHR.
>Show me a general-purpose programming language that doesn't.
Sure this can be done. But programmers not used to write protable code 
often don't care about that. And most programmers only program on one 

Personally I prefere to use DIV/"/" and MOD/"%" if I don't have a 
function like ntohl/ntohs.

>>- The protocols that have success are the simple protocols: SMTP, NNTP, 
>>SNMP, HTTP, POP3, IMAP, ... all of them are protocols you could even 
>>implement without much hacking and reading the specs.
>Any protocol that tries to squeeze performance is binary. I'm squeezing
>performance -- at the framing level _only_.
See above ... I don't think that you get much more performance with this 
because I don't think that XML parsing is the bottle neck of a Jabber 

>>- If we want to support virtual channels we don't have to invent a new 
>>protocol for that: There is already BEEP and others that can be used for 
>Well, first, my opinion is that there's no good reason for BEEP not to
>be a binary protocol. It imposes 32-bit limits on its sequence numbers
>anyway. Having to parse text integers and not having fixed-sized headers
>just makes things slower, so I really don't understand why it's not
I don't think that we need BEEP ... but I can't see why we should invent 
a protocol for that same thing. - I also don't want to transfer files 
"in band". Not because of performance but because of network traffic.

And it's not binary because the protocol uses text to transfer the 
protocol information.

Tot kijk

Fon: +49-700 77007770		http://matthias-wimmer.de/
Fax: +49-89 312 88654		jabber://mawis@charente.de

More information about the Standards mailing list