[standards-jig] JNG Ramblings.

David Waite mass at akuma.org
Mon Aug 12 06:43:48 UTC 2002

Nathaniel Borenstein wrote:

>The only tricky bit is that when using a multipart, you have to be
>careful that your boundary doesn't appear in the body of any of the
>objects being encapsulated.  (This is easier for base64-encoded data,
>because you can include in your boundary a character that isn't in the
>base64 alphabet.)  But you could even eliminate that problem by
>modifying MIME to add a "Content-Size" header.  That was much-discussed
>during the MIME design, but it ultimately was rejected as being
>incoherent for non-binary data over SMTP.  It should be fine for true
>binary data in a binary protocol, as long as nobody is quietly
>converting the data (say, between differing newline conventions) along
>the way.
I was thinking about this earlier. The argument for telnet operation 
becomes rather pointless if you allow binary data to be framed and 
transferred, but don't require feature negotiation first.  Text framing 
does not help for telnet operation if content length is required.

One way to eliminate the problem of boundary characters appearing within 
a text or binary chunk is to allow a boundary to be ignored, something like:

--simple boundary

Content-Type: multipart/continuation

<data continues>

I think everyone would agree that we would like a single method for 
framing though, rather than having to handle multiple different framing 
mechanisms (such as HTTP 1.1, which has five).

>>Besides, I think MIME is really heinous.
>If you think that MIME requires C-T-E's and 7-bit transport, I can
>certainly understand why you'd view it that way.  But if you eliminate
>that problem, I think it is rather less heinous in this case than having
>a maximum of 128 choices for the "Type" field and therefore an inherent
>need for a content-type mechanism underneath it.
This is the primary problem with binary protocols; they must be designed 
with any required future extensibility in mind.

An example:

If the four-orders-of-magnitude increase in both storage medium and in 
network bandwidth we have seen over the last decade continues into the 
future, a 32-bit 'size' field will become insufficient. If that happens, 
how would we upgrade a protocol like the one Mike Lin proposed @ 
http://mikelin.mit.edu/xmpp/jng/ ? In a similar vein, look at the 
evolution of Microsoft's (V)FAT filesystem.

Another example:

The ICQ and Yahoo IM protocols are both binary protocols. Both have had 
changes to their protocols over the years due to the normal course of 
development. In these cases, new variations on protocol were developed. 
But since the extensibility was only possible through non-compatible 
changes in things like presence and message format, the options became:

1) translate all messages on the server to and from the new protocol 
through the transition period
2) ship in all future versions of the client the capability to receive 
and author all variations on the binary protocol.
3) A flag day, where the server switches over and all clients (and all 
servers which communicate with that server if its a distributed system) 
break until being upgraded (such as what AOL finally did with ICQ at 
protocol version 7)

This is my qualm with binary protocols; they are almost always 'packed', 
allowing only for binary-incompatible changes, and usually requiring 
close administration to prevent conflicts of the few extensibility 
mechanisms they have. The OSCAR SNAC packet, for example, uses 32-bit 
fields for protocol family and subtypes, which would require 
administration and registration services to prevent conflicts (just as 
the IANA does for the 16-bit space for port numbers).

Even if they had doubled the size and went with GUIDs to represent 
packet type,  {F7FC395C-3045-4368-9A02-1F7CA17043AE} is a lot less 
readable than "text/plain" or "application/x-realaudio".

>On the larger issue of whether a protocol such as Jabber's should be
>binary or textual, you might be surprised to hear the author of MIME
>coming down on the side of a binary protocol, but that's exactly what
>I'm doing, primarily for the reasons given by Richard Dobson in his
>message.  We did not design MIME as a textual protocol for ease of
>developing/debugging.  We designed it that way because of the
>constraints of real-world SMTP implementations that simply couldn't
>handle 8-bit or binary data.  I don't *think* Jabber should have to deal
>with similar constraints, although this may be an area where my
>newbie-hood is showing.  And while I sympathize with the folks who want
>to telnet directly, I don't think it would be very hard to write a
>telnet gateway that makes it easy to type (almost) directly to the
>server from a textual interface, and that's a small price to pay for a
>33% speedup on big binary objects.  (Alternately, we could use an
>approach like ESMTP which makes binary transport optional, and this
>could be nearly always used by software but rarely or never by people
>connecting with telnet.)
I am a bit unclear: are you stating a case for binary framing, a full 
binary protocol, or just for the capability of being able to send and 
receive 8-bit clean data?

-David Waite

More information about the Standards mailing list