[Standards] Binary data over XMPP

Dave Cridland dave at cridland.net
Mon Nov 12 10:19:39 UTC 2007

On Sat Nov 10 02:07:08 2007, Justin Karneges wrote:
> On Friday 09 November 2007 3:35 pm, Dave Cridland wrote:
> > ubiquitous encryption
> Best laugh of the day!
Oh, I'm not laughing.

> Other protocols have been fighting this battle for years.  Is XMPP  
> so much different?  I can see the headlines: "XMPP finally gets  
> everyone in the world to use encryption.  Email working group  
> wasted their lives."

To understand why those efforts failed, it's worth looking at what's  
changed over the years.

When Internet Mail started, it was purely an interoperability  
facility between heterogeneous systems - as were pretty well all  
protocols back then. You can see this in the way that an email  
address is specified - there's no specification at all for the  
local-part - it can contain pretty much anything at all, it may or  
may not be case-sensitive, etc.

As I said, most protocols of the time were similar. FTP exposes the  
host's filesystem semantics, so using FTP requires that you know the  
remote host's filesystem layout. IMAP, similarly, exposes the host's  
mailbox layout and hierarchy - giving endless fun for client  
developers who usually expect all IMAP servers to look the same.

So providing any end-to-end service over email is tricky, because the  
majority of email servers - still - are not "Internet" mailservers,  
but LAN mail systems that have a gateway. (Exchange is, now, finally  
dealing with Internet Mail internally, but until very recently it was  
X.400 internally, and was much happier talking X.400 P1 rather than  
ESMTP). Hence most ESMTP extensions assume that somewhere, the  
Internet Mail system stops, and gets gatewayed into something local.

A sea-change (or paradigm shift, if you like playing buzzword bingo)  
in protocol design happened around the early 90's, when protocol  
designers shifted from exposing local semantics into providing a  
homogenous model. XMPP is a late protocol, by this metric, as is  
HTTP. Many protocols have shifted toward this style, too - FTP now  
has TVFS, IMAP servers increasingly provide a fairly homogeneous  
layout, etc. This makes deploying end to end services significantly  

The other factor is that email isn't a close-knit community. At the  
SDO level it is - the majority of email standards developers know  
each other to some degree. However the vast majority of client - and  
even server - developers don't participate. This contrasts heavily  
with XMPP, where the vast majority of client and server developers  
are active on this list.

Finally, we're a much younger protocol. Email is thoroughly ancient,  
and encryption is a comparitively new issue, and even there, multiple  
paths have been explored, and problems discovered. We've got the  
benefit of hindsight here - we know which bits have proven difficult  
to deploy, and which bits have proven easy. We know what end-users  
actually want, as well. All of this knowledge has effectively come  
from email.

I strongly suspect that we're in a much better position to achieve  
ubiquitous (or near ubiquitous) encryption than email ever was, and I  
certainly don't think that it's worth giving up before we've started.

Dave Cridland - mailto:dave at cridland.net - xmpp:dwd at jabber.org
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

More information about the Standards mailing list