[Standards] All your problems, solved ;)

Jonathan Chayce Dickinson chayce.za at gmail.com
Mon Aug 27 22:24:01 UTC 2007


Oh, one thing I forgot to put to the XEP. The even if it is small on the 
(compressed) wire, Base64 is large in memory. Keeping it binary makes 
more sense, it offloads from the XML parser and your app spends 33% less 
time in loops, which adds up to quite a lot with 10 000 concurrent 
clients. That's the last of it.

Once again Dave, commendable response: I applaud you.

Cheers

(there are two small inserts)

Dave Cridland wrote:
> On Mon Aug 27 20:55:05 2007, Jonathan Chayce Dickinson wrote:
>> I was perusing over a couple of MSDN articles, when I came across DIME 
>> (Direct Internet Message Encapsulation). Jeez, I thought, that would 
>> fix all the file issues in Jabber (like SI). So I wrote a draft, any 
>> takers? What do you all think? Didn't want to push this forward until 
>> I got some feedback.
> 
> DIME seems to have been orphaned by its authors, in favour of 
> soap12-mtom, so I'm not convinced it'd be a good plan to go this route 
> whatever the merits of the general concept.
> 
> However, as a general note, encapsulating small amounts of "binary" data 
> in base64 is generally okay - you'll recover the majority of the 33% 
> bloat through compression (which you hopefully have via TLS, and may 
> have via other means).
> 
> This isn't suitable for large amounts of data, though, since compression 
> doesn't fully recover the overhead.
> 
> Moreover, compression algorithms will become "poisoned" through trying 
> to compress different types of data - typically via skewing the Huffman 
> encoder and destroying the backreference "dictionary". This is 
> especially true if the data being transferred is already compressed, in 
> this case you can observe surprisingly high increases in data transfer 
> cost (ie, negative compression).
> 
> But sending bulk data through XMPP is a bad idea for a number of other 
> reasons, not least of which it's trashing the server, and clogging your 
> XMPP channel (thus reducing its response time).

Look at the interleaving part, it rectifies sending large amounts of 
data over XMPP (and clogging, or locking, the channel): all it makes is 
an interesting argument.

> 
> A better method is to negotiate a peer-to-peer session suited to the 
> kind of transfer you're doing, although I admit that's not exactly a 
> ground-breaking suggestion. :-)

But constructive none-the-less. Maybe I will open another port for 
binary data on my server/farm or make a dime-like protocol. Thanks for 
the input.

> 
> Dave.

-- 
jonathan chayce dickinson
ruby/c# developer

email:  chayce.za at gmail.com
jabber: moitoi at inflecto.org

<some profound piece of wisdom>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 6974 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.jabber.org/pipermail/standards/attachments/20070828/d901897c/attachment.bin>


More information about the Standards mailing list