[standards-jig] Pub/Sub for JNG?

Dave dave at dave.tj
Wed May 1 04:11:29 UTC 2002

Reply inline:

 - Dave

Dave Smith wrote:
> On 4/30/02 9:08 PM, "Dave" <dave at dave.tj> wrote:
> > How about allowing multiple "connection types?"  One type can be standard;
> > another can be SSL; yet another can be gzipped.  In the last "connection
> > type," every message would be piped through a gzip-compatible deflation
> > before being pumped out to the network.  The gzip format (like all
> > LZW algorithms) can tell where the end of the file is supposed to be,
> > so there's no need to send meta information to alert the receiver to
> > the point where one message ends and the next begins.  I suspect this
> > "connection type" would be most useful for s2s connections between
> > servers exchanging reasonably heavy traffic.
> Connection types aren't that bad, but show me some stats that gzipping will
> actually do anything to improve "speed" or "efficiency". Empirical data is
> of essence.
Text is very highly compressible.  Any server that has its network
bandwidth as a bigger bottleneck than its own processing power (which
is rather cheap to increase) is going to benefit from the awesome
plain-text compression abilities of gzip.  I really don't have the energy
to produce a scientifically correct experiment, but try transferring
a bunch of files across a small pipe without compression, and you'll
probably be able to convince yourself easily that compression is key,
because bandwidth isn't _that_ cheap.

> > Another interesting "connection type" is aimed squarely at the average
>  ..blah..blah..blah.. (see "Greg the Bunny")
> > and you'll see a dramatic improvement in your IM response time.
> *Dizzy hands Dave a nice asbestos suit*
LOL. . .

> Are you under the impression that TCP packets actually move slower across
> ethernet than UDP packets? Are they heavier, or harder to route, or
> something? I bet that the little guys in the network don't like having to
> carry  those heavy TCP packets all over the place.
/me starts to fall off his chair. . .

> You know, to be truthful, UDP is quite a bit faster. In fact, it can achieve
> zero-time delivery -- more often than not it opts not to deliver packets. I
> know it's crazy, but we're really not all that interested in such a fast
> transport layer. We like the slow, but reliable pace of TCP -- kinda like a
> mule. 
Whatever happend to the first letter of IM???  Clients that are going to
be sending lots of tiny messages can improve their latency (and the load
on their favorite Jabber server - UDP is cheaper for an OS to handle
than TCP) by using a UDP connection, while mules can continue to use
their reliable TCP connections.  That harmony makes the world a rather
happy place :-)

> Jabber absolutely needs TCP -- even for so-called "lightweight" IM.  If you
> would look at all the heartache that the IETF IMWG has been through, one of
> their fundamental discoveries is that congestion control (something TCP
> provides happily) is absolutely necessary for any sort of scalable IM
> system.
Congestion control isn't exactly at the top of an average guy's
priority list.  I'd be somewhat suspicious of anybody who decided to
code a UDP-based s2s module, but I'd certainly vote to allow clients to
use UDP c2s if they prefer.  The packet loss statistics on most of the
modern Internet are negligible, and the TCP control connection (which
simply gets a ping packet sent every few minutes to keep it alive, most
of the time) can be used to request retransmission of any packet (since
each packet can be numbered - you get two packets with the same number,
toss the second one out; you get a packet with an out-of-sequence number,
request a retransmission).  I don't think I'd mind not finding out about a
lost message until the next message arrives (which probably won't be more
than a few seconds later); if it really becomes a problem for some people,
they can ask every minute over the TCP connection whether X is the number
of the last message sent, or they can ask the server to send an alert
over the TCP connection when the next message is sent.  There are plenty
of ways to make UDP as reliable as TCP without losing most of the speed
advantage UDP enjoys, while trying to deal with the occasional lost packet
in a reasonably time-sensitive manner.  UDP really is a good thing :-)

> Other than that, I think you're completely missing the point of Jabber.
> It's not just about IM. We don't want to sink significant amounts of effort
> into optimizing for IM traffic.
I'm trying to sink some effort into being able to optimize particular
applications for particular purposes.  A Jabber client that's actually
a bot fetching large XML documents from another Jabber agent (a Jabber
gateway to a database, if you will) will certainly want to find some way
of compressing all the data coming in, especially if the poor client
is on a lousy dial-up connection.  If that client can tell its Jabber
server when it connects that it wants all packets in that c2s connection
compressed, it'll be able to cut the time required to get the data in
2-5, or even better, if the data is very repititious (as it tends to
be, with XML).  In fact, it'd be neat if it could request queueing of
packets, so they're only sent every 4K or whatever - that'd allow the
server to optimize the TCP packets to be as large as possible, improving
the throughput.  "Optimizing IM" is only one facet of my proposal: my
proposal is all about choices - the ability to optimize your particular
environment to whatever task you happen to be using Jabber for, ATM.
That, if you ask me, is the secret to scalability for Jabber under
the load of many different types of applications running concurrently,
because it enables each application to make optimal use of the network.

> Morale of the story: UDP == blah, blah, blah.
> Diz
> _______________________________________________
> Standards-JIG mailing list
> Standards-JIG at jabber.org
> http://mailman.jabber.org/listinfo/standards-jig

More information about the Standards mailing list