[Standards] s2s and gracelessly broken streams

[Dave Cridland]

On Wed Apr  4 15:09:12 2007, Matthias Wimmer wrote:
> If you are using libpth for example, you won't be able to have more 
> than
> 1024 file descriptors. So considering the raw limits of the 
> operating
> systems is not enough, you have to consider the libraries running 
> on top
> as well.
> 
> 
If so, that's a compelling argument for not using libpth anywhere 
near your server, of course. :-)

> But I won't really start counting connections on different systems.
> Doesn't matter how many are possible, it is still a requirement for 
> good
> protocol design, to allow implementations to be conservative in 
> resource
> usage. When even some servers start to hold connections open to 
> other
> servers and these connected servers have now change to get rid of
> connections if they have to get free resources again, you won't be 
> able
> anymore to implement servers on resource constrain systems.
> 
> 
It depends on which resources are constrained, though.

I'm not saying "don't close s2s connections". This is obviously going 
to be needed eventually - otherwise you'd end up maintaining a 
connection to every server in existence, which I'd heartily agree was 
a dumb idea.

I'm very much saying "don't believe that the number of IP addresses 
affects the number of TCP connections possible in any way that is 
remotely relevant to this discussion", but I think I'm done saying 
that.

I am saying "don't arbitrarily decide that closing s2s connections is 
a good way of conserving resources" - because that is not always the 
case, it's purely a trade-off. Establishing a connection, including 
TLS setup and so on, has a significant cost on most platforms.

I would also say "don't impose a requirement to send non-information 
bearing octets around purely to maintain a connection", too. 
Basically, if you say "s2s connections MAY be timed out when idle, 
and such a timeout MUST NOT be lower than 30 minutes", then this gets 
translated into implementations as "servers MUST send their peers 
some useless stanza every 25 minutes", which - supposing we take 
Tony's figures - means that he might be getting or sending 3.3 of 
these every second on average.


> (Hey where are the people in this thread, that even consider TLS
> compression too resource expensive on their s2s links?)

Maybe they've realized that compressing and/or encrypting an idle 
link doesn't have a major impact on CPU cost, since the levels of 
data involved are, erm, low.

Of course, the two LZ back-buffers *are* a significant memory cost, 
at something like 32k each. Assuming around 5,000 of these, that's 
around 300M. There's no easy way around that either. I'd assume the 
memory-constrained servers would use a drastically smaller 
back-buffer size, though.

Dave.
-- 
Dave Cridland - mailto:dave at cridland.net - xmpp:dwd at jabber.org
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade