[Standards-JIG] Re: What happened to the ACK proposal?

David Chisnall theraven at sucs.org
Tue Aug 16 15:31:03 UTC 2005

On 16 Aug 2005, at 16:13, Ian Paterson wrote:
> The server should not tell the sender the message was delivered  
> until it
> receives the TCP ACK. If it doesn't receive the ACK within a  
> configured
> time then it should close the stream and store the message for later
> delivery (assuming it can't be delivered to another resource).

Unfortunately, the TCP ACK is not received by the server.  Unless you  
want to:
a) Run as root, and
b) write your own TCP stack on top of raw sockets
then you are not going to get this feedback.

> This is a problem with the implementation of TCP. The application  
> should
> be told as soon as the previous ACK fails to appear within the
> configured time.

True, but not helpful.  The standard API for TCP (and other network  
protocols) is Berkley sockets.  Even Windows uses a superset of this  
API.  This API does not provide any mechanism for

> XMPP devolves to the transport layer responsibility for a) confirming
> delivery over a hop, and b) detecting broken connections. XMPP isn't
> supposed to know or care which transport layer is being used.  
> Should RFC
> 3920 be changed just because implementations of TCP are bad?

Neither is SMTP, and yet SMTP does not consider a message delivered  
until it has received an acknowledgement from the server that it has  

> The fact that TCP ACKs every single packet sent should be  
> sufficient to
> provide a) and enable b) - via whitespace pings. (These pings are
> optional with TCP since most protocols that run over it don't need
> them.)

Actually, most protocols that are inactive for a long time do use  
them - it's just that most protocols don't remain inactive for a long  

> Perhaps we need an informative JEP that describes exactly how XMPP may
> be transported over the TCP *protocol* in a perfectly reliable way?  
> The
> JEP would document the way applications currently extend the TCP
> transport layer with pings.

Not even that.  A JEP for ensuring delivery over any arbitrary  
protocol which does not provide a mechanism for detecting dropped  
connections (of which TCP is an implementation).

> One section of the JEP could discuss the problems with various  
> existing
> TCP implementations, thus providing a basis for our efforts to get TCP
> implementations fixed. We could then have this discussion somewhere
> else, to encourage the people responsible for the bad  
> implementations to
> fix them. After all, XMPP applications aren't the only ones that would
> benefit.

Not going to happen - at least not in the next decade.  The Berkley  
socket API is broken, but it's been used since before the Internet  
existed.  People are not going to change suddenly.  Better APIs exist  
for some specific stacks or hardware, but the Berkley API is, and  
will remain for some time. the lowest common denominator.

> IMHO the implementors will listen. [Until recently many OSes,  
> including
> Linux, could not maintain tens of thousands of inactive connections.
> Linux 2.6 introduced new APIs to fix the problem.] We might end up  
> with
> a new connection class for Java.

Maintaining tens of thousands of connections is something that can be  
done without changing the API.  Detecting when they go active was  
done with poll/select, and is now done on *BSD with the more scalable  
KQueue (and on Linux with some ugly hacks).

> However, if we don't get any joy from the TCP implementors, then  
> perhaps
> the TCP transport JEP could also define an optional extension to the
> TCP-transport layer which works around the problems with bad TCP
> implementations. This would work above the TCP layer but *underneath*
> the XMPP layer (i.e. at exactly the same level as whitespace pings and
> JEP-0124 HTTP Binding do today). This protocol might be similar to the
> one Justin proposed, but it would be negotiated *before* the  
> stream. It
> could conceivably (but not necessarily) be transported in the  
> character
> data between stanzas (i.e. without XML).

Sounds very, very messy.  Take a look at SMTP.  They solved this  
problem 20 years ago.  It's not difficult.

Note that moving this down a layer causes the problem to remain the  
situation where the data is read into a buffer but the application  
crashes / is killed before it has a chance to process it.

More information about the Standards mailing list