[Standards-JIG] rfc3920bis: Stanza Acknowledgements

[Bruce Fitzsimons]

Hi Justin,

I've heavily snipped, if I didn't respond then assume I've accepted your 
correction :-)

Justin Karneges wrote:
>> The way SCTP copes with this is to ignore the duplicates <SNIP>
> How do you ignore duplicates?
>   
You can ignore duplicates if they have id's by the client knowing which 
id was last received and ignoring "older" ones. Hence the need to 
increase the id in a controlled way. You can't completely avoid 
duplicates any other way (see below) as there is no "watching the 
watcher" model for ack reliability. Is duplicate avoidance a design 
goal, or a nice to have? Should it be done only on the real 'id' tag by 
the recipient?

Having sequence numbers progress in an orderly fashion also helps 
efficient implementation -- if every id is just an opaque tag then you 
have to linearly search your sending queue to find the matching start 
point. The potential for duplicates also creeps in fairly easily if 
people pluck them from undefined places, you end up needing another 
requirement saying "don't duplicate id's within <criteria> time". The 
criteria involves a time envelope including both sender and reciever so 
it is a bit hard to define programmatically.

>   
>> I really can't see the need for "previous" requests.
>>     
>
> The sequencing stuff is based on Dave Cridland's "mobile" scenarios.  He wants 
> to have delivery tracking without acks.  Simply tagging a stanza with a 
> sequence number adds no considerable overhead, but using round-trip acks do.  
> Sequence numbers are also useful for avoiding duplicate transmission, but I 
> don't think that was Dave's primary motivation.
>   

I think we're getting to the point of recognising the difference between 
the lifetimes of a transport session (TCP connection) and the XMPP 
stream. I've been considering that the stream lifetime is tightly bound 
to the transport session, but I can see the other point of view. I just 
don't know which one is intended...

On the (+) side:
1. Transport session "down" (or too slow for the sender) doesn't mean 
you've gone away, the transport can be high-latency or flaky.
2. The sender probably shouldn't be making decisions about what the 
recipient considers "bad".
3. Presence should be recipient controlled rather than server dictated 
if possible.

On the (-) side:
1. If we disassociate the two then how long does anyone wait for the 
XMPP stream to close if they just disappear? How can it be recognised?
2. If the server does intelligent resource switching based on 
availability then it may want to swap to a lower priority resource 
immediately rather than sit and queue for the "better" resource.

Perhaps I'm overthinking this...
>   
>> 1. Um, I see an internal contradiction here. You say: "Acks SHOULD be
>> sent as soon as possible, and MUST NOT be withheld for any condition
>> other than a timeout.". Fair enough. But then your example shows the
>> opposite behaviour to that which you've recommended? I'm confused.
>>     
> <snip>
>
> Do you have a suggestion about how to make this more clear?
>
>   
Ok. I think Dave Cridland has provided an excellent response, although 
I'd like to make the timers shorter. I think we're all agreeing that a 
one-ack for many packet model is the scalable way to go though. We're 
just debating how explicit we need to be about what we're acking, and 
how often that needs to be :-)
>   
>> 2. Un-numbered acks are not a wise thing to include support for. If
>> every message has one on, and you get many acks, where did you get up
>> to? It becomes something like a heartbeat, it doesn't give any absolute
>> message-by-message reliability, only a hand-waving "got to about here
>> and it broke".
>>     
>
> This is over TCP, so we don't need numbers.
>   
Okay. I think I have an answer for why you always need sequence numbers 
if you want reliability (hop by hop). I think your argument is that the 
TCP stream is ordered and so things pulled from the stream will be ack'd 
in that order, so you can count the acks to determine what to resend?

A view supported by your referenced section: "Compliant server 
implementations *MUST* ensure in-order processing of XML stanzas between 
any two entities. " (RFC 3920 s 10).  Hmmm.

I think my only problem with this is around duplicates. Just as there is 
no contract between the XMPP protocol and the TCP stack regarding what 
is submitted versus what is sent, there is no contract in the other 
direction between the TCP stack and XMPP regarding what is received vs 
processed. What is my point? You may get ack's sent but not received, or 
received but not processed; this will cause duplicates on reconnection. 
We return to my earlier question about the design goal regarding duplicates.

< argument about multi-threaded processing deleted after reading s10 
again :-) >

I'm intrigued by your argument that heartbeat/ping is different to 
reliable delivery. This is just so different from other protocols I'm 
familiar with that have them interlinked (including TCP, SCTP, SMPP, EMI 
and internal protocols). Heartbeats are fill-ins for when there is 
insufficient real data to validate the connectivity in these protocols.

It comes back to the discussion of TCP connections vs XMPP streams -- 
are pings checking the heath of the TCP connection while acks 
potentially cross multiple TCP connections without concern?

Regards,
Bruce