[Standards-JIG] The Ack Hack.

[Dave Cridland]

After a lengthy, and very interesting, discussion with Pavel 
Šimerda, we refined my initial idea, and came up with the following. 
Note that I'm not saying whether Pavel thinks this is better than the 
JEP-ACK proposal, but he understood this a lot clearer.

1) The Sequence Number: Stanzas MAY have an attribute attached to 
their top-level element indicating their "sequence number". This has 
a strictly increasing interger value, meaning that it's higher than 
any previously sent stanza in the same connection.

	Rationale: This is expected to be on all stanzas or none. It's used 
to acknowledge receipt of stanzas, both implicitly and explicitly.
	
	Wire cost: Around 20 octets.

2) Implicit Acks: Stanzas MAY have an attribute attached to their 
top-level element indicating the highest sequence number seen by 
stanzas the transmitting entity has received.

	Rationale: Whenever you're sending a stanza, you can ACK all the 
stanzas you've received if you need to.

	Wire Cost: Again, 20 octets.

3) Explicit Acks: A Sender MAY require the Receiver to transmit the 
highest sequence number it has received at any time. (This can be 
implemented as a Ping, or NOOP, because the response indicates that 
all stanzas have been received to this point).
	
	Rationale: When the connection's quiet, this can both test the 
connection and eliminate a build-up of potentially unreceived stanzas.
	Note: Unlike JEP-ACK's ping, the receiving entity processes pings 
after accepting prior stanzas.

4) Reconnection: A Receiver MAY store the highest sequence number it 
has received when the connection fails, and present it to the Sender 
upon reconnection. (Server in a stream feature, and Client in an iq? 
That would eliminate a round-trip.)

5) Hop only: The sequence number attribute and any Implicit Ack 
attribute MUST be removed prior to further processing - they are 
semantically purely a stream feature, and not part of the stanza per 
se.

So that's the protocol description. Namespace URIs, attribute names, 
etc all up for grabs.

Now how to use it:

First, note that this is not symmetric - a client may send sequence 
numbered stanzas without offering the same service to a server - 
assuming the server offers the feature anyway.

As a receiver:

a) When sending a stanza, you need to add a sequence number 
attribute, and store the stanza in an "unacknowledged" collection.

b) Every time you receive any stanza, check to see if it has the 
Implicit Ack attribute. If it does, you can delete all stanzas in the 
"unacknowledged" collection with a sequence number equal to or lower 
than the one you've received.

c) If the connection goes suspiciously quiet for long enough, or the 
number of stanzas (or size of them) is large, you can ping the 
connection, thus causing an Explicit Ack. If you receive a return, 
then that's an incoming packet, so see (b).

d) If you receive no response to a ping, or you get an explicit TCP 
fatal error, then reconnect. On reconnection, obtain the highest 
sequence value for the now defunct session. Treat this value as an 
Ack value, then retransmit all stanzas in your "unacknowledged" 
collection. Note these may have to use new sequence numbers.

Receiver side:

For the receiver, you have to store one sequence value per active 
connection, the highest received sequence value.

The receiver also should store persistently the highest received 
sequence store for each resource, for a minimum of around an hour 
after the connection is lost. I'll call this the last highest 
sequence value.

a) Whenever you receive a stanza, note the sequence value, and store 
it as the highest received sequence value.

b) Whenever you send a stanza, see if you have acknowledged the 
current highest received sequence value. If not, add the Implicit Ack 
attribute. You can always add the Implicit Ack attribute, but you're 
wasting data.

c) When a new connection is made, transmit the last highest sequence 
value.

Pros: Many fewer packets, reliable connections are less costly, 
unreliable connections are made much more reliable.

Cons:
- Bandwidth is increased, although by 40 octets or so worst case 
(usually much less) - a TCP packet empty of data is around 40 octets, 
for comparison.
- Complexity is increased, however this is only an issue during 
reconnection, not during more normal operation.
- Servers need to rewrite stanzas when forwarding. On the plus side, 
at least it's only the outer element.

With the Implicit Acks, we've put a lot more acking back in. This is 
now, roughly, JEP-ACK, but with Acks made more lazy, compensated for 
by having connection failure itself made non-fatal.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw