[Standards-JIG] The Ack Hack.
dave at cridland.net
Wed May 3 22:43:30 UTC 2006
After a lengthy, and very interesting, discussion with Pavel
Šimerda, we refined my initial idea, and came up with the following.
Note that I'm not saying whether Pavel thinks this is better than the
JEP-ACK proposal, but he understood this a lot clearer.
1) The Sequence Number: Stanzas MAY have an attribute attached to
their top-level element indicating their "sequence number". This has
a strictly increasing interger value, meaning that it's higher than
any previously sent stanza in the same connection.
Rationale: This is expected to be on all stanzas or none. It's used
to acknowledge receipt of stanzas, both implicitly and explicitly.
Wire cost: Around 20 octets.
2) Implicit Acks: Stanzas MAY have an attribute attached to their
top-level element indicating the highest sequence number seen by
stanzas the transmitting entity has received.
Rationale: Whenever you're sending a stanza, you can ACK all the
stanzas you've received if you need to.
Wire Cost: Again, 20 octets.
3) Explicit Acks: A Sender MAY require the Receiver to transmit the
highest sequence number it has received at any time. (This can be
implemented as a Ping, or NOOP, because the response indicates that
all stanzas have been received to this point).
Rationale: When the connection's quiet, this can both test the
connection and eliminate a build-up of potentially unreceived stanzas.
Note: Unlike JEP-ACK's ping, the receiving entity processes pings
after accepting prior stanzas.
4) Reconnection: A Receiver MAY store the highest sequence number it
has received when the connection fails, and present it to the Sender
upon reconnection. (Server in a stream feature, and Client in an iq?
That would eliminate a round-trip.)
5) Hop only: The sequence number attribute and any Implicit Ack
attribute MUST be removed prior to further processing - they are
semantically purely a stream feature, and not part of the stanza per
So that's the protocol description. Namespace URIs, attribute names,
etc all up for grabs.
Now how to use it:
First, note that this is not symmetric - a client may send sequence
numbered stanzas without offering the same service to a server -
assuming the server offers the feature anyway.
As a receiver:
a) When sending a stanza, you need to add a sequence number
attribute, and store the stanza in an "unacknowledged" collection.
b) Every time you receive any stanza, check to see if it has the
Implicit Ack attribute. If it does, you can delete all stanzas in the
"unacknowledged" collection with a sequence number equal to or lower
than the one you've received.
c) If the connection goes suspiciously quiet for long enough, or the
number of stanzas (or size of them) is large, you can ping the
connection, thus causing an Explicit Ack. If you receive a return,
then that's an incoming packet, so see (b).
d) If you receive no response to a ping, or you get an explicit TCP
fatal error, then reconnect. On reconnection, obtain the highest
sequence value for the now defunct session. Treat this value as an
Ack value, then retransmit all stanzas in your "unacknowledged"
collection. Note these may have to use new sequence numbers.
For the receiver, you have to store one sequence value per active
connection, the highest received sequence value.
The receiver also should store persistently the highest received
sequence store for each resource, for a minimum of around an hour
after the connection is lost. I'll call this the last highest
a) Whenever you receive a stanza, note the sequence value, and store
it as the highest received sequence value.
b) Whenever you send a stanza, see if you have acknowledged the
current highest received sequence value. If not, add the Implicit Ack
attribute. You can always add the Implicit Ack attribute, but you're
c) When a new connection is made, transmit the last highest sequence
Pros: Many fewer packets, reliable connections are less costly,
unreliable connections are made much more reliable.
- Bandwidth is increased, although by 40 octets or so worst case
(usually much less) - a TCP packet empty of data is around 40 octets,
- Complexity is increased, however this is only an issue during
reconnection, not during more normal operation.
- Servers need to rewrite stanzas when forwarding. On the plus side,
at least it's only the outer element.
With the Implicit Acks, we've put a lot more acking back in. This is
now, roughly, JEP-ACK, but with Acks made more lazy, compensated for
by having connection failure itself made non-fatal.
You see things; and you say "Why?"
But I dream things that never were; and I say "Why not?"
- George Bernard Shaw
More information about the Standards