[Standards] Proposed XMPP Extension: Quality of Service

Dave Cridland dave at cridland.net
Thu Dec 10 13:10:40 UTC 2015


On 10 December 2015 at 10:10, Peter Waher <peterwaher at hotmail.com> wrote:

> Hello Dave
>
> Thanks for your comments. I've responded to your concerns below:
>
> > My initial reaction to this is that it's not providing anything beyond
> the
> > state of the art, and it implies that <message/> delivery is highly
> > unreliable (and suited to flood messaging), which is rather worrying.
>
> Not really. It's part of a mechanism called "reliable messaging", and is
> available for all sorts of protocols, even HTTP. (MQTT is mentioned in the
> proposal, but it definitely not limited to MQTT, it was just an example.)
> Reliable messaging is available in most protocols supporting queueing,
> where an item can only be delivered once, and exactly once (or, the sender
> receives a notification that it was not possible to deliver).
>
> It is used in cases where retries are not sufficient. This includes all
> non-idempotent actions, such as counting (including bank transactions) or
> accumulation, as opposed to absolute-valued information (such as set
> on/off, or set to N%) which are idempotent.
>
> All asynchronous communication protocols are somewhat unreliable,
> especially if clients have intermittent network connections. While its
> improbable that a client disconnects while a message is being transmitted,
> it can happen. And if many messages are sent, it finally will happen.
>
>

Yes, this is Lamport's Two Generals problem. It is impossible to solve,
however it can be addressed such that it is solved unless one or other end
permanently loses connectivity.

But you explicitly state that ordinary unacknowledged message stanzas are
suited to sending a stream of momentary values. Loss is *very* rare in
XMPP, so treating this as UDP is not ideal.


> >
> > I think XEP-0198 provides sufficient reliability to traffic that the
> > additional overhead here is redundant, and we have XEP-0184 message
> > acknowledgements to provide end-to-end acknowledgement, too.
>
> "sufficiently reliability" depends on use case, it's completely
> subjective. For chatting maybe. Not for monetary transactions. The proposal
> outlines a mechanism (taken from reliable messaging) that makes sure a
> message is delivered exactly once, or provides the client with information
> that it could not be delivered at all.
>
> Message delivery receipts defined in XEP-0184 is not sufficient either.
> First, it does not implement the "Exactly Once" level, which is also the
> most important, and the principal reason for writing the XEP.
>
> While it can be used for the acknowledged message case, It also has
> several weaknesses: One is that an acknowledgement is only requested.
> There's no mechanism the request is understood. So the message can be sent,
> parsed, body of message processed, but no acknowledgement returned,
> generating new requests, etc. To have a solution that either works or does
> not work, with no in-between state of working partially, you need to embed
> the message in another construct. If that construct is not understood, the
> entire message is discarded.
>
>
There are two problems you're trying to address, but you're conflating them.

1) Ensuring the message reaches the recipient at least, or exactly, once.

2) Ensuring the sender knows if, and when, this has happened.

As stated above, these are impossible in combination; but rather than dwell
on that I think it's more interesting to observe why I think they're
already met as closely as possible with XEP-0198 and XEP-0184.

(1) is satisfied by XEP-0198, without resumption required. As the sender,
we can know if, when we are unexpectedly disconnected, the stanza had been
accepted by the service for certain. If it has been, then we're done -
otherwise, we can resend (for a possible duplicate). If we need
exactly-once, then the stream resumption mechanics will tell us if the
stanza was handled by the previous session.

(2) is satisfied by XEP-0184, in combination with XEP-0198. XEP-0184
provides the acknowledgement, and XEP-0198 ensures its delivery. Just as
before, different QoI for XEP-0198 means the available semantics may change.

Obviously to obtain reliability across multiple hops, each hop requires
XEP-0198; it may be useful to indicate reliability requirements in the
stanza, but I'd note that if XEP-0198 is in effect across all hopes, *all*
stanzas are delivered with exactly-once semantics, within the bounds of the
solution. (And my bounds are better than your bounds). Similarly, you might
prefer a different mechanism to XEP-0184. I'd even go for an <iq/> wrapped
system.

One of the possible reasons your proposed solution conflates these two
issues is because you don't consider the effects of the existing mechanisms
already developed to address reliability concerns. This seems to be a
recurring pattern with a number of XEPs; a cursory glance over the existing
corpus would have suggested that, as well as leveraging XEP-0198 and
XEP-0184, one would obviously use XEP-0297 in this particular approach.

Of course, the solution outlined in this XEP would (mostly; see below) work
on an end-to-end basis, without requiring participating clients and server
to be using XEP-0198 - however I've generally found that protocol
extensions which assume that other existing protocols are not or should not
be implemented end up causing a lot of problems further down the line.
We've seen this with the numerous attempts to deal with Google's lack of
PEP support, for example.


> >
> > If <iq/> is used, you'll always get at-least-once, no matter the support
> > for 198 along the route.
>
> Correct. But it would be a proprietary, non-interoperable solution.
>

Incorrect. This is bespoke, not proprietary. For example, a pubsub publish
is always at-least-once, and can be made very close to exactly-once.


>
> >
> > To get exactly-once semantics, the protoXEP recommends the receiver has
> an
> > infinite-sized store of previously received message identifiers - if the
> > store ever expires old message ids, then it's potentially allowing
> multiple
> > copies of the request to come through, which is (presumably) highly
> > undesirable.
>
> No, it does not actually. See §5.2
> http://xmpp.org/extensions/inbox/qos.html#memory
>
> The point here is that if it cannot be received (at the moment) there's a
> clear error message returned, so the client is alerted. The state is never
> unknown.
>
> Furthermore, processed messages are forgotten, and the double
> request/response mechanism still makes sure the message is only delivered
> to the application layer only once.
>
>
Right, yes, I'd missed that implication. You are, of course, reliant on
strict message ordering across multiple sequential sessions, but that's
relatively safe. You may wish to have the delivery phase cause a short
timeout rather than instant removal though.

Furthermore, I'd note that if the server assigns a new resource string on a
reconnect this protocol breaks down unless only bare jids are considered
(and then you need to ensure uniqueness of message ids across multiple
sessions).


> >
> > This could be avoided by having an if-not-modified-since semantic, which
> is
> > very useful in other cases beyond simple 1:1 QoS.
>
> No, it would not work, since that would require the receiver to remember
> processed messages. The two-step transmission/delivery process, which is
> standard in reliable messaging, does not require the receiver to remember
> processed messages.
>
>
No, that's nonsense - if the state can be encapsulated (for example, as a
hash-chain of operations), then a repeated operation is rejected due to a
state mismatch, and the receiver (and state controller) only has to record
one single value of the current state; no lists of outstanding message
identifiers, etc.

As a trivial example, you can consider a counter of operations (although
you really do want to have a hash-chain incorporating the sender's jid, and
so on).

In fact, even simple counting works perfectly well between two entities
where the messages are delivered strictly in the order they were sent; it's
how XEP-0198 works.


> >
> > Overall, considering the above, I'm inclined to push back until I've
> > understood why existing methods are not suitable for the semantics this
> > specification seeks to provide.
>
> I ask you to reconsider. The pattern described in this proposal conforms
> to common implementations of the reliable messaging Communication pattern,
> and is very useful, especially if you want to implement queueing and
> transaction support. (Reliable messaging comes with many names. Other
> popular names are Assured service, or Exactly Once delivery.)
>
>
No, I'm not reconsidering, I'm still pushing back until I've understood why
existing methods are not suitable.

There is already clear evidence that you have ignored existing XEPs when
writing this; I need to understand if you have ignored them out of simple
ignorance or for some reason I have yet to understand and you have yet to
explain.

>
> > On 8 December 2015 at 17:40, XMPP Extensions Editor <editor at xmpp.org>
> wrote:
> >
> > > The XMPP Extensions Editor has received a proposal for a new XEP.
> > >
> > > Title: Quality of Service
> > >
> > > Abstract: This specification describes a generic method whereby
> messages
> > > can be sent between clients using a predefined Quality of Service
> level.
> > >
> > > URL: http://xmpp.org/extensions/inbox/qos.html
> > >
> > > The XMPP Council will decide in the next two weeks whether to accept
> this
> > > proposal as an official XEP.
> > >
>
> _______________________________________________
> Standards mailing list
> Info: http://mail.jabber.org/mailman/listinfo/standards
> Unsubscribe: Standards-unsubscribe at xmpp.org
> _______________________________________________
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.jabber.org/pipermail/standards/attachments/20151210/b44d7633/attachment-0001.html>


More information about the Standards mailing list