[Standards] Problems with IM Message Routing: Message Types

Kevin Smith kevin.smith at isode.com
Fri Nov 10 20:30:57 UTC 2017

On 10 Nov 2017, at 19:57, Georg Lukas <georg at op-co.de> wrote:
> Hi,
> Hello, this is part 3 of the thread about how broken XMPP is
> <https://mail.jabber.org/pipermail/standards/2017-October/033499.html>,
> today we are going to cover Message Types.
> There are five different well-defined message types that influence how a
> message is stored and forwarded by servers and how it is displayed on
> clinets. However, message types are not applied correctly in practice,
> the rules in RFC6121, Carbons and MAM are inconsistent and leave big
> "implementation defined behavior" gaps, leading to a surprisingly large
> number of possible issues behind a simple "why didn't my message arrive"
> problem.
> The message type attribute is defined in RFC 6121
> <https://xmpp.org/rfcs/rfc6121.html#message-syntax-type>, and it has the
> following well-defined values and sematics:
> - „chat“ = typical IM
> - „error“ = things gone wrong
> - „groupchat“ = essentially reserved for XEP-0045
> - „headline“ = annoying pop-up
> - „normal“ = kind of like email
> The server-side routing rules vary for different incoming message types,
> depend on whether 'to' is a full or bare JID, on the number of online
> clients on that account and their presence priority value, and on
> implementation-defined behavior.
> Some examples of the rules, when applied correctly:
> - "chat" messages to an offline full JID MUST be re-routed to a
>  different "available" client of the user.
> - if multiple clients are connected, a bare-JID "normal" or "chat"
>  message might be delivered to none, one or any subset of those
>  clients, as deemed awesome by the server authors(*).
> (*) the Carbons rules somewhat improve on that front by always
> delivering all qualifying messages to all Carbons-enabled clients, but
> it is again implementation-defined what exactly qualifies a message for
> Carbons.
> Problems related to message types
> =================================
> There is no dedicated message type for system messages like MAM and
> Carbons, and it is not practically possible to add one - so we use
> "normal" by default.
> Some XEPs that define message payloads do not care to specify the
> message type to be used. As an example, "XEP-0184: Message Delivery
> Receipts" defines the types of messages that can contain a
> receipt request, but does not say what message type the receipt response
> message should have. The example provided implies type=normal, whereas
> some client developers implement type=chat or reflect the type that
> was set in the original receipt-requesting message.
> Some clients send IM content to full JIDs with type=normal.
> Because of this, some servers (ejabberd) apply type=chat full-JID
> rerouting rules to type=normal messages as well, which will sooner or
> later affect MAM responses from external archives.
> Some clients display type=headline messages as a focus-stealing popup
> (Pidgin).
> Some clients don't display certain message types at all (e.g. Gajim
> currently has a bug that prevents "normal" messages from being shown, and
> nobody noticed it).
> Rules for Carbons and MAM lead to an incomplete picture on some clients,
> like lack of ACKs or error-bounces on messages in the history.
> Analysis
> ========
> As it is now, the message type encodes multiple somewhat orthogonal
> properties of a message:
> - semantics (what is it supposed to mean and how is it supposed to be
>  shown at the receiving entity)
> - routing (where should a server send it)
> - persistence (will it go into an archive or not?)
> We have tried to address routing and persistence with Carbons and MAM,
> respectively. However, we were making assumptions about those properties
> based on the message type and existence of certain payloads, leading to
> imperfect results.
> I think that routing and persistence belong together (a persistent
> message SHOULD be delivered to all clients, eventually) in the general
> case, but there are probably exceptions like CSN, which might not need
> to be persisted.
> We could separate routing+persitence from the message type as far as
> possible, e.g. by explicitly using the resource identifier:
> - bare-JID = all-clients + archive
> - full-JID = single client, no carbons, no archive, no redirection
> This would bring us closer to the initial RFC6121 rules, but wouldn't be
> quite compatible with them or with Carbons/MAM, so it would require
> something like a new session type to work correctly (see also
> https://wiki.xmpp.org/web/XMPP_2.0)
> We could use message hints to explicitly mark exception from this, e.g.
> <no-archive/> for transient (CSN) messages to be routed to all clients.
> Feedback please :)

Thanks for the write-up again.

I don’t think this needs a new session type. It would be sufficient to enable these rules when clients enable ‘mamsub’ (for want of a better term).

I’m not sure I like <no-archive/> on its own (because it has unpleasant security implications), but we can probably work something up for it as long as it was in the context of clients that understand that if they receive ‘usual’ (I would say normal, but that’s confusing) messages with no-archive, the remote entity is trying to do Bad Things. (e.g. if you have a <body/> and <no-archive/> don’t show the message, because the remote entity is trying to send messages but have your archive lie - something along these lines).

We still probably want resource-locking in the short term, but nothing stops us resource locking for iqs while continuing to bare-jid messages.

Probably if we had some simple rule (I think <transient/> might be better than <no-archive/>, for semantics) where <transient/> is used for anything without ‘content’, but only metadata, then we can probably converge on the same routing rules for carbons and MAM. This is, incidentally, exactly why I wanted Carbons to have weasel words - because it lets us sort these things out later without breaking anything, and we can just add a new feature somewhere to tie it all together.


More information about the Standards mailing list