[Standards] Proposed XMPP Extension: MAM Fastening Collation

Marvin W xmpp at larma.de
Thu Jan 2 20:41:52 UTC 2020

On 1/2/20 6:23 PM, Dave Cridland wrote:
> It's not clear to me whether sending two poop emojis should count as two
> reactions or just one. (Surevine once had an internal product that, by
> accident, allowed "multilikes". We liked it. Often more than once)
> As things stand, we're effectively defining that people can like (or
> poop emoji) other messages as many times as they like.

As a reminder, https://xmpp.org/extensions/inbox/reactions.html
specifically said that there can only be one of each reaction per user:
"A <reactions> element MUST NOT contain the same reaction more than
once. A receiving entity SHOULD ignore duplicate reactions inside a
<reactions> element." This is a feature specifically wanted and desired
by the authors, if MAM-FC is not supposed to be able to handle this
feature, then Reactions can't make use fastening.

> In general, the "edit" case (as with any retraction case and others) is
> a little shaky here because of authorisation. I'm very much open to
> ideas here - it might be sensible to have a switch like the "external"
> which pulled in the stanza from attribute, for example, or we might
> simply choose to have servers "know" about edits (but see below).

In the unfinished proposal I shared in the last mail, each <applied>
also had a from attribute which is the from of the message that included
the fastening. This however does not work in your model due to multiple
fastenings being collapsed if they have the same type.

>     Also note that the edit proposed in this ProtoXEP is not really
>     compatible with the edit proposed in the fastening XEP
>     <https://xmpp.org/extensions/xep-0422.html#external-payloads> as it is
>     completely unclear what is supposed to happen with the <external
>     name="body"/>.
> Yes, quite, these need pulling into the fastening summary if they exist
> (ie, they're not being pulled in from the encrypted blob). Totally happy
> to make that clear.

I think you missed the point that the <external /> is not a child of
<edit /> but a sibbling. According to your proposal, only childs of
<edit /> would be inside the <applied> element. So even pulling in the
body doesn't describe how to handle this at all. This might as well be
an issue in the XEP-0422 as it also says that an <apply-to> may contain
several fastenings, but they must all be of the same type, and body
doesn't seem to be the same type as <edit />

> FWIW, we use Chat Markers as super-receipts only, so I may have a blind
> spot there.

This pretty much confirms my feeling that this XEP is targeted for very
few specific usecases which is exactly the opposite of generic. The
whole purpose of fastening is to have something generic and that's the
reason why we are delaying the proposed reactions XEP. If we now build
something that is again not generic (and can't even handle the concept
and ideas layed out in that proposed XEP) we won nothing and
unnecessarily delayed a real feature that is often requested by end users.

>     I also don't like that if I have a shell fastening, I only get to know
>     that there are fastened messages. If there is 20 edits and one reaction
>     and I want to know who send this reaction, I shouldn't be requires to
>     download the 20 edit fastenings (using your proposed "fastenings" MAM
>     call). I think all <applied> elements should carry the id of the
>     underlying message from the archive, so that I can specifically
>     fetch those.
> Shell fastenings are for encrypted cases, so the server simply cannot
> tell what these are and has no way to tell, by design. This was
> specifically to satisfy cases where "everything must be encrypted".

What I was proposing is that instead of adding the shell="true", add an
id="mam-archive-id-of-the-fastening-message", so that clients can
specifically fetch those instead of having to fetch all others (which
may have not been shell fastenings). As shell fastenings don't have a
count (at least I presumed that, because you'd just be counting the
number of shell fastenings which sounds pretty useless), one shell
applied element always references exactly one message.
> But if each individual fastening type has to have code support in the
> server, that really slows down the rate of innovation.

I am aware this is an issue because servers update slow. But rather than
putting arbitrary restrictions of what can be done with fastenings, I'd
prefer to have two levels of fastening collation, one that only
describes the existence of fastenings (pretty much like shell fastenings
in your proposal) and one that does smart collation. That second level
would then only work if server supported that specific fastening type,
so it could sum the reactions for reactions (including the limit of one
reaction per kind), directly apply edits or whatever else comes in your
mind. Of course both clients and servers would need to announce which
smart fastening collations they support and servers should only use
those that the client announced to support.

> I'm absolutely not averse to having the client add in how a fastening
> should be summarized - that was my original design actually - but we
> decided that the count + latest was sufficient for all cases.

I think this count thing is a bad idea. It is totally targeted towards
reactions but even fails to provide the guarantees needed for proper
usage of them. This is by the way getting way more complicated in
semi-anon MUCs, because servers have to ensure that I cannot create an
arbitrary number of likes by just changing the nick or changing the
reactions someone else did by taking over their nick while they are
offline, so we probably would need to incorporate occupant id somehow.

The latest thing is at least not a bad thing in most cases, so I'd be
fine with that, as long as it is filtered per sender (showing the
content of the latest of that fastening type for each sender instead of
latest in total).
> It *is* unfinished - of course. It doesn't even have a XEP number
> (though it does have a schema etc already).

This is perfectly fine of course. I was only mentioning this because we
are already requiring everyone to use fastening even though we kind of
agree that this isn't really a good idea at this stage because it is
rather unfinished.

>     Just to come up with a new potential idea that would be hard to
>     impossible to realize with your proposal: Clapping; on a popular blog
>     platform you can clap for blog posts. Every user is entitled to up to 50
>     claps on a blog post, but you can't undo claps. Usually only the total
>     number of claps is displayed, but there is a detail view where you can
>     see which persons clapped and how often.
> Right, so, how would this work with MAMFC as written.
> So, users can clap on a message - OK, so we have a fastening type of
> <clap/>. This summarizes to <clap/>, and has a full fastening of
> <clap/>, so that's trivial for MAMFC so far.
> The total number of claps is shown, which again is fine for MAMFC.
> Our problem is that each user can only clap 50 times, so one assumes
> that in a chatroom of 10 users, the maximum clappage can be 500? (or
> 450, if one cannot clap oneself, which is at least gauche...). MAMFC
> cannot currently enforce that, you'd have to enforce it client-side, or
> by introducing a blog service which mediating clapping.
> But equally, it would have to be enforced somewhere, and to enforce it
> inside MAM would mean it would have to be enforced in code on every
> users' server who participated.
> If we ignore the limit, it becomes trivial - if we have "undo" claps,
> then I think MAMFC should be able to handle that, though I've not
> specified that yet because I was slightly ahead of Kev's XEP-0422
> update. But I will do.
> So, summary: We cannot do your clap proposal with MAMFC, nor with any
> similar system which doesn't involve either hardcoding the support in
> every server or including an arbitrarily complex summarizer DSL code
> fragment alongside every fastening.
> Is this a problem?
If we picked a different MAM-FC the 50 per user (instead of an n*50 for
n users) limit would entirely be possible: we don't have MAM-FC do any
summing and we add the from to the <applied> and instead have each user
always sends their total number of claps instead of just <clap /> for
each single clap and replace the previous fastening. We then end up with
a list of the latest number of claps of each user like this:

<applied from="X"><clap count="20" /></applied>
<applied from="Y"><clap count="40" /></applied>
<applied from="Z"><clap count="60" /></applied>

The element from Z has a too high number, so we just cut it to 50, the
total count thus would be 110.

What is not possible using this "latest" approach is that we can't
verify that users are not removing claps. If we get rid of the "latest"
mentality completely, we might see that X had send more claps before:

<applied from="X"><clap count="10" /></applied>
<applied from="Y"><clap count="20" /></applied>
<applied from="Z"><clap count="30" /></applied>
<applied from="X"><clap count="50" /></applied>
<applied from="X"><clap count="20" /></applied>
<applied from="Y"><clap count="40" /></applied>
<applied from="Z"><clap count="60" /></applied>

The client would then be able to calculate a sum of 140 for this which
is the correct number under that desired specification. Of course this
has a higher overhead. If the server has support for understanding
clappings, they could just send

<applied from="X"><clap count="50" /></applied>
<applied from="Y"><clap count="40" /></applied>
<applied from="Z"><clap count="50" /></applied>

if both server and client announce support for smart summarizing of
clappings, it could even become

<clappings count="140" />

The point I want to make and the problem here is that your restrictions
for MAM-FC will effectively make certain usecases impossible. The only
way to prevent that is to not make restrictions of fastenings that you
don't understand. If you don't want fastening to be generic enough to be
used for most usecases, fastening itself becomes useless.

Yes, requiring server support is bad, but I think an approach is
possible where the server doesn't need to support a fastening specific
collation, it just can improve its bandwidth usage by doing so. Clients
need to be able to support the non-summarized version anyway (because
that's what they receive when not fetching from MAM) and they should
also be interested in using smart summaries where possible to reduce
bandwidth (thus downloading from MAM faster). Thus there is a natural
wish of both clients and servers to implement the smart summaries when
certain fastening types become popular whereas unpopular fastening types
can still be used with no issues because you didn't put any unnecessary
server side restrictions (and if they are unpopular anyways, the server
doesn't need to fear bandwidth created by them not being summarized).


More information about the Standards mailing list