[Standards] Proposed XMPP Extension: MAM Fastening Collation

Marvin W xmpp at larma.de
Mon Jan 6 23:53:21 UTC 2020


On 1/3/20 12:56 AM, Dave Cridland wrote:
> Would this work if we just counted unique fastening summaries for each
> user, removing any duplicates?
> 
> It doesn't make much difference to me - it's just a unique constraint
> with different fields in.
> 
> Interestingly that would show edits via fastenings as only ever having
> one (which is probably OK), receipts etc would show a count per user but
> avoid duplicates, which seems sensible.

This sounds not good because then it's no longer possible to find out if
there were intermediate edits, so clients would have to request all
fastenings in some cases even if they already have everything they want
to display, just to find out if there is nothing else that they don't
know about.
> Also trivial to include two counts, one with and one without duplicates,
> which would satisfy either case.

That would work again for both edits and fastening, but probably will
cause issues in other cases.

> Yes, but that's needed if we want to support, for example, reactions (or
> even "likes") in a large chatroom.
> 
> Sending 50+ likes for each message would be extraordinarily painful in
> terms of bandwidth, and it's not unreasonable to assume that might
> happen quite often.

I agree that we want to have summarized reactions for chatrooms, but I
am not that set on extending summarizing to all fastenings with the same
concept. IMO, if we can't come up with an example where this counting is
actually a good idea other than reactions, it shouldn't be part of
fastening (the generic concept) but part of reactions (the only
realization needing that kind of summary).

> This is massively amplified by receipts - it's very useful to know if
> anyone, or everyone, has seen a message, but in a large chatroom that's
> a lot of receipts.
> [...]
> It may well be that Chat Markers don't make a suitable case for
> fastening, and I'm fine with that.

I don't think you can get the information that "everyone" in a larger
group chat has seen a message, because you don't know what the
recipient's clients actually do, they might just not support read
receipts or have disabled them for privacy reasons. The only really
sensible thing you can know, is if the group chat server received the
message, and that's already given if you fetch it from MAM.

> See last para of XEP-0422 §3.2
> 
> But in any case, the external things direct the client to the external
> parts, which logically form part of the fastening.

I haven't noticed that part, but it seems to be an issue in 422: if you
want to have two fastenings of the same type at the same time, they
can't have externals, because else you wouldn't know which external
belongs to which fastening.

> Shell fastenings are pretty useless, yes. But having them individually
> listed would, i think, be worse - and we can already pull the complete
> list of them if we need to. It's an additional round-trip, certainly,
> but that's a penalty if you're encrypting everything.

I was proposing to list the archive id of each fastening, not the
content. The reason for this is that I don't want to be required to pull
all fastenings every time there is a change for a message. If I have the
archive id on all the fastenings, I can easily see which of those I
already downloaded in the past/have in my local cache and only download
the new ones.

Yes, encryption makes some things harder. I'd still like to make things
with encryption as good as possible and not improve the unencrypted case
on cost of the encrypted case just because the encrypted case can never
be perfect.
People already start to expect encryption as an essential feature for
instant messaging and XMPP should be the last to stop this from
happening. And as we are mostly talking about IM usecase here
(reactions, edits, receipts, group chat), we should rather put a focus
on encryption instead of mostly excluding the complexity that comes with it.

> I'm not saying that having per-fastening code in every server is slow
> just because there's a lag to them updating, I'm saying it'll be slow
> because you'll have several round-trips between server developers and
> client developers as the feature is developed and it'll take literal
> years for each one to become stably deployed, as well as creating a vast
> amount of workload for the XSF and its participants.

How is this worse than having a "generic" feature that works good for
reactions and edits, but doesn't work for future extensions and thus
will need to be updated every now and then, having the same cycle you
described but at the same time may break all the existing infrastructure
for previous fastenings.

If we do a rather basic MAM-FC (that doesn't try to solve problems it
isn't even aware of as they aren't known yet) and have per-fastening
extensions, we might end up with re-iterations on the per-fastening
extensions, but those wouldn't affect a) the functionality of the basic
MAM-FC (thus keeping every server functional, just maybe not optimized)
and b) all other fastenings that don't need to be updated anymore. To me
this sounds like the better way.

Currently no client or server has implemented fastening, MAM-FC or any
server side optimization for reactions, thus arguing that it needs to be
done in a way where servers don't need to implement something
specifically for reactions makes little sense, because servers would
implement the "add number of occurrences of a 'fastening summary'" rule
specifically for reactions and the "have full content of latest
fastening" rule in MAM-FC specifically for a (yet-to-be-written) edit
feature.

Another issue that I noticed: you are relying on the thing you call
'fastening summary'. What happens if we allow for more complex reactions
that can't be put as an attribute, but are realized as an child element,
like custom images. This was already expressed as a desired feature here
on the list and is easy to realize from the reactions proposal in the
inbox, but hard using the "hidden" reactions proposal in your (Proto)XEPs.

> You're expecting servers to track individual authorisation of likes
> within chatrooms?

I'd expect clients to track reactions in group chats by occupant id
(XEP-0421) instead of senders nickname. This was already teased in the
reactions proto xep
<https://xmpp.org/extensions/inbox/reactions.html#acceptable-reactions>,
although occupant ids weren't a thing back then.
Thus I'd expect servers to do the very same. For edits, that's even a
stronger case and I would hope for a server at some point to just filter
out invalid edits. Even if you don't make that part of the XEP it would
be a sensible idea for servers to do that to reduce load for both server
and clients at no practical cost (as clients would have to filter them
out anyway).

> You're saying that the XSF should only accept completely finished
> proposals in this space?

I am saying we should not block mostly finished proposals for the lack
of missing other proposals or requiring proposals to use mostly
unfinished proposals that likely are going to break backwards
compatibility at some point for reasons that could be solved completely
different. If the reactions XEP would just define how reactions are to
be summarized, it wouldn't rely on a XEP like MAM-FC. Also other XEPs
that are not strictly MAM-FC could make use of such rules as well (for
example, MUC servers could apply reactions also in MUC history and not
only in MAM).

> OK, so you're saying that the client sees there are claps, and can
> retrieve all the clap fastenings and process them locally.

I wasn't talking about the client retrieving all fastenings, in this
example the <applied> elements are directly in the <result>, the same as
in your ProtoXEP. It doesn't need another round trip to request anything.

> How does the server know what the legal limit is for claps, though? Have
> you declared that every server in existence should support your limit of
> 50 and no take-backs? Otherwise this cannot possibly work.

Yes, the limit of 50 was part of my protocol specification. Yes, this is
a hypothetical specification. In a real specification, the per-person
limit would probably set by the chat room or server admin, but that
would still work (as long as clients can fetch that number).

> Although MAMFC will work, it just may require an additional round-trip
> for this contrived case.

As mentioned above, I see the risk of the summarizing - if done wrong -
to cause the client to be unable to see that they need to fetch
something from the server. And we want clients to only load the minimum
necessary.

> I'm suggesting that MAMFC provides a compromise that works well enough
> for most cases, and imposes an additional round-trip for some others.
> 
> I don't doubt it can be improved, but I think the fundamental shape of
> this works.

I am certain it will work. The same applies to other possible ideas.

Marvin


More information about the Standards mailing list