[Standards] xep-0313 missing features
jonas at wielicki.name
Tue Mar 6 13:14:06 UTC 2018
On Dienstag, 6. März 2018 13:23:21 CET Lazar Otasevic wrote:
> 1. it is not possible to iterate efficiently backwards by including both
> <before>&<after> in the query because once <after> is included in the query
> then it iterates forwards. that means when iterating backwards client has
> to omit <after> and fetch entire pages and then the last page will mostly
> overlap with some of the local messages, which is a waste.
> 2. it is not possible to determine "holes" in the archive reliably, because
> client can not know what is the last message archive-id, because our own
> sent messages have no feedback from the server once the message is archived
> what is its archive-id ... that means that client has to fetch ALL messages
> from MAM once again just to be sure that holes are filled, even though many
> message-bodies are already received/sent during live communication.
> Basically In the current state all our own messages are "holes" in the
> local archive, not to mention all kind of "bad network" scenarios,
> multi-device and longer offline periods.
> Making separate requests, one for archive ids and one for content would
> - much less waste in the sync because only ids would be wasted, and not the
> - possible to fill multiple holes in one request by fetching content that
> is really needed
> - make possible for push payloads to contain only message ids (when clients
> want to handle encrypted messages locally by fetching them and only them)
> currently it is doable by giving to the push one id before the wanted
So I might be wrong, but for the sake of it, I think there is a sane and not
too complex way to do archive sync with MAM:
1. On startup, you put a "hole marker" at the end of your local archive. A
hole marker is essentially just the stanza-id of the last message at the time
the marker is created.
2. Iterate over all hole markers, from newest to oldest. Download everything
between the last message *before* and the first message *behind* the hole
marker. During download, move the hole marker accordingly to deal wtih
disconnects while downloading. When finished, remove the hole marker; move on
to the next hole marker if there’s any.
This should work and should also work with current semantics. I appreciate
that there *might* be some overlap between the last page of a hole and already
received messages. This is unfortunate, but can trivially be solved by
comparing stanza-id (if available locally) or origin-id or message id in that
order. (N.B.: once we get self-carbons, we’ll always have the stanza-id)
I also wrote that down in more detail here:
I think it would be great to have a way to limit the MAM query to an end-ID
indeed. Matt, Kevin, any chance we get that in?
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 833 bytes
Desc: This is a digitally signed message part.
More information about the Standards