[Summit] day 1 notes

Peter Saint-Andre - &yet peter at andyet.net
Sat Oct 10 02:19:22 UTC 2015


Backing these up by posting to the mailing list. :-)

###

This pad is for notes and discussion about XMPP Summit 18, especially MUC2.

Topics:

     MUC2
     MAM & Carbons
     Push
     Unread Markers
     Work through open issues at https://github.com/xsf/xeps

MUC2

Kev will provide an overview of what we're trying to achieve here. :-)

Pubsub/PEP Subscriptions




Minutes:
     Kev: a long we've people have said we should ditch MUC, and we've 
laughed at them. for good reasons, but this year we've changed our minds
     a lot of people have designed their own MUC systems because MUC 
didn't do what they needed
     like MUC where presence is not tied to occupancy
     nick sharing is basically broken in MUC 1, based on hacks
     presence-less mucs, messages only
     doesn't play nicely with MAM (storing lots of copies, one for each 
client in the muc)
     MUC2 uses pubsub
     Peter: What other problems? ghost users? iq requests?
     Kev: Have spent the day writing the MUC2 spec
     MUC2 we decouple nicks from all addressing - can see all clients 
behind a nick and address them
     in a non-anonymous room there are no fake JIDs at all
     in semi-anonymous rooms there still need to be fake JIDs, but not 
tied to nicks
     ghosts if you no longer tie occupancy to a room to online status, 
ghosts kind of go away. problem is down to stale presence
     Dave: i think ghosts *are* stale presence
     Kev: in muc1 ghosts are worse than stale presence, because 
occupancy is based on presence
     Dave: from the user's point of view, it seems to be the exact same 
problem. trying to talk to someon thats not there
     Kev: But messages will still end up arriving to the right person
     Dave: Maybe, but the problem is still there
     Kev: Its the same as what happens with contacts in your roster
     If we're going to solve that one, we'll need to solve it outside of 
MUC2 itself
     Basic premise of MUC2 is that it is like PEP, similar scheme
     every room is a PubSub service
     each type of data has a node on that service
     anything you can put in a node, you do. to avoid any extra 
protocols. same flexibility for everything
     tie MAM into MUC2 by doing everything by using MAM
     can do MAM with pubsub nodes, so MUC2 gets lots of history/syncing 
for free via MAM
     fast resync happens via MAM
     can sync only nodes you are interested in
     no longer need for the history hacks in MUC1
     written up at: http://hector.doomsong.co.uk/scratch/muc2/muc2.html

     Peter: to the requirements, i would add something about data types, 
such as files or video streams
     Kev: a point about MUC2 is that it makes no attempts to be 
backwards to MUC1. It might now be sensible to MUCs in your roster now
     Sam: what are peoples thought about not doing an anonymous MUC but 
handle that via things like SASL anonymous
     Waqas: there are still cases where you want to not reveal how ?to? 
join a room, semi-anonymous rooms
     Dave: two cases - people still prefer to join rooms that are 
semi-anon as people prefer to be very seelctive about revealing their 
identity; also the military case which is where the nicknames within a 
chat room represent active roles rather than actual people - take for 
example a cmdr may go into a room as the commander and how a shift 
change happens is that they physically take the seat (the role) of the 
cmdr and just assume control of the JID that is already active. In MUC1 
that would be done by nick sharing across different bare JIDs
     ??: is that a general MUC problem or a nick sharing problem
     Kev: ??
     Dave: that is true, on the other hand I have a mild concern about 
MUC proxying solution becoming a massive abuse magnet
     Kev: more often than not, when they do want to share the role they 
still want to have their own JID active in a room
     Dave: the reason I don't want to introduce a middle ware that is a 
JID proxy is that the real JID is not exposed to a chat room, so that if 
I want to ban Kev from the chat room then a JID proxy would allow Kev to 
still manage to connect to the room
     Kev: what i'm proposing is that JID proxy would be performed by a 
muc service, it hands out a JId...???
     Dave: so your differnt nodes in a MUC2 hash messages, we should 
just give them URNs and avoid using short names like 'muc2#messages' - 
everytime we try that it comes back to bite us
     Peter: agreed, those things were hacks from before we knew what we 
were doing
     Dave: you have a MUC2 hash message, if you want the chat forms in 
this, it would be a MUC2 chat message
     Kev: you would use both messages and chat forms, you would publish 
the chat forms node and then you publish a reference to the node
     Peter: what are chat forms
     Kev: these are things like "send a medivac", they are big in 
certain communities
     Dave: they are things you use when people are either dead or dying; 
what I'm wondering is.. we want them in the msg stream anyways but we 
don't want them poluting clients that don't want them
     Kev: ???
     Dave: the reason why I do think it matters is that I'm trying to 
get the overal design and for most I do, a subj is not a message it's a 
different thing, a presence is kinda as it's a stream but it's not a 
message,
     Kev: yes, the easiest case to understand, is that someone places an 
image into the stream and being able to centrally handle/do this, now 
for images there is a potentially interesting case is that they can be 
large and for large things having them by reference seems to make sense
     Dave: so what does my client that doesn't support images see
     Kev: that depeneds on the additional data that would be sent along 
with the image reference. one of the things that we want is to have a 
message that has a reference to something else that the client can then 
fetch, what matters is not the immediate format but that there be a way 
to do "hey client, go look over there" and that there be a framework
     Peter: the case of someone posting a huge GIF that may have to be 
downloaded
     Kev: if it's done by reference, you got the URL and can go outside 
of XMPP to retrieve it
     Lance: we have jingle-pub for similar purposes - 
http://xmpp.org/extensions/xep-0358.html
     Dave: what i'm driving at for these references is that a client 
that understands images, the example, it's going to potentially 
subscribe to the images node of the MUC and it can reassemble the 
message stream and then say "oh, kev said hello and then someone sent an 
image and then someone replied" but if you have a ref then the ref has 
to be the sequence of record so that you have to upload and then insert 
the ref into the stream
     Kev: yes
     Dave: for example a github commit record, these kind of 
integrations can now be seperated out would we  add ref to them for these
     Kev: well, it depends on whether they are wanted to be in the 
reference stream, for say a large image node, it could be subscribed to
     Dave: you could sub to it for notif only
     Peter: for early days of group chat we had "ralphm went away in a 
puff of smoke" but we really don't need them, although we miss them as 
they were a lot fun
     Dave: if we were to take msg changes and display them in the 
stream, if we are going to do that then the client will have to insert 
them into the stream
     Kev: presence doesn't have to be reflected in the stream
     Dave: ???
     Kev: you still need references in some of the examples as they are 
outside of the room, you still need the reference
     Dave: oh, ok - it might nice to publish the form to the room, the 
usecase of refereing to something outside of the room may be a side 
issue to this
     Kev: arguing the point that it doesn't matter, we know that it may 
be a reference to an outside stream item and that someone will be pub to 
a node, the detail is that if you arereferencing something that is in 
the muc's nodes can be handled later
     Peter: it can be handled out side of the muc and can be handled by 
the application, it seems that it's not something that needs to be specified
     Dave: trying to get a handle on the overall design philosophy
     Kev: the overall architecture is clear and this is a detail, he 
wants to get a general agreement that the text he has wrotten is not 
stupid and that if people can help and implement and that this can then 
be proven as handling this usecase
     Dave: this is ??? ground breaking and we will be able to build it 
and he is worried that the use cases are covered for the long term and 
we will ge the usability we want
     Kev: if we quickly try to hack something we will be able to 
discover the edge cases and flaws that are not obvious immediately
     Waqas: if we have mulitple streams will we be able to unify the 
streams and if we have multiple we would need ordering would that be 
done with timestamps
     Kev: for the client in the room, it's just received order; when not 
in the room, you would use the forwarded time stamps that are provided 
by MAM
     Waqas: and if you have two timestamps do you resolve that by order 
of receipt - I would not be able to tell if ??? so ordering by the 
client would be ???
     Kev:
     Waqas: that sounds fair, when you pub to a pubsub service does it 
happen to serialize ???
     Kev: when you publish to a muc you serialize across the service
     Waqas: it has to be seriliazed across a service and we cannot have 
eventual consistency convergence across different muc services
     Waqas: ??? about two cluster nodes ??? nodes ???
     Peter: every node that is created is associated to the room root 
node so wouldn't they have to be associated? then how do you figure out 
how different nodes / streams are associated
     Kev: if one person in muc1 rooms joins and then you send a message, 
you will get eventual convergencie
     Waqas: in muc1 serialization would be done per room and not per service
     Kev: you still have it per service ????
     Waqas: serialization / ordering in MAM is problematic
     Waqas: a client would need to define ordering
     Kev: do we do sub-second timestamps
     Sam: you can always make the timestampls less and less
     Kev: equally the problem to solve is not sycn across the messages - 
this is slithly less of a problem than temporal ordering
     Peter: there are a few use cases where someone needs to know the 
precise ordering to prove something authoritatively
     Waqas: anything that is done machine to machine, and in certain 
cases that machine use cases you need precise ordering
     Kev:
     Matt: even for the human use case - auditing and logging
     Sam: for example, turn on guest access, can't see messages before 
then - might be security implications
     Waqas: time is interesting - leap seconds and such
     Matthew: timestamps should be only FYI
     Sam: also system-dependent (e.g., leap seconds on BSD vs. Linux)
     Kev: any other suggestions?
     Bear: in ops, set on system side and don't depend on client timestamps
     Dave: an alternative, why not have all events in one node? clients 
can filter
     Kev: why not have a #events node for sync - I assert that for vast 
majority of use cases, sub-second stuff is not needed
     Dave: ???
     Bear: in most distributed systems, use sequence number - for 
tighter precision, receive time vector clock from a central node that 
handles sequencing that all of the nodes subscribe against
     Peter: if I understand correctly, a central synchronization service 
could have its own node in the room
     Bear: ???
     Dave: clients care about the presentation, why not just do it the 
right way the first time?
     Kev: because it matters only when you're in the room, 
reconstruction is a different thing
     Matthew: you're describing the MUC1 behavior as desirable, but I'm 
not sure it is - can use subscription options to filter
     Kev: we don't get the benefits of using PubSub, configuring 
separate configs for different data streams, etc.
     Peter: agreed
     Kev: for systems that require that level of auditing, they can 
provide that service with a separate meta node
     Peter: isn't that just the root node?
     Kev: potentially - if have MAM on the service, you get things in 
order - could add a form field to MAM for filtering
     Peter: I'm not sure that the only examples are for historical 
reconstruction
     Kev: if you're live, you depend on received order
     Bear: you're pushing precise ordering to MAM instead of MUC2
     Kev: ???
     Waqas: ???
     Kev: is this live or for MAM queries?
     Waqas: we do need to sync MAM to the live stream
     Kev: not really, because we have IDs - when you're live, you 
receive what's live, and you can make MAM requests to get things before 
that happened
     Waqas: would be nice to have sequence ID in MAM responses
     Kev: you have that because MAM responses are ordered by ID
     Kev: which is why in the case where you are required to have 
precise ordering you do a MAM query and receive the order based on IDs
     <-- new topic -->
     Dave: you have an occupants node, what is the difference between 
that and it's subscribers, the reason I ask is because buddycloud 
handles the who is in this channel(room) by looking at the subscribers
     Kev: because you may have diff subs for each node in a service and 
potentially you may want to have some subs who are not participants
     Dave: is that the invisible observers, what does occupancy mean
     Kev: it means tht you are a mmember of the room, because what 
happens in modern chat severices is that your not only a member only byt 
you also a member of a group and contains additional meta info about the 
subscriber/member
     Dave: ok, if I publish my occupancy to the occupants
     Kev: you don't
     Dave: I don't so this is automatically maintained
     Kev: yes, what happens is that ... ignoring the fact that there may 
be admin cases to this, what happens is that you send some variation of 
a join element to the room which is possilby a sub to a node (or 
something) then the muc service does something based on this and one of 
the things it does is add the relevant meta informtionto the occupants 
node for you and tho this is pbusub and we want to use pbusub services 
for the sake of this ext an for the sake of this data and these are 
nodes whose contents are generated automatically -- and indeed the 
message node may be published (generated) automatically -- so I think 
all of the standard nodes will be server maintained
     <-- new topic -->
     Dave: ok, that seems reasonable, next question, you got subject but 
you also got topic - is the room's topic not just a config item
     Dave: you have subject and you have config - isn't the room subject 
just part of the config?
     Kev: I would argue that it's just data and not meta data and this 
is not something he does give a high F value to
     Sam: would be nice to have ability to set friendly room name
     Kev: we have that in MUC1 but it's part of the room configuration 
and only admins can get at it
     Dave: the right way of doing this is to have a config and a meta 
data node where the meta data node is the room name, subject and the 
config is something only the admins of a room would write to or even read
     Kev: the reason to have subject in the event stream is that it's 
the way it is in MUC1 and that is how Kev's brain is wired - there's a 
question of whether we want a node per metadatum
     Dave: for comparison, buddycloud for config and for things like 
rooms name and description, he doesn't think is great but he thinks 
there is some degree of merit - he is not sure why you need a config 
node but a meta data node means you can reconstruct topic changes
     Kev: some people care a great deal about what state exists in a 
point in time so exposing this data as a node we have a way of 
extracting the config's change history
     Dave: that's reasonable
     Kev: anything that could be a node and be could be fetched 
historically we just make them nodes explicitly
     Waqas: do you expect the client to be able to make changes to the 
config based on auth level, from readers point of view can they get all 
of the config node chnages or only filtered one
     Kev: for a general case you not let them sub to the node but that 
anything that is necessary for clients to change is that that datum go 
into a different node, changes to config would be done via data form and 
then the service would push data items into the proper nodes
     Waqas: might not want to mix security-related items - e.g., 
blocklist or spam-fighting
     Kev: split things logically depending on whether it's in the same 
"security domain"
     Bear: knowledge of item IDs can be based on role
     Kev: you could do that but we don't have infrastructure for that 
now - who is able to read was controlled in MUC1
     Peter: in MUC1 we had events that we told the client that we 
thought were of interest, the room is non-anon, the room is being logged
     Waqas: it was very adhoc who implmeneted them
     Kev: what we could do is not that this is an open issue and then 
circle back to talk about this on the thought that you could do an MUC2 
implementation that has totally open config and then circle back to 
resolve this issue
     AI: Kev logs an open issue

     <-- new topic -->
     Sam: what is the story for private msgs in semi-autonomous mucs
     Kev: for non-anon it's straight forward you have the full jid and 
nothing is hidden; for semi-anon you have the fake jids that the room 
creates for you so you use them for private - if you want to allow it or 
not you disal;low it in config
     Waqas: I'm curious if we do want to tie the anon status of the room 
to the ??? and that may or not be desirable, it seems interesting that 
the room may want to track this status - I may want to ???
     Kev: there is nothing that stops a muc service from making the fake 
jid from acting like a real jid so that all of the things that happen 
with a real jid would happen - the advantage to doing that for muc1 was 
that the nicks were associated to a jid, in muc2 they (the nick) survive 
as long as they are wanted by the muc2 service
     Waqas: ???
     Kev: why would you want to do that
     Waqs: some of the use cases have that some people discussing that a 
mam for my nick and that would let me get archive of my messages because 
I have spun up a new client
     Kev: you would do that from your jid's own mam archive - and now 
that the jids are now longer lived in muc2 that makes more sense -- ?? a 
non-muc2 client could connect, see the nick history in it's mam and then 
continue the conversation all without knowing that it's a muc2 room
     Waqas ?? anon proxy ??
     Kev: yes your wrong in that you don't want to go there, if you tie 
it to a muc room the muc knows about the psued-anon and it can still do 
blocking based on real jid and show the real jid to the admin and it 
still knows what the server is so it can do spam prevention, where if 
you have a anonymizing proxy you lose a lot of that
     Waqas: and the whole anonymzing proxy that exists ???
     Kev:
     Sam: this is interesting that the room is no longer the source of 
the anonymity but the user's connection to the muc2 service
     Lance: you could have only admins be the ones who are using 
anonymous proxies
     Sam: leaves that up to server policy
     Dave: intersection of server policy and user preference - you 
discover anonymity you discover it only after you've joined
     Kev: that would no longer be the case because in Muc2 the config is 
discoverable so you would be able to pull the config and know that
     Dave: you can have anon requirements as a property of your 
subscribe/join and if the service doesn't want to support that or grant 
it , you will get rejected, which is neat and tidy
     Sam: it's nice and atomic, you will get a discrete/distinct error

     <-- short break -->
     Kev: I'm happy with MUC2 but it will improve through more questions
     Peter: at a higher level it would be good to talk with people who 
have been trying to work around issues with muc1 and if that muc2 solves 
those issues
     Kev: certainly he has floated this idea at fosdem that this does 
address those issues that he has heard of
     Waqas: the only thing that he is thinking about is references as 
that is clearly a non-atomic operation and does that matter
     Kev: what your doing is making a resource available and then you 
are referencing it
     waqas: i'm happy and that it covers most of the use cases that he 
needs and that it should be possible to offer a old muc service to 
interface to muc2
     kev: mlink doesn't do muc over xep60 and it has an abstracted 
pubsub interface and it's not quite the same thing
     kev: one of the details of this that makes this work very well and 
maybe dav'es dog acn proxy for him, is that it works much better if you 
implement "don't do pubsub from your client, pubsub from your account" 
because if you have that things get much better because you can instead 
of linking nodes that you subscribe to, you can finally solve "which 
rooms do I know" "which rooms should I be in" - having admins put you in 
muc rooms, so like sysadmins may want to shove your roster to you, with 
this protocol because your client doesn't , let me rephrase, with muc1 
your client knows what rooms your in because it has to sub to them, with 
muc2 it knows because the client asks the server so the sys admin could 
put you into a muc2 room and then the client will know that
     Peter: we've had many cases where that would be helpful (e.g., 
online education system)
     Lance: we abused the roster for that
     kev: there is another point, muc2 doesn't go into yuor roster, on 
the basis that sending your presence to a muc is ???
     lance: that is something that I tried to do for a client because of 
the number of rooms, it did not work for muc1 because the initial 
presence did not have a resource, but if it had worked lance would have 
been very happy
     kev: do we want to make this the model for muc2, do wwe simple say 
you put muc2 in your roster
     peter: yes
     kev: i'm aware that this may have clients have to mix muc1 and muc2 
in the roster, how bad is that
     lance: I don't see it as a problem as it's not the humans managing 
the roster, it's the same as a muc1 room because you don't know it's not 
a contact until you query all of your contacts
     kev: but for muc2 you do because you do pubsub from the account, 
not from the client
     sam: ???
     kev: kind of but as a muc2 client your going to need to ask what is 
the list of your muc2 rooms anyways because you will want to know that 
at startup
     sam: what is the benefit of knowing in the roster
     kev: because it keeps you from having to send out presence requests
     waqas: should that be the muc2 room in the roster or should that be 
the muc2 service
     kev: I think it should be room, not service, because you want to 
send presence to the room - this feels like a general XMPP model and 
keep to it
     waqas: if I'm a remote admin and I add Lance to the room, what happens?
     kev: you don't add, you invite - maybe for muc2 we merge the old 
direct and mediate invites
     waqas: you can't invite anon can you
     kev: in theory one could invite anon but I don't think you want to, 
in principle this invite could be anon or come from the room, we should 
speak that the invite ??? after telling the muc that your going to 
invite this person, leave open as implementation, you could have a 
marker for invites that the muc has intercepted - {???} and add invitees 
to subscriber node but not occupant node
     lance: he wants to see what rooms he has been invited to after 
coming back from offline, the roster was the only place to see/store 
this information
     kev: the muc2 service should be able to do this by querying the 
nodes for rooms that the jid is sub'd to - this works if we're using 
pubsub on account

     <-- pubsub on account -->
     kev: the basic premise is that currently all pubsub sub management 
is to the room because you do it with full jids or bare jid subs that 
you subcribe from your client and the only thing that knows is the 
client that did the sub, bare jid so that the client is the one that 
knows where it needs to go or you sub full jid and ???
     kev: you ask your server to sub your bare jid, your server knows 
about subs and then you do a reverse PEP thing on the client side of 
your server where your server sends the pubsub notifications to your 
client so it's your server that is doing the de-muxing on the 
notifications based on the clients that want them. if you sub while 
offline the server knows about the subs so the account will be received 
as each muc msg once, so your server knows everythign and your client 
can ask your server everything and you now have a way of finding out 
what subs you have, additionly adv if your server has Mam you can also 
capture all of the notifications while you wwere offline
     sam: helpful because your local MAM store can be quite large even 
if remote (e.g., MUC) server has limited storage
     lance: ??? where you have to query your own mam server and you 
don't care that other servers dont have mam support
     kev: it will work without it but if you can still do full jid subs 
to muc2 however it will be much uglier than ???
     waqas: would you expect your server to resync if s2s goes down
     kev: I haven't assumed that
     lance: if I was writing a client then i would not ever have to care 
about that again
     kev: we should get Fritzy to write up that pubsub proxy idea from 
FOSDEM a few years ago
     lance: wasn't fritzy's pubsub proxy like a pubsub inbox?
     kev: yes

     kev: do hipchat and slack at least basically have the same model
     kev: the model is the same in that slack you have per scene views 
and you share everything between devices and you can post rubbish to 
rooms and you can comment on things hanging off the side of rooms, if 
sam says that hipchat has the same model of slack then we can also now 
check-off most of the boxes that hipchat needs or wants to support
     sam: I'm reasonably sure we've ticked off most of the boxes that we 
support or want to support in the near future - big one is s2s 
synchronization, which we don't have right now

     kev: the ability for your server to resync if it wants to requires 
no extra protocol is important, if we're using pubsub from account
     sam: the arbitrary meta data is also an important one - we tack on 
a lot of extra data, the clustering story is good
     waqas: does the pubsub proxy need to be muc-aware?

     kev: I think it might end up needing to be aware depending on how 
subs are handled, the open question is are we making our join stanza for 
the muc a simple modified xep60 stanza then the pubsub account knows 
about the muc - not quite clear to me and we will have to solve this as 
the first things - whether the pubsub sub for a join works
     ralphm: my preference would be to have a special MUC join instead of
     waqas: subscribe should be different from available
     kev: sub is already a different operation than a join
     lance: what are you sub to because of all the nodes available
     kev: you sub to the root node, which could give you default subs to 
the various sub-nodes
     ralphm: if you do a specific join using something other than a 
pubsub sub you can do something silimiar to an auto-sub for pep. so in 
pep when you send presence to an entity ??? if you have pep you 
baiscally sub to all the nodes and then you have a caps filter on then 
to you effectively sub to the things you want to see, if you do that for 
muc2 then you would have an explicit join trigger something similiar
     kev: we do the rev for muc2 because in muc2 we tie it to a specific 
account and if you use your server to act as a pubsub proxy (or whatever)
     ralphm: your server will always get everything even if none of the 
clients require everything?
     kev: it will always get all of the defaults unless you explicity 
say otherwise, if you have a client ??? your other client in the mam 
archive is always on the client then the other client ???
     ralphm: so assumption is that MAM is only local?
     kev: the possibility is that you can have both local and remote
     ralphm: in that case you will always get the history
     dave: I'm with ralph, I think we'll need some equivalent of +notify 
- I subscribe to everything but here's what I want
     kev: this does move greatly away from the pubsub acct model
     dave: no, we have to solve this even for the pubsub acct model 
anyways, if you come online for the first time your pubsub acct service 
will want to sub to all possible nodes
     kev: its a diff problem in a sense, I think I agree with the 
message your saying but I don't think it's fully expressed - what you 
need to solve is a way for your server for your account is a way to sub 
to a remote service it's that your acct, be it pep or a muc2, needs to 
know which subs you want and then receive the super set of what you want
     dave: ah, which isn't waht I said, we have to make sure that we are 
able to sub to a subset of nodes on a remote service, like what it works 
now but in a sensible way
     kev: the muc2 proto xep says you can sub to individual nodes, but 
if you do a default sub to a root node you will get all of the nodes 
that are defined in muc2 and if you want to their will be a way to 
limit/filter those nodes
     dave: we need a way to say I sub to muc2 and *these* kind of nodes 
are what I want to see
     kev: ProtoXEP says "Send pubsub subscriptionish stanza to the room. 
By default subscribes to all the 'standard' nodes, but can specify just 
those required. Server injects a new item into #occupants automatically."
     dave: with that caveat that what exists is nodes that what exist 
now or are created in the future, we may not have an elephant node but 
if we want to create one then that the clients that want to see the 
elephant node then they will get sub'd to that node
     waqas: to clarify, we have the root node and the default list of 
nodes, so what happens if the elephant node is added to the default set
     kev: no, the default set is defined by muc2 and does not change
     waqas: but what about custom nodes so what happens if we need a 
large image node and do we make it default and will it ever be a default
     kev: we don't make a default, if you want to sub to the large image 
node then you include that namespace in your sub request
     waqas: ok, so the defaults are static
     dave: we shouldn't have to have a default and that clients should 
specifty what streams they want to sub to
     kev: I don't see the harm and it seems more elegant, but I don't 
see a compelling reason why we would not make it explicit each time
     lance: the big thing would be indicating that you don't want a sub 
to one of the default node
     dave: what worries me about having a default, then how in years to 
come how can we respond to the question "how could not have known how 
awesome an elephant node was", it will make it harder to introduce new 
best practices nodes
     kev: that's fine
     peter: what to focus on next?
     kev: please make mental implementations :-)
     kev: do we have a way to subscribe to multiple nodes simultaneously?
     dave: the answer is no and we do need to address that, there is one 
other thing, give a muc2 domain you can then try to discover the muc1 
domain - he would like that to be the same so that a muc2 implementation 
can just provide the muc1 domain
     kev: I'm proposing we don't do that in the spec - I would rather we 
spelled out that it's not necessary and have clients able to do the 
fallback. It's easier to make these logically separate, mostly for the 
disco cases. If an implementation figures out to do that, more power to 
them.
     dave: ok, so the counter argument is: here is the address of a 
room, presumably that is the address of a muc2 room and then either 
their client speaks muc2 so that it gets that and a muc1 client doesn't 
even know how to use that muc2 address, the reverse point solution would 
be to always give out the muc1 address
     kev: assuming that a stable address is required
     lance: I would implement this by doing muc1 join to the service and 
receive a presence error with a redirect
     kev: yes, that would work for the case where you are giving out 
addresses directly but currently no clients are following redirects
     dave: don't depend on the fact that an older muc1 client would know 
how to handle a new muc2 related protocol to work properly
     kev: not really - your case was not disco, but you're given an 
address of a room
     dave: room discovery principles are fine
     kev: if you're given muc2 address, room could send message to 
client saying "here's a muc1 room over here" (or an invite)
     dave: what I would like to see is if they try and join with a muc1 
address that they get a subscription tot he muc2 room with a lifetime 
presence
     kev: if you want to implement that clever way, you are entirely 
free to do that, but he wants it to be considered a failure???
     kev: unless we hear otherwise from server devs, I'd prefer to keep 
them separate

     dave: so the case that i'm seeing is the disco one, I suppose that 
we could start off assuming that it's impossible and then we ned up with 
a crappy client exp, rather let's enumerate the points where it won't 
work and if we can address them as solvable
     waqas: the way I see it is, the protocols don't overlap at all so I 
don't see a need for separate domains

     dave: I agreee with everything you say but the point of overlap, we 
know that there is overlap with disco, but what we don't know is how 
many clients use disco for
     dave: for the domain is easy, ???

     peter: it works for both of the protocols and no one cares about 
what we put into the ???
     dave: only risk is difference between getting JIDs and getting 
pubsub nodes back from the disco query to the room

     waqas: do the clients do anything with the response other than 
displaying it
     dave: muc2 we could be saying that this subset of the protocol will 
work

     sam: I don't think we ever made guaratees that disco would only 
return certain information

     waqas: we kind of do have to deal with backward compat issues
     lance: for occupants, that's supposed to be under its own node
     peter: right, there's a special occupants node that tells you who 
is here
     lance: it's not just things hanging off of the node

     kev: can he request that someone with an easily hackable server 
with an easy hackable language do this and then discover how a client 
handles it
     waqas: I can trivially do that

     <-- lunch break -->

     related topic: mentions - out of scope for now, might need a 
separate spec

     Waqas and Matthew talked about how to do this using existing 
mod_muc and it appears to map well

     Radical change of topic...

     MAM & Carbons

     kev: I thought the spec was in shape for moving to Draft
     matt: there are things that are essential that are not in the 
revision, namely the ?? itself is not in there
     kev: it's kind of in there
     matt: that was sneaky
     kev: it also says that there may be future profiles of carbon that 
will say exactly should be carbon'd so I think that mamsub will be come 
a profile of carbons including messages back to yourself
     matt: I didn't get that would be the intention of that
     dave: i really don't want carbons to yourself
     matt: ??
     dave: doesn't want that by default at all, if it's negotiated 
that's fine
     matt: why
     dave: because it takes us down an alley we don't want to visit
     matt: ??
     dave: if someone wants to do additional negotiations that is fine 
but I don't want to introduce this at this stage of the draft, he would 
rather deal with this in another spec
     kev: do what
     dave: blind reflections

     lance: speaking of reflections, is it ok to have in muc2
     dave: that is kind of the way it's always been and also in pubsub, 
so yea your probably to get it back
     kev: I once chatted with a user community that they thought it was 
very useful to have echoing, say compared to irc, that your message is 
useful to have appear in the return stream
     dave: I don't mind keeping it in the places where it has been, but 
he doesn't want to introduce it to carbons or places where it isn't 
currently

     kev: I don't see why that is a problem, your already going to get 
reflections of other clients
     dave: but not your own, none of the existing carbon imple send it back
     lance: clients right now will show your messages twice because it 
won't know to igore
     matt: are you worried about this because of interop grounds
     dave: yes
     matt: that is a different argument and do you have a solution
     kev: the reservation to making this change for carbons was "here is 
what you should be doing but it was non-normative" because we didn't 
want to do the namespace change but that it was compatible to the old 
way by making a new namespace a requirement - also allowing us to do a 
future profile that allowed it by negotiation
     kev: I don't see why this is a problem because we left it up to 
server implementations to do it - any server that does it or chooses 
not, it will be not a useful implementation
     dave: yes, but if you do include reflects then that also change 
which category of clients get carbons especially if it's only some 
servers and handling of duplicates will have to be implemented
     kev: it seems that this may a bit of a surprise because of what 
carbons xep now say
     dave: ??
     kev: what is the problem in the current xep
     dave: because people can do reflect in the current xep
     kev: what do you think the problem is in the current xep? it allows 
reflection if negotiated
     dave: I'm not blocking current negotiations as long as it's not by 
default - any reflection *must* be by negotiations
     matt miller: the spec does not discuss that
     kev: nothing says you may not or may, it's just not mentioned
     dave: ok, then lets just say you don't do reflections on outgoing 
messages
     matt: but that defeats the use case of doing it for mam
     kev: then we should make the current rules even more clear
     dave: if the imple turns out to be rubbish then ...
     kev: ???
     waqas: I agree with Dave in the sense that this is how implementers 
have done things now

     kev: that is why we updated the xep to follow what some imple did
     waqas: with the namespace bump?
     kev: without
     waqas: ??
     matthew: I have a proposal (other is use of hints) - bite the 
bullet, fix things, and see what happens

     kev: if we are bumping the namespace then we may as well do that 
for mam sync, we didn't because we already had carbon deployed
     matthew: my argument is that existing impls can relatively easily 
update, whereas implementing a new protocol is a lot more work
     kev: there is no need to have a new namespace or anything mamsub or 
mamsub by carbons is ??? if we are updating carbons we may as well 
update everything that is needed
     waqas: I like that idea
     matthew: it's just carbons and MAM all the way
     kev: thats' right it just means that instead of leaving it for the 
current version we say instead we say explitly what stanzas get
     matthew: it sounds like Dave would like that
     dave: no, I just want to say no reflect without negotiations
     matthew: I'm assuming this would include negotiation
     dave: all I want to do is say no reflect without nego and then ship 
carbons and if we want another spec that says that we want to nego then 
that's a whole other spec
     matthew: I think that mam sub is irrelevant - what do you think 
about bumping the namespace
     dave: I think we're one line of text from shipping this - let's do 
that and figure out reflection, MAM-sub, hinting, etc. - I suspect 
that's a large bit of work
     matthew it sounds like it's no longer one of text because it sounds 
like you disagree with the revision about non-nego reflection change
     dave: yes, I'm saying you add one line that ways "no reflection 
without negotiation" - we can safely add that and we're done
     matthew: so you want to leave carbons as is, the changes to carbon 
are pretty small and they ...

     kev: it sounds like we have sufficient disagreement and that we go 
back to our prev state
     matthew: this is really annoying lots of people that we have't been 
able to solve this simple problem

     dave: i'm not blocking any further work, i'm just saying that we 
have something that with a single line that it will document existing 
work precisely
     matthew: carbons is only half of the problem that people are trying 
to solve, but we've still got a lot of things still needing to be solved
     dave: that's fine and carbons doesn't solve video chats and conferences
     kev: the point of going to draft is not saying we need incremental 
additions
     dave: no we are saying that this protocol is stable, and if you 
want to add another spec to tighten it down we can, not a problem
     matthew: if we tie up this stuff now, people will define something 
else and we could have an entirely different carbons replacement spec 
(Florian's message routing protocol)
     dave: I don't understand why if we do carbons now and if we have 
another xep that adds the nego part about reflects
     matthew: that would be carbons2, I don't see how that would be 
neatly way
     kev: another way, what would be the result of not pushing carbons 
in the current state to draft
     dave: I think we can push carbons to draft in another week
     peter: matthew's point is that people want mam+carbons and this 
doesn't solve that problem
     matthew: carbons + mam is part of one problem, they need to work 
alongside each other (MAM doesn't solve the offline problem to send 
everything to all devices)
     kev: the only tweak we need to make is to apply that they ???
     matthew: I'm assuming Dave is also against hints in carbons because 
that's a further modification
     dave: the discussion on the list ended up that we should do hints 
because it's own xep and carbons private thing that seemed to be the 
consensus
     matthew: I don't know that they mean slightly different things - at 
the end of the day, all that's needed is the ability to turn carbons off 
for a particular message
     dave: but private controls the senders end but the no copy hint 
controls the other end
     matthew: I was not aware that <private/> controls the sender's end
     kev: that is not quite the only that is needed because that is ??? 
because your allowed to hide parts of your archive from your client
     matt miller: I kind of agree with Dave, but it does solve a fair 
amount - shipping carbons seems like a good idea and handling 
incrementals is something we've done many times
     kev: is that saying that moving carbons to draft without 
specifiying what the routing rules are and despite knowing what the 
routing rules are and then adding them to a later change to just add the 
routing bits we know are
     matt miller: I was under the impression that the PR documented what 
impls are doing today but leaving open future flexibility
     kev: if we did make it more specific that would imply that we have 
to bump the namespace
     matt miller: do we have to bump the namespace if it's what people 
are doing now?
     kev: no in that case there would be a strong argument to not bump 
it but the changes we're talking about are not what impls are doing 
right now
     kev: and the reasons to want to specify it further is that for as 
long as your client is online you will have a copy of everything and if 
your offline you have to just do a reseycn and then your update carbons 
and your archive is accurate
     matt miller: I suggest that we ship and fix with increment or replace
     peter: "ship and iterate"
     lance: for the record, I agree with dave with that at the moment if 
we say it is ok for a server to start sending reflections and that will 
break existing clients, the enabling carbons is negotiated so we can now 
insert a new item to say "allow reflects" and that is all that we should 
have to need
     peter: if we change carbons to "selfie" then everyone will 
implement it right away
     kev: I'm about 4 or 5 on the "F" scale
     dave: he is quite a large number of Fs to ship
     matthew: i'm the same that we ship something now that works
     dave: so we have been talking about it for 18 months so...
     matthew: I think we're two small changes away
     peter: I agree that we need to solve the full problem
     matthew: there are overlapping issues here
     dave: I've already said that he wants to ship what we already have 
and then fix it later
     matthew: do you just mean advance this to Draft?
     dave: yes advance to draft
     matthew: what does that accomplish?
     dave: it means that as an org we can recommend that it can be 
implemented
     matthew: ???
     kev: my reading of XEP-0001 is that we shouldn't advance to Draft 
if we expect any change that would require a namespace bump
     matthew: ok, that's fine and that leads to the questiohn that if we 
make the changes do we need to make a namespace bump even if adding an 
attribute
     dave: why would that need a namespace bump?
     matthew: then I need may have been a misreading of a mailing list post
     sam: I think existing clients would get the same behaviour so would 
not require a namespace
     matthew: then we add in this flag into the enable attribute to say 
that reflection is on or off (default off) and that means we don't need 
to bump namespace
     dave: I don't think we want to just add negotiation because you're 
going to shoot yourselves in your foot because then you want an archival ID
     dave: that's fine then you can specify all of this in another xep
     matthew: is that a problem/part of carbon
     dave:
     sam: I'm wondering if we are making this a bigger deal than it is, 
why does it matter if selfie negotiation is in another spec?
     kev: issue isn't that, but that if we move Carbons to Draft means 
we think it's ready
     sam: if we go to draft with the existing text, if we go to draft 
today, then we would want to do that knowing that we wouldn't make 
anymore changes
     peter: we are always making changes, but we don't move to draft if 
we *know* there are changes that will be coming, we always are making 
small changes
     lance: if we take the mam spec and we say if you are going to be 
using carbons and mam, ...
     matthew: then that would be a new xep, let's take the parts that we 
are not having consensus on and that would define what mam and carbons 
interoperate and move to a new xep
     lance: that seems OK because we want to include things like 
including reflected IDs
     matthew: i'll be modestly happy that it seems we have acheived some 
kind of consensus with that new xep
     lance: so no selfies by default, handled via other spec
     kev: we add another line saying that we think other specs will be 
needed to specify interaction with archiving
     matthew: that is what i'm trying to avoid - linking them, even if 
mam didn't exist reflecting your id and ?? is useful and the reason we 
didn't do it sooner is because it was already implemented
     dave: and you need an archive ID as well
     matthew: we already have that spec'd out
     dave: there is no need to say this spec will change
     peter: all that kev was saying was that we should add one line 
saying "watch this space"
     matthew: we don't need that as long as we can say "these rules may 
change later"
     peter: sure
     kev: what I really think we *want* clients to implement is a 
solution that works with archiving, fast MAM resync with IDs, etc. - I 
think that's uncontentious. Is there a danger that when we say "carbons 
is great" we cause trouble for ourselves down the road when later one we 
say "you need something else"?
     peter: its not something else, but something in addition
     matthew: any implementation of it will be next to something, it 
could be even better after we figure out what it is really
     kev: for the client side, i'm fairly convinced it doesn't make it 
even worse for mamsub, but for the server side i'm less convinced that 
there are any benefits for having an existing carbons ... what you meant 
to do was to ... other than having a hash defined for your namespace - 
concerned to have the servers get asked to do the work that they would 
have to reimplement it
     dave: the carbons spec as-is is a tiny amount of code on the server
     matthew: I think this is a minor concern
     kev: it may not be a lot of code, but the tests used to make sure 
you don't have regressions and that must be complicated/large
     dave: nobody tests servers
     kev: *thanks dave for giving him something to fix legacy wise*
     kev: by and large it's a moot point because he isn't going to have 
a vote when it does arrive to the council

     [ random discussion about compliance suites and such]

     kev: we have to make sure that no-store is ignore in MAM-sub
     dave: no-store and no-copy are equivalent
     matthew: you want no-store to be ingored because ???
     dave: no-store and no-copy are equivalent, because ??? - 
whatever you copy, if it's for the puposes of mam sync it has to be in mam
     kev: if it's in the archive you have to have a copy, howerver if 
you have a copy and you know it's not in your archive ??
     kev: no-store is not the problem, but no-copy is ...
     kev: you would never get that message is that you ??

     kev: we should just do away with no-copy, when your doing mam sync ...
     matthew: I'm fine with that - but why is <private/> in carbons anyway?
     sam: for things like OTR or IBB file transfers
     kev: no <body/> for IBB
     dave: and they wouldn't be in the archive?
     kev: if it wasn't in your archive you would expect to go to all 
copies if you were doing mamsub
     matthew: currently spec says <private/> applies to sending server 
and receiving server
     dave: I misread either the xep or the message ... but then yes...

     waqas: if we do push the XEPs as-is, does that help us get more 
implementations?
     sam: Pidgin for example has had a patch waiting for awhile
     waqas: does the xsf want them to implement what they currently have 
or the version that will be going to draft
     sam: I mean if I you know i'm a user using pidgin, and pidgin says 
they are not going to implement it because it's exxperimental..
     peter: we should find out what the hold up is - does not seem to be 
the standardization status of the specs


     <-- new topic -->

     kev: I don't think that our current read-markers stuff is the right 
solution - when we come to MAM + carbons we need this. And I am the only 
person who sometimes wishes to mark a chat message as unread?
     lance: I would prefer to just have a way to arbitrarily mark 
message ;-)

     [scribe missed a bunch of conversation here]

     dave: I think the server can keep a count
     kev: I'm proposing that the client can tell the server "this is the 
last message I received", which we can do with carbons + message IDs, 
have a PEP node that you put items into?
     waqas: then you need to query data when you log in
     dave: don't need an item per conversation, only an item per 
conversation with unread messages
     kev: if also store last message that any client received, then you 
know from that point onward everything is unread
     dave: I think that works
     waqas: so need to update that every time you receive a message
     kev: doesn't need to be on every message
     waqas: at that point it's kind of like a cross-client XEP-0198
     dave: but we're not dealing with order etc.
     waqas: as part of normal history sync it gets messages from MAM
     kev: you log in, ask what's in the node, node says unread since 
this point, client clears that once messages are read
     sam: I like this and it's consistent with our existing implementation
     kev: it gets better! if you do this and you do the global 
last-received ID, client can just request messages since any client was 
last online, that means resync is only 5 minutes of messages if you 
haven't been on your mobile in a month but you've been online at your 
desktop until just now

     lance: at lunch we talked about syncing recent conversations, not 
just messages - I think this will probably let us do that
     kev: sadly, no - however, I think we could define a way to query 
your MAM archive for JIDs with which you've had conversation

     waqas: it'd be nice to get the list of open tabs
     sam: couldn't we do that with bookmarks?
     kev: we could do this with bookmarks but the UX I think people want 
is something close to what Lance describes - I'm wondering if we solve 
Lance's problem (who was I last talking to?) would that be enough, but 
do we really need to sync tab-open state across clients?
     lance: not just tabs, but conversations - if we did bookmarks over 
PEP then I think we have what we need, but we'd need private PEP nodes
     waqas: how about chats with people not in your roster?
     lance: other clients would receive indication that the conversation 
is retracted and not show it

     <-- Push / XEP-0357 -->

     lance: it's done, we just need implementations
     waqas: when do we need or expect to send push notifications?
     lance: servers push to backend services, which push to clients
     kev: client wants to receive a push if it's offline
     waqas: but even if it's just in the background
     kev: basically push all the time
     sam: doesn't that result in duplicates?
     waqas: can you suppress pushes on the client side?
     lance: yes, that's up to the client developer to figure out
     kev: one of my server developers said there was no point in client 
side indications, you only ever do push
     lance: you can't rely on push notifications or that it arrives in a 
timely manner (perhaps not even order)
     sam: CSI might help with other things (annoying presence or whatever)
     kev: my point wasn't "don't do CSI" but "don't disconnect completely"
     kev: how does this work on a server that's on an XMPP server off 
the 'net?
     waqas: I don't think so
     kev: in that case CSI would be more important







More information about the Summit mailing list