[Standards] distributed MUC rooms

[Mridul Muralidharan]

Peter Saint-Andre wrote:
> Mridul Muralidharan wrote:
>> Peter Saint-Andre wrote:
>>> Last October I talked with some folks in the XMPP community about the 
>>> idea of distributing MUC rooms across domains. Late this afternoon 
>>> and this evening I finally got around to writing up an approach to 
>>> this problem. The document I've started working on is quite 
>>> tentative, but I figured I'd send it out for feedback anyway (release 
>>> early, release often!).
>>>
>>> http://www.xmpp.org/extensions/inbox/distributedmuc.html
>>>
>>> I'll try to work on it more this weekend, since I'd like to present 
>>> some of the ideas at a conference I'm attending next week.
>>>
>>> Peter
>>>
>>
>> A few thoughts in general.
>>
>> 1) What is the scope of the distribution - is it going to be an muc 
>> service spread across domains ? Or is it going to be a single logical 
>> muc service (hosted on multiple machines/nodes) ? (With reference to 
>> 1.3.3)
> 
> Multiple domains.


"<message from='room at muc.domain1/nick1' to='room at muc.domain2/nick2'> 
<some /> <xml /> </message>"
(replace message with any of the other stanzas too).

In this case, multiple hops through both rooms ?
Or will muc.domain1 know how to resolve 'room at muc.domain2/nick2' to the 
actual user jid ? Even in an anonymous room ?


> 
>>  From examples 5 onwards, it looks like the former case ... might get 
>> a bit tricky. Why would a user join muc.example.net for a conference 
>> on chat.example.com ?
> 
> The conference is (say) "topside" and the conference is the same logical 
> conference whether you join on muc.example.net or chat.example.com. Or 
> at least that is the idea.


It would be quite interesting to see what happens when a room gets 
switched on to distributed and there are other rooms already distributed 
with same name or locally present with same name - in the remote 
domain(s) I mean.

> 
>> Wont this not be a potential routing issue ?
>> I assumed we will be using something like dix[1] for this - where bare 
>> jid is the same, just that it is distributed and fault tolerant. (or 
>> integrated into the cluster/pool'ed server).
> 
> IMHO clustering or pooling is trying to solve a different problem.

If intention is just high availability of rooms, it can also be solved 
by restricting to a single domain.
But if the xmpp domain goes down, then yes we could lose the room - and 
thinking more about the usecase you mentioned, it would not be 
performance reasons which guide this decision.

> 
>> This could also become potentially confusing for clients - where user 
>> of a conference is receiving messages from multiple room jids.
> 
> Maybe -- I'm not sure what you mean. If I join topside at muc.example.net 
> then as far as I can see, every occupant is in that room. The s2s aspect 
> of it is hidden from the user.

I said this in assumption that participants in a room hosted on remote 
domain will get shown with the muc jid's with their domain (not as jid's 
on local muc). Makes sense now.

> 
>> 2) After creation, do we need to maintain the distinction between 
>> FIRSTHOST and PEERHOST ? Merging of config can be based on 'timestamp' 
>> for example (use ntp on all nodes to keep them in sync) ? This way, 
>> FIRSTHOST could be down, or not participating in the conference which 
>> was created on it ... and still things would continue working.
>> With reference to 1.3.4
> 
> Possibly, yes. I think it's a requirement that any and all peerhosts can 
> continue operating even if the firsthost is not available temporarily or 
> goes down permanently. But it's not yet clear to me how the all the 
> peerhosts figure out what the canonical room configuration is. In a way, 
> the firsthost is like the room owner and the peerhosts are like room 
> admins (across domains). Right now we don't have a good way of 
> proceeding if in a non-distributed room the room owner disappears from 
> the scene (but for non-distributed rooms we at least have a service-wide 
> MUC admin). So it seems we almost want a way to have net-wide admins who 
> can administer the network of peerhosts in some way. I have not given a 
> lot of thought to these issues yet.

imo this would be something which we might need to be sort out.
FIRSTSERVER need not be around. So is there necessity to special case it.
Also, if there are no participants in the room in a domain, should it 
receive messages from other peer rooms ?

> 
>> 3) After example 4, can we add the peerhost as a hidden config param 
>> to the room config (firsthost gets auto-added if room is distributed) ?
>> This way, recovery would be easier in case of restart/etc - and 
>> peerhost will know 'who all' to talk to for a room.
>> Though in light of (1), I am not very sure of this - if it is a single 
>> logical entity, it will talk to 'all' instances it discovers ... so 
>> wont be necessary.
> 
> Yes, I think that peering is something that is set up at the service 
> level. So muc.example.net knows that it peers with chat.example.com and 
> conference.example.org. If a room is distributed, then the service knows 
> which peers it must sync with. So I don't think we need a room-level 
> param for this.

Can this be overridden/customized at the room level ?

> 
>> 4) Section 3.4 - is it that the peer room which received the message 
>> supposed to be multicast to other peer rooms - which will inturn 
>> multicasts to their hosted participants ? 
> 
> Yes.
> 
>> Or is it that the recipients room multicast directly to participants 
>> of the remote rooms ?
> 
> No.
> 
>> Would prefer the former - if latter, use xep 33 ?
>> I was not sure which approach this xep is taking (I could have missed 
>> it).
> 
> Perhaps I didn't make it clear yet.
> 
>> We have been supporting distributed conferences for a while now, and 
>> there are some interesting routing challenges - even though in our 
>> case it is directly integrated into the server.
>> It would be great to see a spec which allow remote components to also 
>> become distributed and provide highly available conference service !
> 
> Do you mean in general (i.e., a generalized spec for service 
> distribution)? Or just for MUC?

We distribute services within the server pool - and so servicing a set 
of hosted domains. Currently, they are limited to the services the 
server offers off the shelf - muc, pubsub, jud, etc. The motivation is 
mainly from a performance angle though scalability and high availability 
are as important.

The usecases targeted in this spec though seem to be for high 
availability in face of unreliablity (i think) ... not something we were 
designing for : but does make distribution quite interesting :)


Regards,
Mridul

> 
> Peter
>