[Standards] Council on Stanza Repeaters without Multicast

Dave Cridland dave at cridland.net
Fri Apr 4 09:35:06 UTC 2008


On Thu Apr  3 22:40:48 2008, Alexander Gnauck wrote:
> Carlo v. Loesch schrieb:
>> Yes, that's why I started thinking about something better in 1990.
>> I understood there was no way for IRC to be 'fixed' because its  
>> design
>> is fundamentally flawed. XMPP is only syntactically flawed, which  
>> is
>> a much better starting point. And you can argue that's just my  
>> opinion.
>> That's ok. Still IRC does multicast, and XMPP is still missing  
>> that,
>> and Pedro is the kind of person who can see the impact of that  
>> each day.
> 
> I also see this impact.

Sure, I think everyone here does. I sincerely hope so, anyway - we  
will have significant impact here in the future.

>  With the situation we have *today* I agree with Dave that we don't  
> save much traffic and stanzas with repeaters.

That's not what I've said. I've said that we do, indeed, save stanzas  
- well, above a list size of two, anyway, as long as it's stable. But  
I've said that we do not know whether we save bandwidth. Because we  
don't.

I've also tried to make it clear that even if we do save bandwidth,  
I'm not convinced that this is the best way to do so.

>  There is nearly no PubSub usage at all today, and in the most Muc  
> rooms we have only 5-30 participants. If I look at the participants  
> of jdev today than the most users have local jabber.org Jids, and  
> max. 2 or 3 participants are on the same federated server. People  
> join and leave, so there will be a tie between the stanzas we save  
> in the repeater and the overhead to maintain the distribution list  
> in the repeater.
> 
> 
Right, so you're in agreement that this is not a problem which is  
do-or-die for us, yet. So really, we have time to get the solution  
right, rather than blindly rush in. And it's an important area for  
us, I entirely agree. So let's not screw it up by grasping at the  
first (or, in this case, around the fourth or so) idea to come in the  
door - we did that with XEP-0033, which has left us with an  
interesting and utterly pointless XEP, whose primarily purpose is to  
point to and say "Don't do that". This is not firefighting, there is  
little pressure as yet, so let's take our time and find quality  
solutions to these very real problems.

You do, however, raise a very interesting point - you've said that in  
the current MUC case, there's no gain in using repeaters.

One of my biggest problems with repeaters is that it's really not  
clear how a server (or service) decides to use them. What *is* the  
trade-off?


> But we have to look forward.

Sure... I'm fine with looking forward.

>  We all want to see the killer pubsub application with 100K  
> subscribers and more.

Yes, and as Ralph and Pedro have both stated, this is essentially  
practical now by having repeater-pubsub-nodes. We can clean this up  
by passing through subscriptions, and by having clients redirected to  
their local pubsub node, but these are comparitively simple problems  
to solve.

The result of solving them would be that PubSub would almost  
spontaneously grow directed acyclic routing, and this would be  
virtually transparent to clients, and to a large degree, transparent  
to servers. Note that this is more powerful than repeaters, as well  
as simpler.

>  We also want people to switch from IRC to Muc and see tons of Muc  
> rooms with 100+ participants.

Yes, and we discussed methods for doing this in Brussels, essentially  
similar to the above - a MUC room as a participant, loosely - which  
is slightly more difficult than the above, but generally seems quite  
solid in terms of design. (And, still immune to the majority of  
problems plaguing IRC).

Again, we gain a lot from this. It's a simple design, and one that  
can potentially get us some resilience to failure, which is nice. And  
again, not my design, and not a particularly new suggestion. And  
again, it gives directed acyclic routing to MUC, which repeaters  
don't do. (Well, I hope not).

>  And maybe we define other 1->n protocols in the future. 

No, I doubt that we will. Fundamentally, we have presence itself,  
MUC, and PubSub. I'll count PEP as Presence here for our terms. I  
think creating an entirely new application of XMPP in addition which  
isn't satisfied by any of those would be in error.

If we solve scalability in MUC and PubSub - and I think we can  
*without* repeaters, and I think that repeaters are not well  
understood, and may cause greater harm than good - then I think this  
is way beyond good enough. I believe that for both MUC and PubSub,  
using a slave entity which passes on subscriptions to a master  
entity, and manages redistribution backwards, will yield equal or  
high efficiency, with much simpler implementation, fewer security  
issues (in implementation or design), and will give us all the  
benefits of a tree-based heirarchical distribution without breaking  
any trust models.

I don't think we'll have a problem scalaing presence, because I don't  
think it really needs to be scaled - it's neither low-hanging fruit,  
nor a pressing problem, and while I'll agree that in two cases - PSA  
and bots - it might be problematic, I'm hoping that bots move toward  
pubsub and away from PEP, and likewise users with rosters the size of  
PSA remain a rarity.

It's certainly reasonable that we'd save a few stanzas by doing  
something similar to repeaters or smart presence, but I think we'd be  
saving perhaps 5 stanzas per presence burst, as opposed to several  
thousand, potentially, per MUC or PubSub event. That's not  
insignificant, but I'm far from convinced it's worth doing for that  
sake alone.

> In the long run we will benefit a lot from repeaters.
> 
> 
Maybe. But I'm far from convinced, and I'm concerned that a strategy  
which attempts to convolve MUC, PubSub, and Presence scalability  
without recourse to the model itself is quite likely to do more harm  
than good, for a number of reasons.

Not least of which is that, if this went ahead, we'd probably feel  
under some obligation to use it in a MUC and/or PubSub scalability  
effort, and I happen to think it's entirely the wrong design for that  
- there are much cleaner designs.

Incidentally, repeaters are simply a slightly more explicit, and  
generic, model of smart presence, and the (then-) council dropped  
that. The biggest difference is that Repeaters are proposed by people  
well-known to the community, and that's not a particular useful  
technical argument to me. (It does mean that there's a greater chance  
that the proposal fits the XMPP model, as in this case, but it  
doesn't make it automatically technically better.)


> Compression is great and saves lots of bandwidth. Compression +  
> Repeaters is even better and will save more bandwidth and stanzas.

Ah... That's something that has yet to be proven either way.

We know that compression+XEP-0033, for example, yields fewer stanzas  
but greater bandwidth than compression alone. We know that "Smart  
Presence" yields both fewer stanzas and less bandwidth, but fails to  
match our model - I don't think it matches the model in its last  
incarnation, and there's way too much handwaving in that protocol.

My gut feeling is that repeaters will only save bandwidth over a long  
period with very few list changes, probably particularly in an  
environment which is itself high traffic. I think there are  
considerable resource management issues inherent, and I'm not  
convinced that people are entirely thinking this through in terms of  
processing saved.

Dave.
-- 
Dave Cridland - mailto:dave at cridland.net - xmpp:dwd at jabber.org
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade



More information about the Standards mailing list