[Standards] Council on Stanza Repeaters without Multicast
dave at cridland.net
Fri Apr 4 09:35:06 UTC 2008
On Thu Apr 3 22:40:48 2008, Alexander Gnauck wrote:
> Carlo v. Loesch schrieb:
>> Yes, that's why I started thinking about something better in 1990.
>> I understood there was no way for IRC to be 'fixed' because its
>> is fundamentally flawed. XMPP is only syntactically flawed, which
>> a much better starting point. And you can argue that's just my
>> That's ok. Still IRC does multicast, and XMPP is still missing
>> and Pedro is the kind of person who can see the impact of that
>> each day.
> I also see this impact.
Sure, I think everyone here does. I sincerely hope so, anyway - we
will have significant impact here in the future.
> With the situation we have *today* I agree with Dave that we don't
> save much traffic and stanzas with repeaters.
That's not what I've said. I've said that we do, indeed, save stanzas
- well, above a list size of two, anyway, as long as it's stable. But
I've said that we do not know whether we save bandwidth. Because we
I've also tried to make it clear that even if we do save bandwidth,
I'm not convinced that this is the best way to do so.
> There is nearly no PubSub usage at all today, and in the most Muc
> rooms we have only 5-30 participants. If I look at the participants
> of jdev today than the most users have local jabber.org Jids, and
> max. 2 or 3 participants are on the same federated server. People
> join and leave, so there will be a tie between the stanzas we save
> in the repeater and the overhead to maintain the distribution list
> in the repeater.
Right, so you're in agreement that this is not a problem which is
do-or-die for us, yet. So really, we have time to get the solution
right, rather than blindly rush in. And it's an important area for
us, I entirely agree. So let's not screw it up by grasping at the
first (or, in this case, around the fourth or so) idea to come in the
door - we did that with XEP-0033, which has left us with an
interesting and utterly pointless XEP, whose primarily purpose is to
point to and say "Don't do that". This is not firefighting, there is
little pressure as yet, so let's take our time and find quality
solutions to these very real problems.
You do, however, raise a very interesting point - you've said that in
the current MUC case, there's no gain in using repeaters.
One of my biggest problems with repeaters is that it's really not
clear how a server (or service) decides to use them. What *is* the
> But we have to look forward.
Sure... I'm fine with looking forward.
> We all want to see the killer pubsub application with 100K
> subscribers and more.
Yes, and as Ralph and Pedro have both stated, this is essentially
practical now by having repeater-pubsub-nodes. We can clean this up
by passing through subscriptions, and by having clients redirected to
their local pubsub node, but these are comparitively simple problems
The result of solving them would be that PubSub would almost
spontaneously grow directed acyclic routing, and this would be
virtually transparent to clients, and to a large degree, transparent
to servers. Note that this is more powerful than repeaters, as well
> We also want people to switch from IRC to Muc and see tons of Muc
> rooms with 100+ participants.
Yes, and we discussed methods for doing this in Brussels, essentially
similar to the above - a MUC room as a participant, loosely - which
is slightly more difficult than the above, but generally seems quite
solid in terms of design. (And, still immune to the majority of
problems plaguing IRC).
Again, we gain a lot from this. It's a simple design, and one that
can potentially get us some resilience to failure, which is nice. And
again, not my design, and not a particularly new suggestion. And
again, it gives directed acyclic routing to MUC, which repeaters
don't do. (Well, I hope not).
> And maybe we define other 1->n protocols in the future.
No, I doubt that we will. Fundamentally, we have presence itself,
MUC, and PubSub. I'll count PEP as Presence here for our terms. I
think creating an entirely new application of XMPP in addition which
isn't satisfied by any of those would be in error.
If we solve scalability in MUC and PubSub - and I think we can
*without* repeaters, and I think that repeaters are not well
understood, and may cause greater harm than good - then I think this
is way beyond good enough. I believe that for both MUC and PubSub,
using a slave entity which passes on subscriptions to a master
entity, and manages redistribution backwards, will yield equal or
high efficiency, with much simpler implementation, fewer security
issues (in implementation or design), and will give us all the
benefits of a tree-based heirarchical distribution without breaking
any trust models.
I don't think we'll have a problem scalaing presence, because I don't
think it really needs to be scaled - it's neither low-hanging fruit,
nor a pressing problem, and while I'll agree that in two cases - PSA
and bots - it might be problematic, I'm hoping that bots move toward
pubsub and away from PEP, and likewise users with rosters the size of
PSA remain a rarity.
It's certainly reasonable that we'd save a few stanzas by doing
something similar to repeaters or smart presence, but I think we'd be
saving perhaps 5 stanzas per presence burst, as opposed to several
thousand, potentially, per MUC or PubSub event. That's not
insignificant, but I'm far from convinced it's worth doing for that
> In the long run we will benefit a lot from repeaters.
Maybe. But I'm far from convinced, and I'm concerned that a strategy
which attempts to convolve MUC, PubSub, and Presence scalability
without recourse to the model itself is quite likely to do more harm
than good, for a number of reasons.
Not least of which is that, if this went ahead, we'd probably feel
under some obligation to use it in a MUC and/or PubSub scalability
effort, and I happen to think it's entirely the wrong design for that
- there are much cleaner designs.
Incidentally, repeaters are simply a slightly more explicit, and
generic, model of smart presence, and the (then-) council dropped
that. The biggest difference is that Repeaters are proposed by people
well-known to the community, and that's not a particular useful
technical argument to me. (It does mean that there's a greater chance
that the proposal fits the XMPP model, as in this case, but it
doesn't make it automatically technically better.)
> Compression is great and saves lots of bandwidth. Compression +
> Repeaters is even better and will save more bandwidth and stanzas.
Ah... That's something that has yet to be proven either way.
We know that compression+XEP-0033, for example, yields fewer stanzas
but greater bandwidth than compression alone. We know that "Smart
Presence" yields both fewer stanzas and less bandwidth, but fails to
match our model - I don't think it matches the model in its last
incarnation, and there's way too much handwaving in that protocol.
My gut feeling is that repeaters will only save bandwidth over a long
period with very few list changes, probably particularly in an
environment which is itself high traffic. I think there are
considerable resource management issues inherent, and I'm not
convinced that people are entirely thinking this through in terms of
Dave Cridland - mailto:dave at cridland.net - xmpp:dwd at jabber.org
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade
More information about the Standards