[Standards] Council on Stanza Repeaters without Multicast

Dave Cridland dave at cridland.net
Thu Apr 3 08:48:31 UTC 2008

On Thu Apr  3 08:23:36 2008, Pedro Melo wrote:
>> A) Compression
>> It's crucial on two counts:
>> 1) It's much simpler to implement, and2) Given that we are (or   
>> should be) encrypting every S2S connection, then TLS is giving us   
>> compression anyway, and moreover, it's cheaper to compress than  
>> not  to compress.
>> We've been down the road before where on some admittedly fairly   
>> simple figures I did a while back - and repeated for Peter's   
>> scalability draft in the SIMPLE WG, I think - the existing  
>> presence  "splurges" compress really well, often better than the  
>> "optimized"  alternatives.
>> Smart Presence was one proposal that beat pure compression, as I   
>> recall, but it has security implications which I'm not going to   
>> accept.
> Bandwidth is not the only concern. If I have a very popular pubsub   
> topic (like a Atom node for some high-profile blog or social site)   
> with hundreds of thousands of subscribers, the cost of expanding in  
>  the source is very high, in terms of CPU also.
Well, yes and no - XML parsing appears to hit a lot of servers hard,  
but routing is routing, and repeaters will increase the cost there, I  
suspect. If you're having to expand routers, that expansion has to  
happen somewhere - it's a database lookup, or a filesystem read, or a  
hash table in memory - but it's certainly additional activity.

This cost might, in some implementations, be outweighed by the saving  
in XML parsing. I don't know, and I don't pretend to know, either.  
I'm pretty sure it isn't in our implementation.

> The mailing list people moved to sub-lists to deal with this.   
> Repeaters are just like sub-lists. Please note that I'm talking of   
> the *public* pubsub node use-case.
Right, almost. Repeaters are like sub-lists controlled by the sender  
- that's a quite strinking difference.

>> B) RTT delays
>> The introduction of Stanza Repeaters gives us RTT delays whenever  
>> a  list needs changing. That just strikes me as worrying - I don't  
>>  think anyone has clear figures on how much delay would be   
>> introduced, but I'm sure you'll agree that additional latency does  
>>  not a performance improvement make.
>> It may be possible to alter the protocol to remove these.
> This is important if and only if list changes are the most frequent  
>  operation.
> If sending messages through the repeaters is the most common   
> operation, then the lower RTT of distribution that Stanza Repeaters  
>  give us, will over-shadow the delays with list changing.
Right, kind of - for any given repeater, you need a list-change per  
event to make it happen every time. And I'm not sure where a suitable  
breakpoint is - I don't know how many stanza-fanouts outweigh an RTT.

I don't feel we know anything like enough about habits and metrics  
right now. It could be that even when we have an RTT delay due to a  
list rewrite on every event, it's not a big deal because the line may  
still be efficiently utilised, but introducing new RTT delays just  
doesn't sit right with me.

Now, I'm reasonably confident in suggesting that (without figures to  
back this up), the list-change to stanza-fanout ratio has to be  
pretty good before you beat simple compression.

I'm also of a much more nebulous gut feeling - assuming gut feelings  
*can* be nebulous - that for typical large public pubsub nodes -  
like, say, a popular blog - it's quite likely that the list will  
often change between publication events - which in turn suggests that  
this might be completely the wrong solution for that use-case.

> The cost of updating this list is exactly the same as a normal   
> subscribe in pubsub. A second user will only see steps 1, 2, and 5,  
>  same as a normal subscription. The local pubsub service might  
> signal  the remote pubsub service, but thats optional, can be done  
> out-of- sync with the user flow, and its only required if the  
> central node  wants to have knowledge of the number of subscribers  
> globally.
Right, or control them, in a relatively trusted environment.

A similar mechanism works for MUC, too - a sub-MUC would send  
subscriptions to the master-MUC, which would send only a single copy  
of the messages back to the sub-MUC for further routing. If a user  
tries to subscribe to the sub-MUC and the subscription is rejected by  
master-MUC, the rejection is passed back.

We talked about this in Brussels a bit, and I think that it's an  
interesting area to persue.

FWIW, pubsub and MUC - at least the public ones - are very much  
simpler than presence handling, making this style of fan-out much  
more practical. Indeed, having chains of sub-MUCs becomes practical,  
since either the MUC is public, or potentially a chain can be set up  
with bilateral agreements, which is effectively how IRC and PSYC  

> For presence distribution, I like remote fan-out based on a  
> reverse- roster better, but it doesn't interact properly with  
> privacy lists.  Stanza repeaters deal better with privacy lists,  
> because you can  update the remote list based on the privacy list  
> changes. But I don't  know if the complexity of all this  
> privacy-list interaction will be  worth it.

Right - I think it's just going to get exceedingly complex, very  
quickly, and the net result will be more bugs, and a failure to  
produce scalable specialized support for fan-out in PubSub/MUC.

I'd like to know more about the levels of data we'd save for presence  
fanout broadcasts, really, prior to going forward with this, but I  
have to admit I'd be surprised if this is worthwhile for just that.

> In the end, a generic stanza repeater implementation is something   
> useful to have in theory, but in practice all of the services that  
> it  is trying to replace already have the subscriber list inside,  
> and  they could leverage that to add multi-cast capabilities.


Dave Cridland - mailto:dave at cridland.net - xmpp:dwd at jabber.org
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

More information about the Standards mailing list