[Standards-JIG] proto-JEP: Smart Presence Distribution

[Dave Cridland]

On Thu May 18 00:09:26 2006, Pedro Melo wrote:
> On May 17, 2006, at 11:44 PM, Dave Cridland wrote:
>> On Wed May 17 20:39:35 2006, Carlo v. Loesch wrote:
>>> compression is already being used, and it doesn't save you.
>>> again, see last month's discussion.
>> 
>> I'll take you to task on this one. I'd personally be quite  
>> astonished if the existing multiple presence stanzas, being  
>> presumably compressed into a single deflate block, didn't compress 
>>  very well indeed.
> 
> Of course they will. But a single presence would compress even  
> better, right?
> 
> 
A block of similar presence stanzas will compress better, but it may 
still be larger after compression than a single stanza was 
beforehand, or is after compression. Of course, this is irrelevant, 
the key thing is that in the case where s2s bandwidth is constrained, 
one assumes that you'd spend the extra cycles on compression before 
this protocol, because it's almost certainly cheaper and will save 
more octets on the wire.

Assuming that, does the benefit outweigh the cost?


> It's pure math: the amount of information to begin with is lower in 
>  this proto-jep, so no matter what you do, compress or not, this 
> proto- jep will always be better.

It's not mathematics, it's information theory - good old Shannon. The 
information is this proto-jep is only marginally lower, and across 
the long term is equal. In fact, you've argued this before in the 
thread with Tijl, so I'm a little surprised you're arguing the 
opposite now.

The amount of redundancy is much higher without it, however - when a 
user comes online, or drops offline, there's a significant redundant 
flow. But compression is simply a technique of encoding to reduce 
redundancy - you can view this proto-JEP as a application specific 
compression mechanism, just like multicast can be viewed that way. 
(There's better examples, like the set-syntax in IMAP, for instance, 
that stretch credulity less).

Information theory says, therefore, that with a perfect compression 
mechanism, both will be equal, but that requires a large - probably 
infinite - amount of processing.

So the question is not "Is this better than before", but "is this a 
better compression method than a general compression method, and, if 
so, is it sufficiently simple to implement that the increased cost 
outweighs benefit".

Incidentally, if this were a mechanism which reduced c2s bandwidth 
instead of, or in addition to, s2s, I'd be much more willing to 
support a higher cost/benefit trade-off, but s2s bandwidth is 
generally considerably higher, and the flow rates are sufficiently 
high on higher bandwidth links to benefit more from compression and 
packet reduction.

It's not that I'm against streamlining presence distribution, it's 
more that I think there's better low hanging fruit to deal with first.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw