[Standards-JIG] proto-JEP: Smart Presence Distribution
richard at dobson-i.net
Wed May 31 08:58:44 UTC 2006
> Richard Dobson typeth:
> | Of course they will be used, the same rules that you currently use to
> | send out the individual broadcasted stanzas you will use for building up
> | the explicit list for sending the broadcasted stanzas, the only
> | difference is that the stanzas are fanned out at the remote server
> | rather than the local one, that is the one and only difference. If
> Yes, that's how our proto-JEP operates.
Exactly and is how what I suggested would operate, im glad we agree on
> Just look at examples 2, 3, 8 and 13 of the Presence Examples in 5.5.
> With your strategy in place, they all become outdated. I wouldn't say you
> are breaking the RFC, you are simply not using it anymore.
As stated on the paragraph at the top, you are still using all the
presence rules in the RFC, example 2 would still be used with both of
our suggestions as far as I can see, examples 3, 8 and 13 would work in
your proposal identically to how they would work in my suggestion, they
would both be being sent to single address and then fanned out.
> No we don't have to talk about Jabber sticking its head anywhere, we
> can simply start changing that. Don't tell me it is to hard to code some
> error handling into your server.
> If Jabber wants to move forward, it needs to change some attitudes.
> Conservatism is not going to help you solve the problems it has created.
> You don't design a network protocol without failure management.
Yes but we also cannot have protocols that assume something that is not
true, i.e. that the network is reliable when it isn't, this is just
something that you are going to have to accept.
> | But in your proposal there is no way for either server to know if the
> | list is out of sync, so they are forced to always repeatedly refresh it
> As long as there wasn't an error in transmissions, there is no reason to
> presume any data is lost. It's like presuming the sun could rise from the
> west tomorrow.
The reverse is also true, if you don't know if data is lost or not you
cannot presume that it was received either, when designing network
protocols you would normally err on the side of caution rather than
assuming the data was delivered, unless you know otherwise you assume
the data was not delivered.
> Show me an example how something can get out of sync instead
> of just claiming it is so and so.
As many people stated in response to your previous proposal the roster
is certainly something that can get out of sync for various reasons,
stanzas getting lost etc. And with your spec, it can very easily get out
of sync too for exactly the same reasons (stanza's getting lost) and
even if you manage to detect that there was a problem and subsequently
reset the list, don't you think it would be better to have a protocol
that can recover from errors just like TCP can (i.e. retransmits the
lost packets) without having to start again from scratch?
> | in any circumstances that could have possibly caused it to get out of
> | sync in any way, i.e. a disconnection.
> A disconnection doesn't cause a loss of sync if it was properly terminated.
> You have a </stream:stream> for that. The example 4.8 in RFC3920 shows
> how the first server closes the stream and waits for the other side to
> terminate the TCP link. So a clean shutdown of the connection doesn't
> cause any loss. It's just how every sane protocol is designed.
OK, good this is the sort of explanation we need, yes that helps, but
from looking at my S2S logs not many servers seem to do this when
disconnecting their S2S connections, which granted is probably a bug,
but even so something should be in place to account for this so that you
can detect that something has been lost, you cannot rely on every point
in the network doing the right thing and being bug free. What if there
is a bug in a server that is causing stanza's to get randomly dropped?
You really need something that works reliably even in those kind of
situations, i.e. can detect if the list has gotten out of sync.
> | No using acknowledgements is just simple logic, if the network is not
> Your logic is re-inventing TCP. If you're not making proper use of TCP,
> then THAT is the problem.
Its not reinventing TCP as TCP only guarantees the ordering the data
will be in once it reaches the other site, it doesn't guarantee the
delivery of everything sent to the socket, and plus when dealing with
this kind of thing it goes outside of the boundary of the TCP connection
and into the server and you have no control over what might happen
there, this is how the jep-ack and related proposals work, they dont
just rely on TCP as that's the whole route of the problem in the first
> Then I presume your idea is a great plan to repair the roster sync problem,
> not to address volatile context lists.
Just demonstrating an example of how things can get out of sync as you
> | back in sync, but the difference is it will know exactly what has
> | succeeded and what probably hasn't, rather than just having to dump the
> | whole list and start again as you suggest.
> We don't need any extra information to know that something has gone wrong.
> We have network level errors. We could add this nice idea with the checksums,
> because it is lightweight, but acknowledgements is an unnecessary burden
> which defeats to a certain degree the improvement on bandwidth consumption.
Not really, as if you ack the adds and deletes the list should never get
out of sync and thus the list only needs to be set-up once, rather than
having to completely recreated it if there are any problems.
> you are making a lightweight thing a heavyweight thing, and it's not true
> that this is necessary.
Well that is in the end up to you, but experience tells me that if we
are to standardise this then people will prefer it to be a bit more
resilient to getting out of sync than it currently is, but time will tell.
> | has already been discussed, you just need to accept that stanzas might
> | get lost and you will not get any errors back to know if it was send or not.
> yes we can design a protocol on top of udp, then at least we have some
> useful aspects in this.
Yes I suppose we could design a UDP protocol, it would mean that
stanza's would be far less likely to get lost than they are at the
moment over TCP as you would need to build an acknowledgement mechanism
in and you would have far more control over it than over TCP so it would
be more reliable, bit more complex though.
> | 1) less bandwidth consumed as the list does not need to be continually
> | regenerated
> which is based on your incorrect presumption, that lists need to be
> continually regenerated.
Its not that incorrect of a presumption, looking at my S2S logs it seems
a lot of servers when terminating their S2S connections do not seem to
do it cleanly which would require the list to be regenerated. Whereas if
you know for definite which attempts to add or delete succeeded you can
just retry those, rather than having to resend the whole list again,
this means bandwidth savings even if they did get temporarily out of sync.
> | 2) the lists will be more likely to stay in sync
> likelihood, what a weird criterium.
Its perfectly valid, if you are acking the adding and removing of
addresses, it is more likely that the lists will stay in sync, as the
sending server knows for definite which succeeded.
> | 3) a more custom protocol can provide mechanisms for each server to
> | request things like hashes to allow each server to check if the list is
> | in sync
> there is space for a hash in the to= field
Yes, and... How would you suggest this work, again can you please
explain yourself, these sort of responses aren't very condusive to
understanding what you are trying to accomplish.
> | 4) the multi casting component can be far more easily decoupled from the
> | main server
> that's a very jabber way to think something as fundamental as one-to-many
> delivery can be decoupled anywhere.
Maybe so but this is the xmpp/jabber protocol after all, why shouldn't
we follow our own protocol design principles what exactly is wrong with
that? And what's wrong with having something de-coupled if you can?
> That's the problem I am talking about when I say I want to be able
> to distinguish direct presence from regular, so that it DOESN'T get
> taken into account for future broadcasts.
> At worst we'll have to push a list-delete after the directed presence.
> Then again, we can simply mark the directed presence with a namespace
That's not a very good solution as it is potentially against the RFC as
there is the possibility that in between a presence broadcast could
sneak its way through before you deleted the directed presences, they
need to not get there in the first place.
> That is not nice, but let's see what we can do about it.
> We could restrict the direct presence to always be sent to a resource
> whereas the regular presence is always sent to bare JIDs. The sending
> server would ensure that, as it has to process outgoing directed
> presence anyway.
Not you cant use the fact that the presences are going to a resource or
a bare JID to determine if something is directed presence, that would
break the specification as I haven't seen any restrictions of that sort
in there meaning directed presence can be to a base JID or a resource.
> When redistributing presence one-to-many you are rewriting things
> all the time anyway.
Only the addressing information, that's very different from re-writing
the content of a message potentially changing what it means once it
reaches the other side
> Whenever you have shown we were incorrect, we went back to drawing board
> and thought about something new. But most of the time you were TELLING us
> we were wrong with something, and never making it sufficiently CLEAR.
I'm sorry but you are guilty of exactly the same behaviour, lets just as
i've done here further explain our points, yes? Rather than simply
stating people are wrong as you have been in some of your most recent
emails, if you think they are at least try to explain why.
> I'm not here because I like XMPP. I like Jabber, the idea of it, after all
> it's the same idea that started me working on PSYC in 1991, but I sure don't
> like a lot of technical aspects about it. That is a less cozy approach than
> any Jabber enthusiast's, but still a very good chance for XMPP to get better.
Good i'm glad you like it but if you aren't going to listen to people
and accept certain limitations for the time being and develop solutions
that work around them maybe jabber isn't really for you, as even once
solutions to the problems such as delivery reliability have been agreed
here it will take a very long time if ever before it is widely deployed
enough that you can honestly and truly say the jabber network now has
reliable delivery at every point as it requires every point in the
network to have implemented this enhancement. So until then whatever
protocol you develop has to be resilient enough to work efficiently in
an environment that has unreliable delivery (which I am unconvinced it
does without ack'ing of the additions and deletions), yes you don't have
to like it, many of us don't but its simply a reality that you will just
have to accept.
More information about the Standards