[Standards-JIG] proto-JEP: Smart Presence Distribution

[Richard Dobson]

> | Huh???? "disabling large parts of RFC 3921 in your interserver 
> | communications", sorry but what are you on about? Please explain.
>
> 5.x in XMPP IM describes how to deliver presence information to other
> servers and people. If you're proposing an explicit new syntax to define
> recipient lists on receiving servers, you are delivering similar information
> in a completely new way.
>
> Either you keep on using XMPP IM in parallel, then you have no gain,
> or you have to disable it in favor of the new list syntax, then you are
> designing a new XMPP protocol, because nothing of that part of RFC 3921
> is going to be used anymore.
>   
Of course they will be used, the same rules that you currently use to 
send out the individual broadcasted stanzas you will use for building up 
the explicit list for sending the broadcasted stanzas, the only 
difference is that the stanzas are fanned out at the remote server 
rather than the local one, that is the one and only difference. If 
anything it is your proposal that changes how section 5.x in XMPP IM 
works more as you are using the stanzas to mean something they were not 
originally intended to, and you proposal infact entirely breaks section 
5.1.4 (directed presence). So if anything it is my proposal that is more 
in-line with the RFCs as it doesn't break anything.
> | I've seen it, but as noted and admitted to by Philipp the protocol as 
> | outlined means the lists will need to be recreated over each new S2S 
> | connection and will only be valid for that S2S connection, as soon as 
> | that connection drops the list becomes invalid and needs to be 
> | regenerated at the next connection, this is a problem because it will 
> | not work very well with how xmpp/jabber servers are currently 
> | implemented as the presence sending code and s2s are almost always 
>
> This architectural issue sounds like a general strategic problem.
> Like an ostrich Jabber puts its head under earth surface and doesn't
> want to know about real life network failures and unreliability.
> If this JEP enforces that you do not only route XML content packets from
> different parts of your server architecture, but also expects you to properly
> route exceptions and errors, this may help solve other problems, too.
> So it's about time you stop ensuring stability by redundancy but also by
> error handling!
>   
Yes but just as with rosters it was not as serious a problem as it would 
become with your spec in place and you changed the meaning of a long 
established mechanism, I am starting to get a feeling of deja-vu here 
:), there is no point in going on about "jabber sticking its head in the 
sand" all extensions to protocols need to take into account of what is 
already there including any known problems there are, it cant just 
ignore them or wish they weren't there, it needs to operate in such a 
way as to not be affected by them, which your currently spec does not.
> I think I mentioned in an other mail, that there is a special case 
> allowing your servers to time-out connections without losing the state.
> Other than that only evil things can cause the necessity of a list refresh.
>   
But in your proposal there is no way for either server to know if the 
list is out of sync, so they are forced to always repeatedly refresh it 
in any circumstances that could have possibly caused it to get out of 
sync in any way, i.e. a disconnection.
> | between servers where the S2S connections frequently drop and 
>
> As long as the connections are cleanly closed or timeouted, that's fine.
> It's not exactly an elaborate strategy, but if your servers are so
> monolithic that the amount of open sockets is an issue, then you will
> have to keep doing things this way.
>   
No its not fine as servers cannot know for sure if the connections were 
cleanly closed or timeouted, certainly not be sure enough to guarantee 
that no stanzas get lost and the lists remain in sync.
> | re-establish, as normally in those cases the presence stanzas would only 
> | get sent once, whereas with your suggestion every time the connection is 
> | re-established the stanza's will need to be resent again to populate the 
> | list, if that list doesn't actually end up being used before the 
> | connection drops then all that traffic would be for nothing.
>
> No, even if you had a faulty break of the connection, it is not your duty
> to immediately update the list. You can postpone it to the moment in time
> when you are actually sending some information over it.
>   
I see, but what if you only end up sending it once over the connection, 
its a bit of a waste, better if the sending server knows for sure who it 
successfully added or deleted from the list, then there is no need to 
constantly reset it as would be required by your proposal.
> Introducing acknowledgments is like re-inventing TCP here. Especially for
> such a lightweight operation like on-the-fly context lists. All we need
> is proper exception handling on TCP and XMPP stream level, and the application
> is stable and well-defined. Think of rosters - they have acknowledgments -
> and still they aren't reliable.
No using acknowledgements is just simple logic, if the network is not 
guaranteed to be reliable then you need to know if your message got 
there or not if its something that needs to be kept in sync between two 
different parties. Rosters do not have acknowledgements between servers 
in the same way as I am suggesting, rosters don't automatically send 
back responses indicating whether the last request got to the other end 
or not, it is entirely up to the end user if they want to do anything 
about any subscription requests etc they receive, rosters don't 
automatically do anything about any errors they get back, or if the 
stanza was lost, the mechanism I am suggesting would have quick 
responses if the request got to the other end, if the sending server 
didn't get a response to a request it sent it can assume it didn't get 
there and re try until it does, if it continues to fail it can assume 
the list of out of sync and use whatever mechanisms we define to get it 
back in sync, but the difference is it will know exactly what has 
succeeded and what probably hasn't, rather than just having to dump the 
whole list and start again as you suggest.
> TCP gives you errors. If your server architecture isn't a black hole,
> you can implement things according to the proto-JEP and there will not
> be any loss of sync. Show me a situation that the JEP doesn't handle.
>   
You need to have a read of the archives, reliability has already been 
discussed at length and is not something that is easy to solve as with 
things like sockets even if there is a TCP error there is no way to know 
once you have put something onto the socket whether it got to the other 
end or not, this is simply a fact of life and not something you can just 
dismiss as an implementation issue as its far more complex than that as 
has already been discussed, you just need to accept that stanzas might 
get lost and you will not get any errors back to know if it was send or not.
> | successful or not), you cannot rely on implicit additions and deletions 
> | as you are in this proposal.
>
> That's incorrect. I'm sorry, your guess is just wrong.
>   
Sorry but its not incorrect at all, I have shown here how you cannot 
guarantee that stanzas will get to where they are going (or that they 
will bounce if they don't), meaning the lists can get out of sync very 
easily, im sorry but it is you that is "just wrong", you really need to 
get with it and accept things other people are telling you or risk 
yourself and your work becoming irrelevant, its up to you.
> You think a problem has to be solved in a certain way,
> but there are more ways to do it, and yours only creates overhead.
>   
Yes there are other ways to do it, I never said there wasn't.

Now here's some advice for you, you need to accept and take on board the 
problems in your suggestions that people are pointing out, and stop 
trying to ignore and brush them aside as you tried to when you were 
arguing for your roster integrated mechanism which completely failed. 
The more you protest and not provide solutions to the problems people 
are pointing out and instead say how things should be rather than simply 
accepting how they are the less likely people are going to take you 
seriously, please learn from your mistakes and lets not have a repeat of 
your roster integrated spec.

Also my suggestion does not create any more overhead than your spec 
does, and infact has major benefits over yours:

1) less bandwidth consumed as the list does not need to be continually 
regenerated
2) the lists will be more likely to stay in sync
3) a more custom protocol can provide mechanisms for each server to 
request things like hashes to allow each server to check if the list is 
in sync
4) the multi casting component can be far more easily decoupled from the 
main server

If you continue to fail to provide solutions for the problems being 
outlined (i.e. unreliability, synchronisation etc) I think ill just 
document my solution and we'll let the community decide which is best, 
be far easier than continuing to argue this out with you as it doesn't 
seem to be getting anywhere as you don't seem to be listening.

Richard