[Standards-JIG] proto-JEP: Smart Presence Distribution
richard at dobson-i.net
Thu Jun 1 12:50:55 UTC 2006
> | Well it might be the end of the story for you but its still a big
> | failing in your spec, it doesn't have a decent level of error recovery,
> claim, no facts.
How is it not a fact that your spec does not have a form of error
recovery that only needs to retransmit only what failed, rather than the
whole list to the server?
> you still haven't understood the proto-JEP properly.
> no matter which part of the broad unicast fails and when, only that part
> needs to be repeated or re-established. you don't have to restart
> everything from scratch - you only need to redefine a couple of people
> on the list of one particular host, if that particular host had serious
> delivery problems.
According to your spec, unless you have updated it very recently when
you get an error you reset the whole list, has this changed so that only
the failed items are resent? how did you accomplish this without acks?
> | would be far better if your spec were able to only have to resend only
> | those which did not get there.
> that's what it does. thanks for talking so much before reading it closely.
But it doesn't, you have to send the whole list (for a particular
server) again, rather than just the single JID at that server that you
were trying to add, you really do seem to be good at spinning facts and
what people are saying, you really should get yourself a job in politics
:P, hehe, but what I said is still true when related to the traffic
between two servers.
> | "Like a broadcast, a multicast replicates information from one source to
> | multiple recipients on a network"
I see, so you are saying google and all the sites it is indexing for the
definitions are incorrect, hmmm, I'm sorry but I'm far more inclined to
believe these google results than you.
> | "To transmit a message to a select group of recipients. A simple example
> | of multicasting is sending an e-mail message to a mailing list."
> incorrect. why would one invent "ip multicast" if e-mail already did it.
I see so you are saying *SES SIRIUS AB are liers then, hmm, interesting.
> | "Special form of broadcast in which copies of the packet are delivered
> | to multiple stations, but only a subset of all possible destinations"
> just go into the IETF MMUSIC WG and ask them if these are valid
> definitions of "multicast" - they will, if they spare the time, inform
> you that these are all superficial simplifications of what multicast
> actually does.
Even if they are simplified they are still valid definitions, can we
just put an end to this, you are clearly mistaken as far as I can see
and I doubt you will ever be able to convince me that your definition is
the only possible valid one, so there really is no point continuing this
part of the discussion.
> | "Multicast is the delivery of information to a group of destinations
> | simultaneously using the most efficient strategy to deliver the messages
> | over each link of the network only once and only create copies when the
> | links to the destinations split. By comparison with multicast,
> | conventional point-to-single-point delivery is called unicast, whereas
> | delivery to every node of the network is broadcast"
> this definition is mostly correct, although the intention is that every
> packet gets sent to australia only once, so if you know people on 4
> servers in australia, xmpp will, no matter which jep we apply to it,
> send the packets 4 times towards australia. that's no longer proper
> multicast then. it is what we call 'smarticast' at best.
Just because all xmpp traffic for Australia doesn't get routed though a
single point and then fanned out from there doesn't in and of itself
mean it can't be described as multi casting as its not the countries
that are interested in this traffic, its individual server domains, but
even so my suggested spec could operate in the way you suggest above if
necessary, the multi casting component could possibly be shared by
multiple servers when using my suggestion, when the external servers are
discovering the multi casting component to use for a particular domain
say the sending domain is in France and the receiving server is in
Australia, all the receiving servers in Australia could have an
agreement to share a single multi casting component for international
traffic, now when the server in France tries to discover the multi
casting component for the Australian domains they will all tell it to
use the same one, the server in France when it tries to communicate with
more than one of these Australian servers could realise this and
negotiate a single list with the multi casting component that covers all
the Australian servers that share this multi casting component, easy,
there could also be regional multi casting components that handle
subsets of these servers and there is no real reason that the multi
casting components couldn't negotiate lists with each other to further
fan out the data (although this is getting a little over complicated),
so you see even by your too narrowly defined definition of multicast, my
suggestion could certainly be described as multicast as it is physically
capable of doing it (although whether people would want to set it up
that way is another question entirely due to all the trust and potential
spam issues that arise), so please just drop your assertions about us
being wrong to use the term multicast, as even by your incorrectly
narrowly defined definition its fine, even JEP-0033 could work in a tree
distribution style if you really tried.
Also there is no such term as smarticast you are just making it up,
whereas an existing definition fits fine as you can see above.
> | Protocol verbosity??? I'm confused, how does reducing the number of
> | stanzas significantly and easily providing for error recovery mean its
> | verbose, if anything its less so due to the major reduction in the
> your list generation is verbose, maybe not much more than a "broad unicast"
> of presence though, so if the good ole "broadcast" is no longer wanted, it
> might as well be an iq. we still get to send an extra <presence/> stanza
> to all links, and we have a bunch of redundant acks, since TCP is stable
> enough for the purpose. so all in all your proposal is still more verbose
> and additionally it requires an extra roundtrip for the acks which delays
> presence delivery, as dave pointed out.
My suggested list generation is not verbose when directly compared to
your method of sending lots of individual presence stanzas to build up
the list, and there are not lots of redundant acks, as I showed
demonstrated when initially setting up the list there is only the need
to send a single stanza which will get a single ack, to be classed as a
bunch of acks if would have to be more than one. Also I've got no idea
what you are going on about with regards to not needing to do
"broadcast" any more I thought that was the whole point of both of our
Extra presence stanza??? Not sure what you mean by this either, could
you please explain.
But yes it does delay the initial presence delivery, but this will only
happen once as the lists will not need to be rebuilt and will be able to
be re-used whenever they reconnect, I find it very unlikely anyone would
really notice this, its unlikely to take more than a second or two to
set-up the list due to the fact it can all be done in a single stanza,
if you are referring to the delay that will happen while waiting for the
S2S connections to establish between the domains then that's not really
an issue either as until that is established you wouldn't be able to be
communicating with the people at that server anyway or have even
received their presence in the first place to know that they are
on-line, so again not an issue.
As already proved by the ack discussion TCP is not stable enough or that
discussion would not have happened in the first place, and there
wouldn't be any of the acking proposals, so again please stop repeating
your assertions that TCP is stable enough because there is plenty of
evidence to the contrary, you continuing to state this just demonstrates
a lack of full understanding of the issues at hand.
> 50 stanzas? didn't stpeter once claim average users have 10 people in their
> rosters? so why should they have 50 people all on one server? anyway,
> amount of people doesn't matter.
If its true that on average there are only a few people on peoples
rosters then there is probably no point in even developing this
optimisation in the first place as it will mean it does not make any
> if the amount of stanzas makes a difference though, than a single iq
> with a lot of subtags is better than a series of presence stanzas.
> please elaborate what the gains are moving away from the traditional
> presence fan-out as it has always been, and on the other hand, if
> anyone is contrary to the <iq> list setting strategy, please speak up now.
Well using IQ means not just potential bandwidth savings when setting up
lists (especially large ones as the bandwidth savings will increase the
bigger the list), but it also means its easier for the server or
component to manage the list as it can all be done quickly in a single
operation rather than having to process lots of individual stanzas to
build up the list, also because XMPP stanzas are not guaranteed to be
delivered in the order they were sent (although on the whole they will
normally get there in order) having the built in acking of IQ means the
sending server knows once the operation has been completed and can then
start sending the presence stanzas to the component knowing that they
will definitely be delivered to the entire list that they set-up,
whereas with your spec there is a chance that if you quickly send a
presence stanza that is intended to be broadcast to the list along with
the set-up stanzas that the broadcast stanza could get processed before
all of the set-up stanzas have finished being processed.
> yes i understand the amazing manifold world of timeout problematica,
> but still the most common reason for roster asyncs could be as simple
> as server downtimes. luckily our proto-jep doesn't need to do anything
> if the other side isn't available, so it can always limit its operation
> to a healthy working connection between two entities.
You still seem to be missing the point, its not just server downtime
that causes these connection time-out problems, server downtime will
most likely cause the connections not to be able to be established in
the first place, the main cause of these time-out problems is
intermittent problems with things like firewalls/routers/NATs where
already established connections become broken (which is not able to be
detected often for some time), whereas even while the connection is
broken (which because of things like NATs will likely mean the
connection will never be able to be restored) newer incoming connections
could be fine, but until the original connection times out you don't
know and all of the data sent to the socket in the mean time will be
> it allows for
> [x] unquoted and unparsed binary file transfers
> [x] routing and multicasting content without losing time parsing it
> [x] probably more
> even if we get xmpp to do real multicast, it's a bit sad we cannot
> use it for transparent transfers. you probably don't wanna know
> which technology is suited for all of those [x] up there.
My guess would probably be that you are alluding to SIP, but that's
pretty much irrelevant, part of the whole point of XMPP is that it is a
streaming XML protocol, binary framing doesn't really fit in with this,
but all it means is that everything sent over the socket must be valid
XML so its certainly possible to use it for transfers, it just means
they need to be wrapped up in an XML compatible way which is not exactly
very difficult, have a look at JEP-0047.
> if the jep makes a precise definition on the sort of change it MUST
> make in order for the distribution system to operate correctly, that
> is a very very unfree change of an attribute, and not of the content.
Better to avoid if at all possible, although I've got a feeling that it
will probably be the only way to do it that will work in-line with the
rest of your proposal as it stands.
> also we still haven't dismissed the option that we could as well
> use the recipients roster, since the scenario of somebody hacking his
> roster when he also needs to receive directed presence *and* needs to
> be on the same server with more recipients, makes it a rather faint
> security weakness. it's easier to talk somebody into relaying you
> somebody else's presence.
Well I certainly thought we had already dismissed it, but if you really
want to take a step back to that again then feel free, its up to you,
but I very much doubt you will be any more successful than you were last
> so if we use the second scenario, we don't even need to modify directed
The roster scenario you mean? Well that's up to you but I find it
unlikely many people are going to be prepared to implement that solution
given all the problems that have already been pointed out to you.
> that is your presumption, but by all logic i can't see how a lot of
> tcp connections should keep on breaking continously when both ends
> could have closed them properly.
Well then it sounds like you possibly don't have enough real world
experience of TCP/IP networking.
> but sure, we could add an extra clause that stops presence distribution
> on unreliable links. i mean, if a network connection isn't reliable, then
> it is experiencing some serious problems. then i would want important
> protocols like ssh to get all the priority. the least i need on a messed
> up network is some people's away messages.
Well I would suggest you add the acking protocol cause in right away
then, and wait for it to be widely implemented.
More information about the Standards