[standards-jig] Elvin

Robert Norris rob at cataclysm.cx
Wed May 8 00:26:30 UTC 2002


> On the one hand, content-based pub/sub is a proven idea that has worked
> in any number of enterprise applications. One example is the IBM Gryphon
> broker (http://www.research.ibm.com/gryphon/Gryphon/gryphon.html), which
> was used, ironically enough, at the Australian Open to push realtime
> score information to over 50,000 concurrent clients.

I had a bit of a read of the research papers presented at that URL.
There's quite a bit to think about.

[ snip lots about adoption of technology - Mike, I agree :) ]

> Now, what Elvin does have that we should find very interesting, is that
> they have laid out the same federated architecture that we have forseen,
> with relatively small intercommunicating servers administered at the
> site level. In this sense, they are light years ahead of us just in
> terms of having concrete implementations. I think we would do well to
> read through the papers that the Elvin project has published, because
> there are definitely a lot of things they have already ironed out that
> we would have had to ourselves. It is worth noting with Elvin
> specifically however, that content-based pub/sub is extremely hard to do
> in a federated environment for reasons that should be pretty clear: with
> a global network, you can't republish all messages to all the federated
> services, because that defeats the purpose and doesn't scale. So how do
> you know what to republish to whom? I don't know how Elvin has solved
> this problem, but if they have really satisfactorily done so I would be
> very impressed.

The federation spec is available here:

    http://elvin.dstc.com/doc/spec/drafts/draft-arnold-elvin-federation-prelim-00.txt

Basically, it seems that one server connects to another, and then sends
a subscribe request that incorporates all of the subscriptions it has
from its own clients, ie:

       s1 
   C1 -----
       s2  \       s1+s2+s3
   C2 ------ S1 ------//------ S2
       s3  /
   C3 -----

Where Cx is a client, Sx is a server, and sx is a subscription.

In the elvind code, the servers to federate with are listed in its
configuration file - there doesn't seem to be any dynamic federation
mechanism.

This method of connecting servers won't scale in a global network,
because it requires every server to maintain a link to every other
server.

One way I can think of to alleviate this is to have servers connect to
an arbitrary number of other servers, and then republish messages to
other servers. The downside to this is that each server would be
required to maintain a list of messages that it has already seen, to
protect against cyclic connections. It would also mean that each server
would receive all the messages from the entire network, which obviously
doesn't scale. This could possibly be solved by having each server
propogate subscriptions so that it will only republish things that
connected servers are interested in, but I can see this getting really
hairy really fast.

These are all things we'd have to think about if we went down this
track.

> What I think Elvin could gain technically by looking at Jabber is the
> XML protocol. I think a lot of people underestimate the importance of
> this. I'd be preaching to the choir to go in depth about why this is so
> good, so I won't.

Agreed.

> I do, however, think, that looking at Elvin has bumped pub/sub up a few
> notches on my list of priorities for Jabber.

It's what I've been thinking about alot lately too.

Rob.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 232 bytes
Desc: not available
URL: <http://mail.jabber.org/pipermail/standards/attachments/20020508/cc2220a3/attachment.sig>


More information about the Standards mailing list