[standards-jig] Re: [JDEV] PubSub (0060) Max_items -> Range?

Timothy Carpenter timbeau_hk at yahoo.co.uk
Tue May 20 20:21:51 UTC 2003

>A similar solution could be crafted for pubsub.

The example I have given in JEP0032 for sequence was not just plucked out of
the air, but the mechanism is drawn from three long implemented highly
robust pub sub protocols including Tibco's TIB and Reuters' Triarch.

Tibco's founder, Vivek Ranadive invented the concept of pub sub, so I think
this is not  a bad place to start... :-)

>I think of the IMAP protocol which has addressed a similar problem.  In
>IMAP you have "mailboxes" (think nodes) and messages (think items).

Email is not quite the same beast as a real time datastream. Nodes do not
map to mailboxes, as a node contains a static or expanding or contracting
set of items that can change their value over time. Items to not map to
messages as messages do not necessarily have linkage and do not form part of
a time series (unless it is in such a forum as this :-) ).

The analogy would be more like: a mailbox is like a circuit between pub and
discrete sub, with mail messages being the events. This is as close as it
gets as far as I can see and it is not a good comparison due to the lack of
linkage between mail messages.

For robust pub sub, you need to have the concept of sequence numbering at
different levels. It may be overkill to have separate sequences for node,
channel (i.e. Circuit between pub and sub) and source  (the overall stream
from a pub) but there are many gotchas out there and making the intermediate
pub server efficient and light whilst allowing for gap filling is a good
test of elegant design.

The pub in Jabber terms is a server, not the actual source of the data.
Should a request for data fall outside the cache of the pub server, then the
pub server may need to revert back to the originator. This happens.

The originator (true publisher) does not know about who has subscribed and
what sequencing they are using across their channel. You cannot give the
subscriber the originator's sequencing as the subscriber may not be getting
every event (due to filtering via their subscription). The pub server has to
perform the mapping and management of that, as it is the one that knows who
is where from both sides and who wants to see what.

Sequencing per node can work if a node cannot be caused to be intermittently
delivered due to subscription filtering (some updates not of interest). If
we cannot rely on this, we may have to use curcuit and/or source sequence
numbering, if not both.


More information about the Standards mailing list