[standards-jig] Pub/Sub Explanation

Piers Harding piers at ompa.net
Tue Apr 16 15:14:15 UTC 2002

It is great to see some activity about this!

On Mon, Apr 15, 2002 at 10:20:52PM -0600, Dave Smith wrote:
> Greetings...
> This email is long overdue. There has been a lot of discussion regarding a
> pub/sub protocol, and as a part of that, I (as a developer who works for
> JINC) have stated that there is another protocol proposal forthcoming. Along
> the way, a lot of feelings have been hurt and I know that some people feel
> that JINC hasn't been following the rules that the Foundation is founded on
> for protocol proposal.
> I take personal responsibility for not being more forthcoming with details.
> I could blame it on any number of circumstances, but the fact of the matter
> is that I haven't taken the time and focus to layout the details of what I
> (and JINC, in turn) would like to see in a pub/sub system. I apologize. Let
> me take some time to explain the details...
> First off, let's be very clear that at this point protocol is not as
> important as it seems. From the discussions I've had with DJ, piers, Rob
> Norris, and others it appears that we have some fundamental usage issues
> that need to be addressed. In other words, the various incarnations of the
> pub/sub proposals all have slightly different ideas about what "pub/sub"
> means and what ways it might be used.  The details of the protocol XML can
> be discussed after we have established a common glossary and set of use
> cases (i.e how people are going to use this piece of the Jabber system).

I would argue that pubsub in an IM context with real push technology, as well as pull is a new territory, and we all do not understand the full implications/possibilities yet.

> Secondly, let's be clear that it's completely possible that what JINC (or
> any other developer, company, community, etc) needs in a pub/sub system may
> be different enough to warrant multiple protocols. That's not bad -- it just
> means that different parties need different functionality. As a standards-
> based community we will not have failed if there _does_ turn out to be
> multiple protocols. We only fail when we don't take the time to clearly
> identify the requirements for new Jabber sub-systems.
> Before we can actively discuss use cases for this sub-system, we'll also
> need to agree on some basic terminology:
> Topic -  A focal object which relates Jabber entities which provide
> information to Jabber entities which are interested in that information.
> Topics are composed of two essential collections of information:
> a subscription collection, and an item collection
> Category - An organizational unit which topics are associated with. The
> category in which a topic is created determines the durability of the topic
> and its composite collections.

Topic and category do not need to exist as separate entities - these could be comfortably covered by namespace definitions, as namespaces are infinitely flexible, and are application specific data that the end user and the publisher share meaning with.

> (Topic) Item - An instance of data stored within a topic. Each item may have
> a unique identifier which allows the item to be keyed for updates.

This all presuposes data storage - ie. that the pubsub implementation should store data.  This I fundamentally disagree with.  Data storage is something that I believe should be tackled as a separate JEP/implementation in JAbber, as there is so much potential for the use of data storage through out the Jabber world.  Pubsub should deal with the subscription to information/data and its changes, not the actual management of the data itself.

> Durability - Lifetime of a topic, subscription, or item. Something is said
> to be "durable" if has a relatively long lifetime (multiple Jabber sessions)
> and is typically hard-stored. Along these lines, something which exists for
> the duration of a session (or less) is said to be "transient".  Durability
> can vary in pub/sub depending on what piece of information you are talking
> about. For instance, the a topic may be "durable" while the subscriptions to
> that topic are "transient". This would mean that the topic would exist
> across Jabber sessions, but subscriptions to this topic would only last for
> as long as a single Jabber session. Of all the points, this concept of
> durability will probably be the most difficult to discuss and decide upon.

Durability is also application specific data ass defined by the relationship between the end consumer, and the publisher.  If the data has a TTL, then existing facilities in the jabber protocol ie. jabber expires - could be used.

> With those ideas in mind, let me list the basic use cases that I see pub/sub
> fulfilling:
> UC1 - Manage a Topic (create, delete)
> UC2 - Publish Items to a Topic
> UC3 - Get Published Items for a Topic (retrieve the last 100 items
> published)
> UC4 - Get Topic Information (subscriber list, durability info, header
> name/value pairs)
> UC5 - Discover Topics (find topics that a JID is subscribed to, etc.)
> UC6 - Subscribe to Topic (includes unsubscribe)

> One of the fundamental concepts here is that the pub/sub service is capable
> of storing items which are published. This is a weighty requirement -- it
> implies some sort of data storage. It is my belief, however, that this
> requirement is essential to having a generic pub/sub system that moves the
> Jabber platform forward.
> In conclusion, let me list some possibilities that have been contemplated
> for such a pub/sub service:
> - News-feed system: let's make it possible to publish data to a large group
> of people who want to be notified when a news item is posted. Since we're
> already pushing notifications out to people, let's go ahead and piggyback
> the actual data out within the notification to minimize the polling
> - Presence system: see also Rob Norris' JEP-21. By adding the ability to
> track previous data stored, however, it's possible to subscribe to a
> presence topic and then grab the latest presence info in a nice generic way.
> - Text-conferencing: should be obvious -- probably wouldn't need to use any
> sort of long term storage ('less you wanted to long-lived rooms)

What about more specific examples such as product catalogs in business portals.  What would pubsub need to provide to support them?

> So..that's about it.  I'm sure I've not covered some of the details to the
> degree some people would like; I'll be glad to expound.
> Diz
> _______________________________________________
> Standards-JIG mailing list
> Standards-JIG at jabber.org
> http://mailman.jabber.org/listinfo/standards-jig

More information about the Standards mailing list