[Standards-JIG] Internet-scale subscription protocols

[Benjamin Carlyle]

G'day,

I work for Westinghouse Rail Systems Australia as a software developer
and architect. I work in the SCADA world where soft real-time
notification of the state of field equipment is central to the way our
software operates. It is my goal that we should be using standard
protocols for pub/sub. Ideally, protocols that are designed for the
Internet scale.

I have been trying to get my brain into JEP-0060 from the perspective of
the more HTTP-oriented subscription patterns I have worked with and read
up on in the past. In particular, Rohit Khare's dissertation on
decentralising REST. The XMPP approach jumps the rails a bit from what I
am used to and I would like to understand whether or not it meets my
requirements.

My two fundamental requirements are:
1) That it must be scalable both in terms of how a communications
end-point manages its direct subscriptions and how layering is used to
divide and conquer the notification problem.
2) That is be capable of dealing with notifications that come in faster
than they are being delivered without a meltdown.

I have written up a draft describing how this might be done for HTTP:
http://soundadvice.id.au/blog/draft-carlyle-sena-00.txt
and have an introductary blog entry here:
http://soundadvice.id.au/blog/2006/11/02#introducingSENA

I am particularly interested in how JEP-0060 achieves summarisation, the
organised discarding of undeliverable information during overload
conditions. My full list of features I am looking for (from the draft)
goes something like this:

     Summarisation. This is the organised discarding of older
     information to ensure that slow clients recieve as much fresh data
     as their connection characteristics permit.

     Differential flow control. A slow client should not prevent fast
     ones from getting updates.

     Localised resynchronisation. A client need not reach back to the
     origin server for the current resource status if its immediate
     server is already handling the subscription.

     Patch updates. For large resources (especially lists), the ability
     to deliver a message that indicates the change from last time,
     only. Not the whole state.

     Dial-back. Pub/Sub can be an amplifier of denial of service
     attacks. The subscription mechanism must be able to detect
     when its notifications are being treated as spam and end the
     subscription.

Is JEP-0060 what I want? Can it be pushed in a direction consistent
with what I want? If not, is there some way to achieve constructive
engagement between an xmpp-based approach and a non-xmpp-based
approach?

Benjamin.