From stpeter at stpeter.im Wed Oct 1 22:44:55 2008 From: stpeter at stpeter.im (Peter Saint-Andre) Date: Wed, 01 Oct 2008 21:44:55 -0600 Subject: [PubSub] create node with default configuration Message-ID: <48E443B7.6000104@stpeter.im> XEP-0060 currently says that if you want to create a pubsub node with the default configuration, you MUST include an empty element in the node creation request, for example: Ralph Meijer pinged me the other day and suggest that the intention here was SHOULD, not MUST. What do people think? Peter -- Peter Saint-Andre https://stpeter.im/ From jack at chesspark.com Thu Oct 2 00:19:40 2008 From: jack at chesspark.com (Jack Moffitt) Date: Wed, 1 Oct 2008 23:19:40 -0600 Subject: [PubSub] create node with default configuration In-Reply-To: <48E443B7.6000104@stpeter.im> References: <48E443B7.6000104@stpeter.im> Message-ID: <9b58f4550810012219l7f227c0fx2b8e86e82a91959b@mail.gmail.com> > XEP-0060 currently says that if you want to create a pubsub node with > the default configuration, you MUST include an empty > element in the node creation request, for example: > > Ralph Meijer pinged me the other day and suggest that the intention here > was SHOULD, not MUST. What do people think? What would the server do if it was not set? Just return an error? SHOULD seems fine to me, and I assum with or without it would create a node with the default configuration (assuming creation is otherwise allowed of course). jack. From kevin at kismith.co.uk Thu Oct 2 01:20:40 2008 From: kevin at kismith.co.uk (Kevin Smith) Date: Thu, 2 Oct 2008 07:20:40 +0100 Subject: [PubSub] create node with default configuration In-Reply-To: <48E443B7.6000104@stpeter.im> References: <48E443B7.6000104@stpeter.im> Message-ID: On Thu, Oct 2, 2008 at 4:44 AM, Peter Saint-Andre wrote: > Ralph Meijer pinged me the other day and suggest that the intention here > was SHOULD, not MUST. What do people think? I recall a recent(ish) discussion of the merits of tightening up specs to use SHOULD less and MUST more. Certainly from an implementation pov, it's a pain to not be certain what you're receiving - what's the motivation for changing to a SHOULD? /K From Ovidiu.Craciun at philips.com Thu Oct 2 01:21:50 2008 From: Ovidiu.Craciun at philips.com (Ovidiu Craciun) Date: Wed, 1 Oct 2008 23:21:50 -0700 Subject: [PubSub] create node with default configuration References: <48E443B7.6000104@stpeter.im> Message-ID: <0AA3A2C546487C4A93AE60213FEFF97F074F7C@SFO00MX1.stentor.com> if i can add my 2 cents here, these specs MUST be as clear as posible, as specific as posible; these SHOULDs, MIGHTs are not inline with this idea; they leave room for too many variables, expectations and what this protocol could be but it ends up not been. O. ________________________________ From: pubsub-bounces at xmpp.org on behalf of Kevin Smith Sent: Wed 10/1/2008 11:20 PM To: XMPP publish-subscribe and personal eventing Subject: Re: [PubSub] create node with default configuration On Thu, Oct 2, 2008 at 4:44 AM, Peter Saint-Andre wrote: > Ralph Meijer pinged me the other day and suggest that the intention here > was SHOULD, not MUST. What do people think? I recall a recent(ish) discussion of the merits of tightening up specs to use SHOULD less and MUST more. Certainly from an implementation pov, it's a pain to not be certain what you're receiving - what's the motivation for changing to a SHOULD? /K -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.jabber.org/pipermail/pubsub/attachments/20081001/0372f49b/attachment.htm From stpeter at stpeter.im Thu Oct 2 07:13:13 2008 From: stpeter at stpeter.im (Peter Saint-Andre) Date: Thu, 02 Oct 2008 06:13:13 -0600 Subject: [PubSub] create node with default configuration In-Reply-To: <9b58f4550810012219l7f227c0fx2b8e86e82a91959b@mail.gmail.com> References: <48E443B7.6000104@stpeter.im> <9b58f4550810012219l7f227c0fx2b8e86e82a91959b@mail.gmail.com> Message-ID: <48E4BAD9.6060903@stpeter.im> Jack Moffitt wrote: >> XEP-0060 currently says that if you want to create a pubsub node with >> the default configuration, you MUST include an empty >> element in the node creation request, for example: >> >> Ralph Meijer pinged me the other day and suggest that the intention here >> was SHOULD, not MUST. What do people think? > > What would the server do if it was not set? Just return an error? > > SHOULD seems fine to me, and I assum with or without it would create a > node with the default configuration (assuming creation is otherwise > allowed of course). Right, that's not specified. Given that you're assigning default configuration if a non-default configuration is not specified, I don't particularly see any reason for the empty element! Peter -- Peter Saint-Andre https://stpeter.im/ From ralphm at ik.nu Sat Oct 4 08:32:12 2008 From: ralphm at ik.nu (Ralph Meijer) Date: Sat, 04 Oct 2008 15:32:12 +0200 Subject: [PubSub] create node with default configuration In-Reply-To: <48E4BAD9.6060903@stpeter.im> References: <48E443B7.6000104@stpeter.im> <9b58f4550810012219l7f227c0fx2b8e86e82a91959b@mail.gmail.com> <48E4BAD9.6060903@stpeter.im> Message-ID: <48E7705C.9070905@ik.nu> On 2008-10-02 14:13, Peter Saint-Andre wrote: > Jack Moffitt wrote: >>> XEP-0060 currently says that if you want to create a pubsub node with >>> the default configuration, you MUST include an empty >>> element in the node creation request, for example: >>> >>> Ralph Meijer pinged me the other day and suggest that the intention here >>> was SHOULD, not MUST. What do people think? >> What would the server do if it was not set? Just return an error? >> >> SHOULD seems fine to me, and I assum with or without it would create a >> node with the default configuration (assuming creation is otherwise >> allowed of course). > > Right, that's not specified. Given that you're assigning default > configuration if a non-default configuration is not specified, I don't > particularly see any reason for the empty element! Indeed, that was my thinking as well. Historically, node configuration has not always been around. And it has always been optional to implement the node configuration protocol. As such, it doesn't make sense to always have an empty element to be present. I think that usage came about when we thought up the concept of nodes that MUST be configured before they can be used (much like MUC rooms). An example could be that you must provide a title or somesuch. I don't see how an empty element would resolve an otherwise unconfigured node. As far as I can see, an empty element is exactly equivalent to not sending it along at all, and thus it MUST NOT be required to create a node if a default configuration can be applied. ralphm From pavlix at pavlix.net Sun Oct 5 07:51:46 2008 From: pavlix at pavlix.net (Pavel Simerda) Date: Sun, 5 Oct 2008 14:51:46 +0200 Subject: [PubSub] create node with default configuration In-Reply-To: <48E7705C.9070905@ik.nu> References: <48E443B7.6000104@stpeter.im> <9b58f4550810012219l7f227c0fx2b8e86e82a91959b@mail.gmail.com> <48E4BAD9.6060903@stpeter.im> <48E7705C.9070905@ik.nu> Message-ID: <20081005145146.123fb11e@tiger> On Sat, 04 Oct 2008 15:32:12 +0200 Ralph Meijer wrote: > On 2008-10-02 14:13, Peter Saint-Andre wrote: > > Jack Moffitt wrote: > >>> XEP-0060 currently says that if you want to create a pubsub node > >>> with the default configuration, you MUST include an empty > >>> element in the node creation request, for example: > >>> > >>> Ralph Meijer pinged me the other day and suggest that the > >>> intention here was SHOULD, not MUST. What do people think? > >> What would the server do if it was not set? Just return an error? > >> > >> SHOULD seems fine to me, and I assum with or without it would > >> create a node with the default configuration (assuming creation is > >> otherwise allowed of course). > > > > Right, that's not specified. Given that you're assigning default > > configuration if a non-default configuration is not specified, I > > don't particularly see any reason for the empty element! > > Indeed, that was my thinking as well. > > Historically, node configuration has not always been around. And it > has always been optional to implement the node configuration > protocol. As such, it doesn't make sense to always have an empty > element to be present. > > I think that usage came about when we thought up the concept of nodes > that MUST be configured before they can be used (much like MUC > rooms). An example could be that you must provide a title or > somesuch. I don't see how an empty element would resolve > an otherwise unconfigured node. > > As far as I can see, an empty element is exactly > equivalent to not sending it along at all, and thus it MUST NOT be > required to create a node if a default configuration can be applied. > > ralphm +1 for optional -- Pavel ?imerda Freelancer v oblasti po??ta?ov?ch s?t?, komunikace a bezpe?nosti Web: http://www.pavlix.net/ Jabber & Mail: pavlix(at)pavlix.net OpenID: pavlix.net From pavlix at pavlix.net Sun Oct 5 07:59:31 2008 From: pavlix at pavlix.net (Pavel Simerda) Date: Sun, 5 Oct 2008 14:59:31 +0200 Subject: [PubSub] pubsub & jabber:x:data Message-ID: <20081005145931.0112505d@tiger> Hello people, I am curious about the use of jabber:x:data in PubSub. I'm for using jabber:x:data where it fits and where it brings significant value. What are the usecases for PubSub node configuration, where we need special non-standard fields passed to the user (allong with types/options and such, to fill in. After all, this is what's jabber:x:data for, isn't it? In the PubSub specification I only see examples with a fixed set of fields and no real jabber:x:data use except useless syntactical sugar (=complication). -- Pavel ?imerda Freelancer v oblasti po??ta?ov?ch s?t?, komunikace a bezpe?nosti Web: http://www.pavlix.net/ Jabber & Mail: pavlix(at)pavlix.net OpenID: pavlix.net From kevin at kismith.co.uk Sun Oct 5 09:20:38 2008 From: kevin at kismith.co.uk (Kevin Smith) Date: Sun, 5 Oct 2008 15:20:38 +0100 Subject: [PubSub] create node with default configuration In-Reply-To: <20081005145146.123fb11e@tiger> References: <48E443B7.6000104@stpeter.im> <9b58f4550810012219l7f227c0fx2b8e86e82a91959b@mail.gmail.com> <48E4BAD9.6060903@stpeter.im> <48E7705C.9070905@ik.nu> <20081005145146.123fb11e@tiger> Message-ID: If the intention is for no configure to be the same as empty configure, why not make that the way forward, instead of SHOULDing the empty element? /K From stpeter at stpeter.im Sun Oct 5 09:21:26 2008 From: stpeter at stpeter.im (Peter Saint-Andre) Date: Sun, 05 Oct 2008 08:21:26 -0600 Subject: [PubSub] create node with default configuration In-Reply-To: References: <48E443B7.6000104@stpeter.im> <9b58f4550810012219l7f227c0fx2b8e86e82a91959b@mail.gmail.com> <48E4BAD9.6060903@stpeter.im> <48E7705C.9070905@ik.nu> <20081005145146.123fb11e@tiger> Message-ID: <48E8CD66.50108@stpeter.im> Kevin Smith wrote: > If the intention is for no configure to be the same as empty > configure, why not make that the way forward, instead of SHOULDing the > empty element? That's what I was trying to say in my previous message. :) +1 /psa From jack at chesspark.com Sun Oct 5 10:26:08 2008 From: jack at chesspark.com (Jack Moffitt) Date: Sun, 5 Oct 2008 09:26:08 -0600 Subject: [PubSub] create node with default configuration In-Reply-To: <48E8CD66.50108@stpeter.im> References: <48E443B7.6000104@stpeter.im> <9b58f4550810012219l7f227c0fx2b8e86e82a91959b@mail.gmail.com> <48E4BAD9.6060903@stpeter.im> <48E7705C.9070905@ik.nu> <20081005145146.123fb11e@tiger> <48E8CD66.50108@stpeter.im> Message-ID: <9b58f4550810050826u37d98771la1f5d0098942e1bf@mail.gmail.com> >> If the intention is for no configure to be the same as empty >> configure, why not make that the way forward, instead of SHOULDing the >> empty element? > > That's what I was trying to say in my previous message. :) I think that's what I'd prefer as well. Perhaps we can say that a client MAY send an empty configure element, but that new implementaitons SHOULD NOT do that. And no configure element can just be the normal way to do this. jack. From ralphm at ik.nu Sun Oct 5 11:35:16 2008 From: ralphm at ik.nu (Ralph Meijer) Date: Sun, 05 Oct 2008 18:35:16 +0200 Subject: [PubSub] create node with default configuration In-Reply-To: <9b58f4550810050826u37d98771la1f5d0098942e1bf@mail.gmail.com> References: <48E443B7.6000104@stpeter.im> <9b58f4550810012219l7f227c0fx2b8e86e82a91959b@mail.gmail.com> <48E4BAD9.6060903@stpeter.im> <48E7705C.9070905@ik.nu> <20081005145146.123fb11e@tiger> <48E8CD66.50108@stpeter.im> <9b58f4550810050826u37d98771la1f5d0098942e1bf@mail.gmail.com> Message-ID: <48E8ECC4.9020001@ik.nu> On 2008-10-05 17:26, Jack Moffitt wrote: >>> If the intention is for no configure to be the same as empty >>> configure, why not make that the way forward, instead of SHOULDing the >>> empty element? >> That's what I was trying to say in my previous message. :) > > I think that's what I'd prefer as well. Perhaps we can say that a > client MAY send an empty configure element, but that new > implementaitons SHOULD NOT do that. And no configure element can just > be the normal way to do this. I would not make a distinction between old or new implementations. An empty or no configure element means the same thing, and in that case I would not actively promote using an empty element. That said, the whole processing of configure forms has been specified such that fields that are not specified on submission will not alter the existing configuration for that field, so that basically means that you have to implement the empty configure element case anyway. ralphm From ralphm at ik.nu Sun Oct 5 11:50:55 2008 From: ralphm at ik.nu (Ralph Meijer) Date: Sun, 05 Oct 2008 18:50:55 +0200 Subject: [PubSub] pubsub & jabber:x:data In-Reply-To: <20081005145931.0112505d@tiger> References: <20081005145931.0112505d@tiger> Message-ID: <48E8F06F.4000003@ik.nu> On 2008-10-05 14:59, Pavel Simerda wrote: > Hello people, > > I am curious about the use of jabber:x:data in PubSub. I'm for using > jabber:x:data where it fits and where it brings significant value. > > What are the usecases for PubSub node configuration, where we need > special non-standard fields passed to the user (allong with > types/options and such, to fill in. After all, this is what's > jabber:x:data for, isn't it? > > In the PubSub specification I only see examples with a fixed set of > fields and no real jabber:x:data use except useless syntactical sugar > (=complication). The usage of Data Forms in the Publish-Subscribe specification for node configuration and subscription is pretty standard. It is just XEP-0004 (Data Forms) with XEP-0068 for Field Data Standardization. For the latter, we provide context for the field names by setting a FORM_TYPE field. The fields for those contexts are registered with the XMPP Registrar as mentioned in section 16.4. Back when we started defining node and subscription configuration fields, the idea of field context was relatively new, so on top of that each of the registered field names start with 'pubsub#', which is a bit redundant. If you want to add custom fields, you can use the 'x-' prefix as per section 3.4 of XEP-0068. Of course you can also write a new XEP and/or register new fields if standardization might be useful. ralphm From pavlix at pavlix.net Sun Oct 5 14:52:44 2008 From: pavlix at pavlix.net (Pavel Simerda) Date: Sun, 5 Oct 2008 21:52:44 +0200 Subject: [PubSub] pubsub & jabber:x:data In-Reply-To: <48E8F06F.4000003@ik.nu> References: <20081005145931.0112505d@tiger> <48E8F06F.4000003@ik.nu> Message-ID: <20081005215244.02432831@tiger> I'm afraid this does not fully answer my question. On Sun, 05 Oct 2008 18:50:55 +0200 Ralph Meijer wrote: > On 2008-10-05 14:59, Pavel Simerda wrote: > > Hello people, > > > > I am curious about the use of jabber:x:data in PubSub. I'm for using > > jabber:x:data where it fits and where it brings significant value. > > > > What are the usecases for PubSub node configuration, where we need > > special non-standard fields passed to the user (allong with > > types/options and such, to fill in. After all, this is what's > > jabber:x:data for, isn't it? > > > > In the PubSub specification I only see examples with a fixed set of > > fields and no real jabber:x:data use except useless syntactical > > sugar (=complication). > > The usage of Data Forms in the Publish-Subscribe specification for > node configuration and subscription is pretty standard. It is just > XEP-0004 (Data Forms) with XEP-0068 for Field Data Standardization. > For the latter, we provide context for the field names by setting a > FORM_TYPE field. The fields for those contexts are registered with > the XMPP Registrar as mentioned in section 16.4. If putting useless syntactic sugar is THE standard... than I believe it should be changed. XMPP is getting bigger and bigger because it solves more complex tasks than before. IMO it is unacceptable to introduce complexity that doesn't bring any advantages. So my question whether this is or isn't the case of PubSub remains. Do we have usecases that show advantages of jabber:x:data or it's just a useless format-in-format encapsulation in this particular use. (Please don't forget that I like jabber:x:data in many places where it has some purpose.) > Back when we started defining node and subscription configuration > fields, the idea of field context was relatively new, so on top of > that each of the registered field names start with 'pubsub#', I understand this is a replacement of XML Namespaces for the Data Forms format. > which > is a bit redundant. If you want to add custom fields, you can use the > 'x-' prefix as per section 3.4 of XEP-0068. This is good for cases you really have forms introduced to users with some level of standardazation... and an additional possibility to parse standardized fields without user's intervention. > Of course you can also > write a new XEP and/or register new fields if standardization might > be useful. You can do this with XML as well as with Data Forms. > ralphm -- Pavel ?imerda Freelancer v oblasti po??ta?ov?ch s?t?, komunikace a bezpe?nosti Web: http://www.pavlix.net/ Jabber & Mail: pavlix(at)pavlix.net OpenID: pavlix.net From ralphm at ik.nu Mon Oct 6 03:50:27 2008 From: ralphm at ik.nu (Ralph Meijer) Date: Mon, 06 Oct 2008 10:50:27 +0200 Subject: [PubSub] pubsub & jabber:x:data In-Reply-To: <20081005215244.02432831@tiger> References: <20081005145931.0112505d@tiger> <48E8F06F.4000003@ik.nu> <20081005215244.02432831@tiger> Message-ID: <48E9D153.9020203@ik.nu> On 2008-10-05 21:52, Pavel Simerda wrote: > I'm afraid this does not fully answer my question. > > [..] > > So my question whether this is or isn't the case of PubSub remains. Do > we have usecases that show advantages of jabber:x:data or it's just a > useless format-in-format encapsulation in this particular use. Ah, I misunderstood your concern. Your question seems to be: why do we use Data Forms instead of Plain Old XML? for configuring nodes and subscriptions? The Publish-Subscribe specification was intended to support a large set of use cases, to be used in both IM context and outside of that. The main reason for choosing Data Forms over plain XML is that configuration forms can be uniformly presented to an end-user, even when new or application-specific fields are added later on. For example, if I add a subscription configuration option to send notifications depending on his selected phases of the moon, I can simply define a new field (x-notify-on-moon-phases), and clients can present this option to the end-user. If we had used plain XML instead, the client application would need to be altered for every such field. That said, we specifically left the option for very specific applications to embed possibly complex XML structures in place of or next to a Data Form. Just don't count on generic clients to be able to be able to deal with that other than ignoring those elements. Of course, this also holds for client applications that don't have a UI. If there are fields in the Data Form the application does not know about, it should ignore them. ralphm From cking at mumbo.ca Mon Oct 6 10:47:11 2008 From: cking at mumbo.ca (Curtis King) Date: Mon, 6 Oct 2008 08:47:11 -0700 Subject: [PubSub] create node with default configuration In-Reply-To: References: <48E443B7.6000104@stpeter.im> <9b58f4550810012219l7f227c0fx2b8e86e82a91959b@mail.gmail.com> <48E4BAD9.6060903@stpeter.im> <48E7705C.9070905@ik.nu> <20081005145146.123fb11e@tiger> Message-ID: On 5-Oct-08, at 7:20 AM, Kevin Smith wrote: > If the intention is for no configure to be the same as empty > configure, why not make that the way forward, instead of SHOULDing the > empty element? +1 ck From pavlix at pavlix.net Tue Oct 7 05:57:04 2008 From: pavlix at pavlix.net (Pavel Simerda) Date: Tue, 7 Oct 2008 12:57:04 +0200 Subject: [PubSub] pubsub & jabber:x:data In-Reply-To: <48E9D153.9020203@ik.nu> References: <20081005145931.0112505d@tiger> <48E8F06F.4000003@ik.nu> <20081005215244.02432831@tiger> <48E9D153.9020203@ik.nu> Message-ID: <20081007125704.4e08de81@tiger> On Mon, 06 Oct 2008 10:50:27 +0200 Ralph Meijer wrote: > On 2008-10-05 21:52, Pavel Simerda wrote: > > I'm afraid this does not fully answer my question. > > > > [..] > > > > So my question whether this is or isn't the case of PubSub remains. > > Do we have usecases that show advantages of jabber:x:data or it's > > just a useless format-in-format encapsulation in this particular > > use. > > Ah, I misunderstood your concern. Your question seems to be: why do > we use Data Forms instead of Plain Old XML? for configuring nodes and > subscriptions? Exactly. > The Publish-Subscribe specification was intended to support a large > set of use cases, to be used in both IM context and outside of that. > The main reason for choosing Data Forms over plain XML is that > configuration forms can be uniformly presented to an end-user, even > when new or application-specific fields are added later on. This is what I wanted to know. > For example, if I add a subscription configuration option to send > notifications depending on his selected phases of the moon, I can > simply define a new field (x-notify-on-moon-phases), and clients can > present this option to the end-user. If we had used plain XML > instead, the client application would need to be altered for every > such field. I take this as a use-case for a custom field presented in a form to the user. > That said, we specifically left the option for very specific > applications to embed possibly complex XML structures in place of or > next to a Data Form. Just don't count on generic clients to be able > to be able to deal with that other than ignoring those elements. Wouldn't it be better if these very specific options are properly namespaced, like we do with other similar structures in XMPP (e.g. non-standard or extension s, PubSub nodes, etc)? > Of course, this also holds for client applications that don't have a > UI. If there are fields in the Data Form the application does not > know about, it should ignore them. Do I understand it correctly that only clients without jabber:x:data user interface should ignore them and other ones should use them and present them in a data form? > > ralphm Pavel -- Pavel ?imerda Freelancer v oblasti po??ta?ov?ch s?t?, komunikace a bezpe?nosti Web: http://www.pavlix.net/ Jabber & Mail: pavlix(at)pavlix.net OpenID: pavlix.net From stpeter at stpeter.im Tue Oct 7 13:17:02 2008 From: stpeter at stpeter.im (Peter Saint-Andre) Date: Tue, 07 Oct 2008 12:17:02 -0600 Subject: [PubSub] two pubsub proposals In-Reply-To: <25359.1221518046.326659@peirce.dave.cridland.net> References: <48CEBB43.4030402@stpeter.im> <25359.1221518046.326659@peirce.dave.cridland.net> Message-ID: <48EBA79E.5080305@stpeter.im> Dave Cridland wrote: > On Mon Sep 15 20:45:07 2008, Peter Saint-Andre wrote: >> A while back I wrote up two pubsub-related proposals but did not >> submit them to the XMPP Council for consideration. Feedback would be >> welcome on this list before I do that. The specs are: >> >> 1. http://www.xmpp.org/extensions/inbox/pubsub-chaining.html >> >> This specification defines a method for chaining pubsub nodes >> together, resulting in lightweight repeaters for pubsub notifications. >> >> > The mitigation for a na?ve repeater seems to involve existing > implementations changing. > > Given this, why not pass-through subscriptions, and simply do "per-pro > subscriptions" - so the repeater node says it'll handle the subscription > it's forwarding. > > I'll try to do a proto-XEP explaining things better tomorrow. OK. >> 2. http://www.xmpp.org/extensions/inbox/pubsub-queueing.html >> >> This specification defines an extension to XMPP publish-subscribe for >> queueing information at a node. >> >> > With this one, as the last, too, please change the example domains to > either a reserved example domain (see Frank's draft) or our traditional > shakespeare.lit - introducing another ad-hoc example domain is a bit > annoying. (Although not wrong; I just prefer either example.org et al or > our tradition). Isn't .tld reserved by Frank Ellerman's draft? I note that .lit is not. But I changed them anyway. > Can we give this one a better introduction? It could well be my utter > lack of formal comp sci, but I didn't realise the queue was consumable > (or, perhaps, items within it are) until section 2.3, which caused much > head scratching. > > A better introduction might help. > > I'm not sure whether lock-and-retract is the right model here, nor what > a client sees if it attempts to obtain a locked item. Perhaps a better > solution would be to model the queue as a collection of two nodes; the > "new item" node and the "pending item" node, such that retreiving an > item from the "new item" node 'gets' it, and retreiving an item from the > pending node 'completes' it. > > It might be as well to have an indicator in the item retreival that the > retrieving client knows the queue protocol, so that na?ve clients merely > investigating queue contents don't inadvertantly consume the queues. > > FInally, I'm a bit worried that the general model involves a > considerable number of messages - perhaps sending out item notifications > in a hunt-group method might be better? I misinterpeted what Joe and I had discussed originally. I've just updated it significantly, so check out version 0.0.2: http://xmpp.org/extensions/inbox/pubsub-queueing.html Peter -- Peter Saint-Andre https://stpeter.im/ From Ovidiu.Craciun at philips.com Tue Oct 7 14:03:09 2008 From: Ovidiu.Craciun at philips.com (Ovidiu Craciun) Date: Tue, 7 Oct 2008 12:03:09 -0700 Subject: [PubSub] two pubsub proposals In-Reply-To: <48EBA79E.5080305@stpeter.im> References: <48CEBB43.4030402@stpeter.im><25359.1221518046.326659@peirce.dave.cridland.net> <48EBA79E.5080305@stpeter.im> Message-ID: <0AA3A2C546487C4A93AE60213FEFF97F01080EE6@SFO00MX1.stentor.com> Reading http://xmpp.org/extensions/inbox/pubsub-queueing.html ... only one queue will be maintained per pub-sub node. In this case it is possible that a subscriber is deleting an item before the service will have a chance to deliver it to all the publishers... this should be avoided somehow. Other questions: if the pub rate is faster than the subscriber's processing rate, the queue will fill up eventually and when a new item is published and the queue is full, is the oldest item sacrificed or the publishing is failing with "queue full" error? how can a subscriber lock an item? When an item is locked, can it be read by another subscriber? (write lock only) O. -----Original Message----- From: pubsub-bounces at xmpp.org [mailto:pubsub-bounces at xmpp.org] On Behalf Of Peter Saint-Andre Sent: Tuesday, October 07, 2008 11:17 AM To: XMPP publish-subscribe and personal eventing Subject: Re: [PubSub] two pubsub proposals Dave Cridland wrote: > On Mon Sep 15 20:45:07 2008, Peter Saint-Andre wrote: >> A while back I wrote up two pubsub-related proposals but did not >> submit them to the XMPP Council for consideration. Feedback would be >> welcome on this list before I do that. The specs are: >> >> 1. http://www.xmpp.org/extensions/inbox/pubsub-chaining.html >> >> This specification defines a method for chaining pubsub nodes >> together, resulting in lightweight repeaters for pubsub notifications. >> >> > The mitigation for a na?ve repeater seems to involve existing > implementations changing. > > Given this, why not pass-through subscriptions, and simply do "per-pro > subscriptions" - so the repeater node says it'll handle the subscription > it's forwarding. > > I'll try to do a proto-XEP explaining things better tomorrow. OK. >> 2. http://www.xmpp.org/extensions/inbox/pubsub-queueing.html >> >> This specification defines an extension to XMPP publish-subscribe for >> queueing information at a node. >> >> > With this one, as the last, too, please change the example domains to > either a reserved example domain (see Frank's draft) or our traditional > shakespeare.lit - introducing another ad-hoc example domain is a bit > annoying. (Although not wrong; I just prefer either example.org et al or > our tradition). Isn't .tld reserved by Frank Ellerman's draft? I note that .lit is not. But I changed them anyway. > Can we give this one a better introduction? It could well be my utter > lack of formal comp sci, but I didn't realise the queue was consumable > (or, perhaps, items within it are) until section 2.3, which caused much > head scratching. > > A better introduction might help. > > I'm not sure whether lock-and-retract is the right model here, nor what > a client sees if it attempts to obtain a locked item. Perhaps a better > solution would be to model the queue as a collection of two nodes; the > "new item" node and the "pending item" node, such that retreiving an > item from the "new item" node 'gets' it, and retreiving an item from the > pending node 'completes' it. > > It might be as well to have an indicator in the item retreival that the > retrieving client knows the queue protocol, so that na?ve clients merely > investigating queue contents don't inadvertantly consume the queues. > > FInally, I'm a bit worried that the general model involves a > considerable number of messages - perhaps sending out item notifications > in a hunt-group method might be better? I misinterpeted what Joe and I had discussed originally. I've just updated it significantly, so check out version 0.0.2: http://xmpp.org/extensions/inbox/pubsub-queueing.html Peter -- Peter Saint-Andre https://stpeter.im/ From stpeter at stpeter.im Tue Oct 7 14:33:36 2008 From: stpeter at stpeter.im (Peter Saint-Andre) Date: Tue, 07 Oct 2008 13:33:36 -0600 Subject: [PubSub] two pubsub proposals In-Reply-To: <0AA3A2C546487C4A93AE60213FEFF97F01080EE6@SFO00MX1.stentor.com> References: <48CEBB43.4030402@stpeter.im><25359.1221518046.326659@peirce.dave.cridland.net> <48EBA79E.5080305@stpeter.im> <0AA3A2C546487C4A93AE60213FEFF97F01080EE6@SFO00MX1.stentor.com> Message-ID: <48EBB990.5050603@stpeter.im> Ovidiu Craciun wrote: > Reading http://xmpp.org/extensions/inbox/pubsub-queueing.html ... Are you reading version 0.0.2? I think that version should answer some of your questions (the model was all wrong in 0.0.1, my fault for not verifying my understanding with my co-author first). /psa From Ovidiu.Craciun at philips.com Tue Oct 7 15:08:42 2008 From: Ovidiu.Craciun at philips.com (Ovidiu Craciun) Date: Tue, 7 Oct 2008 13:08:42 -0700 Subject: [PubSub] two pubsub proposals In-Reply-To: <48EBB990.5050603@stpeter.im> References: <48CEBB43.4030402@stpeter.im><25359.1221518046.326659@peirce.dave.cridland.net> <48EBA79E.5080305@stpeter.im><0AA3A2C546487C4A93AE60213FEFF97F01080EE6@SFO00MX1.stentor.com> <48EBB990.5050603@stpeter.im> Message-ID: <0AA3A2C546487C4A93AE60213FEFF97F01080F16@SFO00MX1.stentor.com> Yes http://xmpp.org/extensions/inbox/pubsub-queueing.html points to version 0.0.2 -----Original Message----- From: pubsub-bounces at xmpp.org [mailto:pubsub-bounces at xmpp.org] On Behalf Of Peter Saint-Andre Sent: Tuesday, October 07, 2008 12:34 PM To: XMPP publish-subscribe and personal eventing Subject: Re: [PubSub] two pubsub proposals Ovidiu Craciun wrote: > Reading http://xmpp.org/extensions/inbox/pubsub-queueing.html ... Are you reading version 0.0.2? I think that version should answer some of your questions (the model was all wrong in 0.0.1, my fault for not verifying my understanding with my co-author first). /psa From jack at chesspark.com Wed Oct 8 00:40:42 2008 From: jack at chesspark.com (Jack Moffitt) Date: Tue, 7 Oct 2008 23:40:42 -0600 Subject: [PubSub] two pubsub proposals In-Reply-To: <48CEBB43.4030402@stpeter.im> References: <48CEBB43.4030402@stpeter.im> Message-ID: <9b58f4550810072240i19022d5dj62fdd24181762155@mail.gmail.com> > A while back I wrote up two pubsub-related proposals but did not submit them > to the XMPP Council for consideration. Feedback would be welcome on this > list before I do that. The specs are: This feedback is slightly late since we are talking about these tomorrow, but better late than never I suppose. > 1. http://www.xmpp.org/extensions/inbox/pubsub-chaining.html > > This specification defines a method for chaining pubsub nodes together, > resulting in lightweight repeaters for pubsub notifications. So I have a number of ideas around this. First, I'm not sure why a foreign service is the base example. Repeating a local node is probably very common idiom, especially in aggregation. Perhaps you are thinking of a separate XEP for aggregation. On the aggregation note, it might be useful to note in events from the aggregated node where the origin node of those events is. Currently only the service name is noted and the original node is lost. An example use case is a public feed from an XMPP microblogging service. Services will want to consume an aggregated feed in some situations, but they would still like to know the origin of all items. As for setting such an aggregated node up via Ad-Hoc Commands, some kind of more generic facility than one-at-a-time will be necessary, like asking or it to aggregate all local nodes which are public publishing into a specific namespace. Or perhaps some way to bulk dump a list of nodes. Right now I'm trying to think of a good way to do this at jabber.org once the microblogging code is a little farther along. Perhaps it is best to put in bulk and one-at-a-time relay commands (which work on any combination of local or remote nodes) and leave enhancements on this up to implementations until some kind of consensus is reached about how best to describe aggregation. > 2. http://www.xmpp.org/extensions/inbox/pubsub-queueing.html > > This specification defines an extension to XMPP publish-subscribe for > queueing information at a node. There still needs to be more description here. For starters, it's not clear what the queue_likely is, although I can guess. I also have no idea what queue_requests accomplishes. Is that the maximum number of locked items? What use case does this variable solve? In general I think queueing is important, and several people have been playing with this that I have talked to. I think we should take a good look at AMQP and Amazon's SQS and make sure that we encapsulate a common use case. On a more technical note, I think the field var attributes should not be prefixed with pubsub#. It is a little redundant as the FORM_TYPE is already specified. I notice that the chaining XEP does not do this, and it reads much nicer. jack. From stpeter at stpeter.im Wed Oct 8 09:53:34 2008 From: stpeter at stpeter.im (Peter Saint-Andre) Date: Wed, 08 Oct 2008 08:53:34 -0600 Subject: [PubSub] two pubsub proposals In-Reply-To: <9b58f4550810072240i19022d5dj62fdd24181762155@mail.gmail.com> References: <48CEBB43.4030402@stpeter.im> <9b58f4550810072240i19022d5dj62fdd24181762155@mail.gmail.com> Message-ID: <48ECC96E.7010803@stpeter.im> Jack Moffitt wrote: >> A while back I wrote up two pubsub-related proposals but did not submit them >> to the XMPP Council for consideration. Feedback would be welcome on this >> list before I do that. The specs are: > > This feedback is slightly late since we are talking about these > tomorrow, but better late than never I suppose. I doubt that the Council will get that far in its meeting today. >> 1. http://www.xmpp.org/extensions/inbox/pubsub-chaining.html >> >> This specification defines a method for chaining pubsub nodes together, >> resulting in lightweight repeaters for pubsub notifications. > > So I have a number of ideas around this. > > First, I'm not sure why a foreign service is the base example. > Repeating a local node is probably very common idiom, especially in > aggregation. Perhaps you are thinking of a separate XEP for > aggregation. It's an example. Chaining could be used locally or remotely. > On the aggregation note, it might be useful to note in events from the > aggregated node where the origin node of those events is. Currently > only the service name is noted and the original node is lost. Yes, we need to capture that. Unfortunately a node is not fully addressable in a JID, so to include the node we'd need to specify an XMPP URI or define some other mechanism for denoting the originator. > An example use case is a public feed from an XMPP microblogging > service. Services will want to consume an aggregated feed in some > situations, but they would still like to know the origin of all items. > As for setting such an aggregated node up via Ad-Hoc Commands, some > kind of more generic facility than one-at-a-time will be necessary, > like asking or it to aggregate all local nodes which are public > publishing into a specific namespace. Or perhaps some way to bulk > dump a list of nodes. > > Right now I'm trying to think of a good way to do this at jabber.org > once the microblogging code is a little farther along. My sense is that this kind of overlay on pubsub would occur via a web interface or somesuch. > Perhaps it is best to put in bulk and one-at-a-time relay commands > (which work on any combination of local or remote nodes) and leave > enhancements on this up to implementations until some kind of > consensus is reached about how best to describe aggregation. It's not clear to me what you mean by "bulk dump a list of nodes". Do you mean, say, being able to subscribe to all originating nodes whose payloads match a certain namespace? (For example: give me data from all the tunes nodes at this domain.) That is interesting, but you might also want keyword matching a la Mimir or the old pubsub.com service. And I think those things may be overlays on top of the raw data streams from all the nodes at a service (or across services). >> 2. http://www.xmpp.org/extensions/inbox/pubsub-queueing.html >> >> This specification defines an extension to XMPP publish-subscribe for >> queueing information at a node. > > There still needs to be more description here. For starters, it's not > clear what the queue_likely is, although I can guess. I also have no > idea what queue_requests accomplishes. Is that the maximum number of > locked items? What use case does this variable solve? I think I removed those -- the version is now 0.0.3 and I think those were in 0.0.1 but not later. I might be wrong, though. > In general I think queueing is important, and several people have been > playing with this that I have talked to. I think we should take a > good look at AMQP and Amazon's SQS and make sure that we encapsulate a > common use case. Agreed. I mostly looked at JMS, but AMQP and SQS would be good points of comparison, as well. > On a more technical note, I think the field var attributes should not > be prefixed with pubsub#. It is a little redundant as the FORM_TYPE > is already specified. I notice that the chaining XEP does not do > this, and it reads much nicer. The pubsub# and muc# field names in XEPs 60 and 45 are evidence of an earlier archeological layer, and I can't even remember why we thought that was a good idea. But all the other fields in the subscribe_options form type start with pubsub# so I was maintaining consistency. Perhaps that was a foolish consistency... Peter -- Peter Saint-Andre https://stpeter.im/ From ovidiu.craciun at philips.com Wed Oct 8 16:15:31 2008 From: ovidiu.craciun at philips.com (Craciun, Ovidiu) Date: Wed, 8 Oct 2008 23:15:31 +0200 Subject: [PubSub] RESENDING: two pubsub proposals References: <48CEBB43.4030402@stpeter.im><25359.1221518046.326659@peirce.dave.cridland.net> <48EBA79E.5080305@stpeter.im><0AA3A2C546487C4A93AE60213FEFF97F01080EE6@SFO00MX1.stentor.com> <48EBB990.5050603@stpeter.im> Message-ID: <4EEF8291677CFF47ACC618F0559E9E0403FC842CB4@NLCLUEXM11.connect1.local> It looks like there's been some problems with my email account. I'll resend this. -----Original Message----- Sent: Wednesday, October 08, 2008 10:50 AM Subject: RE: [PubSub] two pubsub proposals reading version 0.0.3, all the issues I raised in 0.0.2 stay, even if they have answers in 0.0.2 they can be better formulated, more clear, explicit. a) "When an item is unlocked, the service would then send a publish notification to another subscriber according to application-specific logic for determining the "next" subscriber." How would the service know about the "application-specific logic for determining the next subscriber"? As it's looking right now this spec is not a queueing spec, is more like a load balancing spec (which is using a queue) based on the "application-specific logic". b) It is not clear that the locking is done automatically by the service as soon as it determined the subscribers to send the item to. (there's no way as of now to lock an item by a subscriber) c) "If the subscriber that received the item attempts to unlock the item but the item is no longer locked by the subscriber (e.g., because of a race condition or a lost notification)" once an item locked, there should be no way to lose that lock, unless a policy of lock expiration is set; no other reason. O. -----Original Message----- From: pubsub-bounces at xmpp.org [mailto:pubsub-bounces at xmpp.org] On Behalf Of Peter Saint-Andre Sent: Tuesday, October 07, 2008 12:34 PM To: XMPP publish-subscribe and personal eventing Subject: Re: [PubSub] two pubsub proposals Ovidiu Craciun wrote: > Reading http://xmpp.org/extensions/inbox/pubsub-queueing.html ... Are you reading version 0.0.2? I think that version should answer some of your questions (the model was all wrong in 0.0.1, my fault for not verifying my understanding with my co-author first). /psa The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message. From hildjj at gmail.com Wed Oct 8 16:54:32 2008 From: hildjj at gmail.com (Joe Hildebrand) Date: Wed, 8 Oct 2008 15:54:32 -0600 Subject: [PubSub] two pubsub proposals In-Reply-To: <0AA3A2C546487C4A93AE60213FEFF97F01080EE6@SFO00MX1.stentor.com> References: <48CEBB43.4030402@stpeter.im><25359.1221518046.326659@peirce.dave.cridland.net> <48EBA79E.5080305@stpeter.im> <0AA3A2C546487C4A93AE60213FEFF97F01080EE6@SFO00MX1.stentor.com> Message-ID: <5B0D410B-F6F7-48FE-95EA-73F2073F8CFD@gmail.com> On Oct 7, 2008, at 1:03 PM, Ovidiu Craciun wrote: > Reading http://xmpp.org/extensions/inbox/pubsub-queueing.html ... > > only one queue will be maintained per pub-sub node. In this case it > is possible that a subscriber is deleting an item before the service > will have a chance to deliver it to all the publishers... this > should be avoided somehow. You might have meant subscriber instead of publisher. The item will be delivered to exactly one *subscriber*. There is no race. > Other questions: > if the pub rate is faster than the subscriber's processing rate, > the queue will fill up eventually and when a new item is published > and the queue is full, is the oldest item sacrificed or the > publishing is failing with "queue full" error? I imagine "queue full" error back to the publisher. For full marks, we should define an added protocol that notifies publishers when then can start publishing again. > how can a subscriber lock an item? They shouldn't need to do this explicitly. It happens implicitly when they are the subscriber that gets notified for that item. > When an item is locked, can it be read by another subscriber? > (write lock only) Interesting question. It won't be delivered to them, but if they do a get items? It might be easiest to disallow get items for queue'd nodes. From stpeter at stpeter.im Wed Oct 8 17:08:46 2008 From: stpeter at stpeter.im (Peter Saint-Andre) Date: Wed, 08 Oct 2008 16:08:46 -0600 Subject: [PubSub] two pubsub proposals In-Reply-To: <5B0D410B-F6F7-48FE-95EA-73F2073F8CFD@gmail.com> References: <48CEBB43.4030402@stpeter.im><25359.1221518046.326659@peirce.dave.cridland.net> <48EBA79E.5080305@stpeter.im> <0AA3A2C546487C4A93AE60213FEFF97F01080EE6@SFO00MX1.stentor.com> <5B0D410B-F6F7-48FE-95EA-73F2073F8CFD@gmail.com> Message-ID: <48ED2F6E.8070901@stpeter.im> Joe Hildebrand wrote: > On Oct 7, 2008, at 1:03 PM, Ovidiu Craciun wrote: > >> Reading http://xmpp.org/extensions/inbox/pubsub-queueing.html ... >> >> only one queue will be maintained per pub-sub node. In this case it is >> possible that a subscriber is deleting an item before the service will >> have a chance to deliver it to all the publishers... this should be >> avoided somehow. > > You might have meant subscriber instead of publisher. The item will be > delivered to exactly one *subscriber*. There is no race. Yes, sorry, that was wrong in version 0.0.1 so people probably got confused. >> Other questions: >> if the pub rate is faster than the subscriber's processing rate, >> the queue will fill up eventually and when a new item is published and >> the queue is full, is the oldest item sacrificed or the publishing is >> failing with "queue full" error? > > I imagine "queue full" error back to the publisher. For full marks, we > should define an added protocol that notifies publishers when then can > start publishing again. Right, I'll add that error flow. >> how can a subscriber lock an item? > > They shouldn't need to do this explicitly. It happens implicitly when > they are the subscriber that gets notified for that item. +1 >> When an item is locked, can it be read by another subscriber? >> (write lock only) > > Interesting question. It won't be delivered to them, but if they do a > get items? It might be easiest to disallow get items for queue'd nodes. I don't have strong feelings about this, but in general I'd agree that the service would return an error for get-items requests received from entities that are not assigned for that item. Peter -- Peter Saint-Andre https://stpeter.im/ From hildjj at gmail.com Wed Oct 8 17:11:34 2008 From: hildjj at gmail.com (Joe Hildebrand) Date: Wed, 8 Oct 2008 16:11:34 -0600 Subject: [PubSub] RESENDING: two pubsub proposals In-Reply-To: <4EEF8291677CFF47ACC618F0559E9E0403FC842CB4@NLCLUEXM11.connect1.local> References: <48CEBB43.4030402@stpeter.im><25359.1221518046.326659@peirce.dave.cridland.net> <48EBA79E.5080305@stpeter.im><0AA3A2C546487C4A93AE60213FEFF97F01080EE6@SFO00MX1.stentor.com> <48EBB990.5050603@stpeter.im> <4EEF8291677CFF47ACC618F0559E9E0403FC842CB4@NLCLUEXM11.connect1.local> Message-ID: On Oct 8, 2008, at 3:15 PM, Craciun, Ovidiu wrote: > a) "When an item is unlocked, the service would then send a publish > notification to another subscriber according to application-specific > logic for determining the "next" subscriber." > > How would the service know about the "application-specific > logic for determining the next subscriber"? The service *is* the application-specific logic. It doesn't matter what it is. The least-recently notified subscriber is one possibility. > As it's looking right now this spec is not a queueing spec, > is more like a load balancing spec (which is using a queue) based on > the "application-specific logic". What significant properties would a queueing system have that we haven't covered? > b) It is not clear that the locking is done automatically by the > service as soon as it determined the subscribers to send the item > to. (there's no way as of now to lock an item by a subscriber) That should be clarified. > c) "If the subscriber that received the item attempts to unlock the > item but the item is no longer locked by the subscriber (e.g., > because of a race condition or a lost notification)" > > once an item locked, there should be no way to lose that > lock, unless a policy of lock expiration is set; no other reason. Let's say that timeout is the most frequent case, but server shutdown and failures might lead to other possible reasons. What the reason is should be moot to the subscriber, however. From hildjj at gmail.com Wed Oct 8 17:12:51 2008 From: hildjj at gmail.com (Joe Hildebrand) Date: Wed, 8 Oct 2008 16:12:51 -0600 Subject: [PubSub] two pubsub proposals In-Reply-To: <48ED2F6E.8070901@stpeter.im> References: <48CEBB43.4030402@stpeter.im><25359.1221518046.326659@peirce.dave.cridland.net> <48EBA79E.5080305@stpeter.im> <0AA3A2C546487C4A93AE60213FEFF97F01080EE6@SFO00MX1.stentor.com> <5B0D410B-F6F7-48FE-95EA-73F2073F8CFD@gmail.com> <48ED2F6E.8070901@stpeter.im> Message-ID: <519F9B07-7A98-40EE-8A7C-0E2A563BEE42@gmail.com> On Oct 8, 2008, at 4:08 PM, Peter Saint-Andre wrote: > I don't have strong feelings about this, but in general I'd agree that > the service would return an error for get-items requests received from > entities that are not assigned for that item. I see two more possibilities: - returns the list of items that have been sent to me - returns the list of items that have not been sent to anyone From stpeter at stpeter.im Wed Oct 8 17:15:20 2008 From: stpeter at stpeter.im (Peter Saint-Andre) Date: Wed, 08 Oct 2008 16:15:20 -0600 Subject: [PubSub] two pubsub proposals In-Reply-To: <519F9B07-7A98-40EE-8A7C-0E2A563BEE42@gmail.com> References: <48CEBB43.4030402@stpeter.im><25359.1221518046.326659@peirce.dave.cridland.net> <48EBA79E.5080305@stpeter.im> <0AA3A2C546487C4A93AE60213FEFF97F01080EE6@SFO00MX1.stentor.com> <5B0D410B-F6F7-48FE-95EA-73F2073F8CFD@gmail.com> <48ED2F6E.8070901@stpeter.im> <519F9B07-7A98-40EE-8A7C-0E2A563BEE42@gmail.com> Message-ID: <48ED30F8.20800@stpeter.im> Joe Hildebrand wrote: > > On Oct 8, 2008, at 4:08 PM, Peter Saint-Andre wrote: >> I don't have strong feelings about this, but in general I'd agree that >> the service would return an error for get-items requests received from >> entities that are not assigned for that item. > > I see two more possibilities: > - returns the list of items that have been sent to me > - returns the list of items that have not been sent to anyone I like the last one. "Hey, got any work for me to do?" From ovidiu.craciun at philips.com Wed Oct 8 18:08:03 2008 From: ovidiu.craciun at philips.com (Craciun, Ovidiu) Date: Thu, 9 Oct 2008 01:08:03 +0200 Subject: [PubSub] RESENDING: two pubsub proposals In-Reply-To: References: <48CEBB43.4030402@stpeter.im><25359.1221518046.326659@peirce.dav e.cridland.net><48EBA79E.5080305@stpeter.im><0AA3A2C546487C4A93AE60213FEFF9 7F01080EE6@SFO00MX1.stentor.com><48EBB990.5050603@stpeter.im><4EEF8291677CF F47ACC618F0559E9E0403FC842CB4@NLCLUEXM11.connect1.local> Message-ID: <4EEF8291677CFF47ACC618F0559E9E0403FC842D05@NLCLUEXM11.connect1.local> Inline. -----Original Message----- From: pubsub-bounces at xmpp.org [mailto:pubsub-bounces at xmpp.org] On Behalf Of Joe Hildebrand Sent: Wednesday, October 08, 2008 3:12 PM To: XMPP publish-subscribe and personal eventing Subject: Re: [PubSub] RESENDING: two pubsub proposals On Oct 8, 2008, at 3:15 PM, Craciun, Ovidiu wrote: > a) "When an item is unlocked, the service would then send a publish > notification to another subscriber according to application-specific > logic for determining the "next" subscriber." > > How would the service know about the "application-specific > logic for determining the next subscriber"? The service *is* the application-specific logic. It doesn't matter what it is. The least-recently notified subscriber is one possibility. [O.] I believe it is important how the next subscriber is picked; round-robin, or always the first subscriber not busy, or how you said it, or sent to all subscribers... or picked randomly from the list of subscribers that have presence "ready to process", these are all valid ways to do it; and you'll find use cases for all of them; hardcoding one would meet the expectations of only a few use cases. having a "dispatch behavior" property at queue-node level that specify how the queue-node will behave can suffice a few common use cases. > As it's looking right now this spec is not a queueing spec, > is more like a load balancing spec (which is using a queue) based on > the "application-specific logic". What significant properties would a queueing system have that we haven't covered? [O.] it is not clear to me yet, because the document is not clearly saying it, what kind of the queue system this wants to be, hence my questions/comments. if this queueing system is a LIFO then it should allow *multiple users* to - insert item in queue, (add item at the tail) - read item from queue with lock, (read from head without remove and lock the item) - ream item form queue without lock, (read from head without remove and not lock the item) - delete item from the queue, (remove from head only if I hold a lock or there's no lock on the item and return the item) - get all items (getall); nice to have - get queue length (how many items in the queue) The rules of lock are the same that govern any generic/simple lock, once I hold one lock, I can lock again (without error), I can un/lock, I don't lose the lock until I unlock or locked expired or locked was released by a hire authority/admin (for various reasons). Anybody can read but can't lock an already locked item. Nobody can do any other operation on an item locked by someone else. > b) It is not clear that the locking is done automatically by the > service as soon as it determined the subscribers to send the item > to. (there's no way as of now to lock an item by a subscriber) That should be clarified. > c) "If the subscriber that received the item attempts to unlock the > item but the item is no longer locked by the subscriber (e.g., > because of a race condition or a lost notification)" > > once an item locked, there should be no way to lose that > lock, unless a policy of lock expiration is set; no other reason. Let's say that timeout is the most frequent case, but server shutdown and failures might lead to other possible reasons. What the reason is should be moot to the subscriber, however. [O.] server shutdown should not lose the locking state of a node; failures should not lose the lock for a node, failing to lock will of course not lock, but once I acquired a lock it should be mine no matter what or until my lock expires if there's such a policy. Of course, admin should be capable of releasing locks that are stuck if there's no expiration policy on the lock. You're saying if my server is restarted, then I lose the state of all my locks? I don't see this as acceptable... The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message. From hildjj at gmail.com Wed Oct 8 18:15:18 2008 From: hildjj at gmail.com (Joe Hildebrand) Date: Wed, 8 Oct 2008 17:15:18 -0600 Subject: [PubSub] two pubsub proposals In-Reply-To: <48ED30F8.20800@stpeter.im> References: <48CEBB43.4030402@stpeter.im><25359.1221518046.326659@peirce.dave.cridland.net> <48EBA79E.5080305@stpeter.im> <0AA3A2C546487C4A93AE60213FEFF97F01080EE6@SFO00MX1.stentor.com> <5B0D410B-F6F7-48FE-95EA-73F2073F8CFD@gmail.com> <48ED2F6E.8070901@stpeter.im> <519F9B07-7A98-40EE-8A7C-0E2A563BEE42@gmail.com> <48ED30F8.20800@stpeter.im> Message-ID: On Oct 8, 2008, at 4:15 PM, Peter Saint-Andre wrote: > Joe Hildebrand wrote: >> - returns the list of items that have not been sent to anyone > > I like the last one. "Hey, got any work for me to do?" In which case, getItem either works (and locks as a side-effect) or fails with forbidden or no-such-item. From hildjj at gmail.com Wed Oct 8 18:26:36 2008 From: hildjj at gmail.com (Joe Hildebrand) Date: Wed, 8 Oct 2008 17:26:36 -0600 Subject: [PubSub] RESENDING: two pubsub proposals In-Reply-To: <4EEF8291677CFF47ACC618F0559E9E0403FC842D05@NLCLUEXM11.connect1.local> References: <48CEBB43.4030402@stpeter.im><25359.1221518046.326659@peirce.dav e.cridland.net><48EBA79E.5080305@stpeter.im><0AA3A2C546487C4A93AE60213FEFF9 7F01080EE6@SFO00MX1.stentor.com><48EBB990.5050603@stpeter.im><4EEF8291677CF F47ACC618F0559E9E0403FC842CB4@NLCLUEXM11.connect1.local> <4EEF8291677CFF47ACC618F0559E9E0403FC842D05@NLCLUEXM11.connect1.local> Message-ID: <7CF7CDE4-7149-4DA1-9010-04674DED2DFA@gmail.com> On Oct 8, 2008, at 5:08 PM, Craciun, Ovidiu wrote: > [O.] I believe it is important how the next subscriber is picked; > round-robin, > or always the first subscriber not busy, or how you said it, or sent > to all > subscribers... or picked randomly from the list of subscribers that > have > presence "ready to process", these are all valid ways to do it; and > you'll > find use cases for all of them; hardcoding one would meet the > expectations > of only a few use cases. having a "dispatch behavior" property at > queue-node > level that specify how the queue-node will behave can suffice a few > common > use cases. I agree that the algorithm may be important for certain applications. My experience leads me to believe that these applications are not very common however. I also believe that we do not know the exhaustive list of algorithms. As well, I don't want to have to specify them. If you want to come up with an exhaustive list and specification language for all of them, feel free to propose something. >> As it's looking right now this spec is not a queueing spec, >> is more like a load balancing spec (which is using a queue) based on >> the "application-specific logic". > > What significant properties would a queueing system have that we > haven't covered? > > [O.] it is not clear to me yet, because the document is not clearly > saying it, > what kind of the queue system this wants to be, hence my questions/ > comments. > > if this queueing system is a LIFO then it should allow *multiple > users* to > - insert item in queue, (add item at the tail) That is implicit by not being disallowed. I would be ok with an implementation note to this effect, however. > - read item from queue with lock, (read from head without > remove and > lock the item) An approach for this was suggested in another thread. I would argue that if you use it in a non-administrative application, you application is likely to be incorrect. > - ream item form queue without lock, (read from head without > remove > and not lock the item) Again, this is likely to be an administrative use case. > - delete item from the queue, (remove from head only if I > hold a lock > or there's no lock on the item and return the item) Delete works if i have the lock. I really don't like the semantic of delete returning the item; why don't you just subscribe? > - get all items (getall); nice to have We've got that already in pubsub. What the semantic is in the face of locking should be decided. > - get queue length (how many items in the queue) A stats function I guess. I'd like to tackle that completely separately. > The rules of lock are the same that govern any generic/simple lock, > once I hold > one lock, I can lock again (without error), I can un/lock, I don't > lose the > lock until I unlock or locked expired or locked was released by a > hire authority/admin > (for various reasons). Anybody can read but can't lock an already > locked item. Nobody > can do any other operation on an item locked by someone else. I think you're seeing a use case of clients that want access but don't want to subscribe. I *really* don't understand that use case. Can you describe why you want it more? Note that "completeness" is often in opposition to "simplicity", and we tend to value simplicity. > [O.] server shutdown should not lose the locking state of a node; > failures > should not lose the lock for a node, failing to lock will of course > not lock, > but once I acquired a lock it should be mine no matter what or until > my lock > expires if there's such a policy. Of course, admin should be capable > of > releasing locks that are stuck if there's no expiration policy on > the lock. > You're saying if my server is restarted, then I lose the state of > all my > locks? I don't see this as acceptable... Those are implementation details. There exists at least one case where my lock might have gone away: timeout. The timeout may have fired, and my notification is on the way. As long as there exists at least one reason for the error code, then clients need to be able to handle the error. At that point, it is moot *why* the error occurred. From stpeter at stpeter.im Thu Oct 9 00:01:16 2008 From: stpeter at stpeter.im (Peter Saint-Andre) Date: Wed, 08 Oct 2008 23:01:16 -0600 Subject: [PubSub] create node with default configuration In-Reply-To: <48E8ECC4.9020001@ik.nu> References: <48E443B7.6000104@stpeter.im> <9b58f4550810012219l7f227c0fx2b8e86e82a91959b@mail.gmail.com> <48E4BAD9.6060903@stpeter.im> <48E7705C.9070905@ik.nu> <20081005145146.123fb11e@tiger> <48E8CD66.50108@stpeter.im> <9b58f4550810050826u37d98771la1f5d0098942e1bf@mail.gmail.com> <48E8ECC4.9020001@ik.nu> Message-ID: <48ED901C.9050802@stpeter.im> Ralph Meijer wrote: > On 2008-10-05 17:26, Jack Moffitt wrote: >>>> If the intention is for no configure to be the same as empty >>>> configure, why not make that the way forward, instead of SHOULDing the >>>> empty element? >>> That's what I was trying to say in my previous message. :) >> >> I think that's what I'd prefer as well. Perhaps we can say that a >> client MAY send an empty configure element, but that new >> implementaitons SHOULD NOT do that. And no configure element can just >> be the normal way to do this. > > I would not make a distinction between old or new implementations. An > empty or no configure element means the same thing, and in that case I > would not actively promote using an empty element. Agreed, that's not necessary. > That said, the whole processing of configure forms has been specified > such that fields that are not specified on submission will not alter the > existing configuration for that field, so that basically means that you > have to implement the empty configure element case anyway. Correct. I'll update XEP-0060 along these lines soon. Peter -- Peter Saint-Andre https://stpeter.im/ From stpeter at stpeter.im Thu Oct 9 00:09:41 2008 From: stpeter at stpeter.im (Peter Saint-Andre) Date: Wed, 08 Oct 2008 23:09:41 -0600 Subject: [PubSub] pubsub & jabber:x:data In-Reply-To: <20081007125704.4e08de81@tiger> References: <20081005145931.0112505d@tiger> <48E8F06F.4000003@ik.nu> <20081005215244.02432831@tiger> <48E9D153.9020203@ik.nu> <20081007125704.4e08de81@tiger> Message-ID: <48ED9215.7030107@stpeter.im> Pavel Simerda wrote: > On Mon, 06 Oct 2008 10:50:27 +0200 > Ralph Meijer wrote: > >> That said, we specifically left the option for very specific >> applications to embed possibly complex XML structures in place of or >> next to a Data Form. Just don't count on generic clients to be able >> to be able to deal with that other than ignoring those elements. > > Wouldn't it be better if these very specific options are properly > namespaced, like we do with other similar structures in XMPP > (e.g. non-standard or extension s, PubSub nodes, etc)? Better esthetically? As far as I can see, this is a protocol beautification project. Nice, but not necessary. Peter -- Peter Saint-Andre https://stpeter.im/ From pavlix at pavlix.net Thu Oct 9 06:56:53 2008 From: pavlix at pavlix.net (Pavel Simerda) Date: Thu, 9 Oct 2008 13:56:53 +0200 Subject: [PubSub] pubsub & jabber:x:data In-Reply-To: <48ED9215.7030107@stpeter.im> References: <20081005145931.0112505d@tiger> <48E8F06F.4000003@ik.nu> <20081005215244.02432831@tiger> <48E9D153.9020203@ik.nu> <20081007125704.4e08de81@tiger> <48ED9215.7030107@stpeter.im> Message-ID: <20081009135653.03084c2a@tiger> On Wed, 08 Oct 2008 23:09:41 -0600 Peter Saint-Andre wrote: > Pavel Simerda wrote: > > On Mon, 06 Oct 2008 10:50:27 +0200 > > Ralph Meijer wrote: > > > >> That said, we specifically left the option for very specific > >> applications to embed possibly complex XML structures in place of > >> or next to a Data Form. Just don't count on generic clients to be > >> able to be able to deal with that other than ignoring those > >> elements. > > > > Wouldn't it be better if these very specific options are properly > > namespaced, like we do with other similar structures in XMPP > > (e.g. non-standard or extension s, PubSub nodes, etc)? > > Better esthetically? As far as I can see, this is a protocol > beautification project. Nice, but not necessary. > > Peter Ah, sorry for that. Don't forget you're speaking with an idealist :). This time I believe it will have future practical impact. Some people were talking about combining multiple groups of fields or introducing custom fields. Especially one use case comes to my mind: you want to add a new field with specialized handling in your own client. You would like the client to recognize this field on any future installation of your service. But it should still be basically handled by other clients. Okay, leaving it as a not-so-important issue. Pavel -- Pavel ?imerda Freelancer v oblasti po??ta?ov?ch s?t?, komunikace a bezpe?nosti Web: http://www.pavlix.net/ Jabber & Mail: pavlix(at)pavlix.net OpenID: pavlix.net From ovidiu.craciun at philips.com Thu Oct 9 12:54:10 2008 From: ovidiu.craciun at philips.com (Craciun, Ovidiu) Date: Thu, 9 Oct 2008 19:54:10 +0200 Subject: [PubSub] RESENDING: two pubsub proposals In-Reply-To: <7CF7CDE4-7149-4DA1-9010-04674DED2DFA@gmail.com> References: <48CEBB43.4030402@stpeter.im><25359.1221518046.326659@peirce.dav e.cridland.net><48EBA79E.5080305@stpeter.im><0AA3A2C546487C4A93AE60213FEFF9 7F01080EE6@SFO00MX1.stentor.com><48EBB990.5050603@stpeter.im><4EEF8291677CF F47ACC618F0559E9E0403FC842CB4@NLCLUEXM11.connect1.local><4EEF8291677CFF47ACC618F0559E9E0403FC842D05@NLCLUEXM11.connect1.local> <7CF7CDE4-7149-4DA1-9010-04674DED2DFA@gmail.com> Message-ID: <4EEF8291677CFF47ACC618F0559E9E0403FC905125@NLCLUEXM11.connect1.local> This is getting clearer, for me at least: An user creates the queue pub-sub node, he's automatically the owner. Because it is a pub-sub node all the pub-sub rules apply (owner can manage affiliations, subscribing modes setup for the node are supported, etc.) Any user other than the owner that has sufficient rights/privileges can publish items to the queue pub-sub node (generic pub-sub behavior). Any user that has the sufficient rights/privileges to subscribe can subscribe to the queue pub-sub node (generic pub-sub behavior). Up to here everything is clear. From here is getting interesting... Any user that has subscribed successfully *can* receive notifications when items are published to the queue pub-sub nodes; this is in contrast to the pub-sub non-queue nodes where *all* the subscribers get notifications of new published items in the node (conditioned by the off-line presence and if the presence mode is set for the node); so who gets the notification and who doesn't? this is not clear. There can be many ways in which this can be specified, who will be notified, and choosing only one (not matter which) will make happy some use cases and unhappy others. If you offer the options I said below, you allow all the subscribers to get notification for new published items, then first come wins, gets the item out of the queue and the next subscriber will get the next item from the queue and so on until there's no more items in the queue or no more subscribers available (non-busy) to read items from queue. I see now there's no need to explicitly lock. Once an item delivered to a subscriber, the item is out of the queue, so next getitem request from a subscriber will return the next available item in the queue if any. The getitem request with itemID, for queue is not applicable then...? O. -----Original Message----- From: pubsub-bounces at xmpp.org [mailto:pubsub-bounces at xmpp.org] On Behalf Of Joe Hildebrand Sent: Wednesday, October 08, 2008 4:27 PM To: XMPP publish-subscribe and personal eventing Subject: Re: [PubSub] RESENDING: two pubsub proposals On Oct 8, 2008, at 5:08 PM, Craciun, Ovidiu wrote: > [O.] I believe it is important how the next subscriber is picked; > round-robin, > or always the first subscriber not busy, or how you said it, or sent > to all > subscribers... or picked randomly from the list of subscribers that > have > presence "ready to process", these are all valid ways to do it; and > you'll > find use cases for all of them; hardcoding one would meet the > expectations > of only a few use cases. having a "dispatch behavior" property at > queue-node > level that specify how the queue-node will behave can suffice a few > common > use cases. I agree that the algorithm may be important for certain applications. My experience leads me to believe that these applications are not very common however. I also believe that we do not know the exhaustive list of algorithms. As well, I don't want to have to specify them. If you want to come up with an exhaustive list and specification language for all of them, feel free to propose something. >> As it's looking right now this spec is not a queueing spec, >> is more like a load balancing spec (which is using a queue) based on >> the "application-specific logic". > > What significant properties would a queueing system have that we > haven't covered? > > [O.] it is not clear to me yet, because the document is not clearly > saying it, > what kind of the queue system this wants to be, hence my questions/ > comments. > > if this queueing system is a LIFO then it should allow *multiple > users* to > - insert item in queue, (add item at the tail) That is implicit by not being disallowed. I would be ok with an implementation note to this effect, however. > - read item from queue with lock, (read from head without > remove and > lock the item) An approach for this was suggested in another thread. I would argue that if you use it in a non-administrative application, you application is likely to be incorrect. > - ream item form queue without lock, (read from head without > remove > and not lock the item) Again, this is likely to be an administrative use case. > - delete item from the queue, (remove from head only if I > hold a lock > or there's no lock on the item and return the item) Delete works if i have the lock. I really don't like the semantic of delete returning the item; why don't you just subscribe? > - get all items (getall); nice to have We've got that already in pubsub. What the semantic is in the face of locking should be decided. > - get queue length (how many items in the queue) A stats function I guess. I'd like to tackle that completely separately. > The rules of lock are the same that govern any generic/simple lock, > once I hold > one lock, I can lock again (without error), I can un/lock, I don't > lose the > lock until I unlock or locked expired or locked was released by a > hire authority/admin > (for various reasons). Anybody can read but can't lock an already > locked item. Nobody > can do any other operation on an item locked by someone else. I think you're seeing a use case of clients that want access but don't want to subscribe. I *really* don't understand that use case. Can you describe why you want it more? Note that "completeness" is often in opposition to "simplicity", and we tend to value simplicity. > [O.] server shutdown should not lose the locking state of a node; > failures > should not lose the lock for a node, failing to lock will of course > not lock, > but once I acquired a lock it should be mine no matter what or until > my lock > expires if there's such a policy. Of course, admin should be capable > of > releasing locks that are stuck if there's no expiration policy on > the lock. > You're saying if my server is restarted, then I lose the state of > all my > locks? I don't see this as acceptable... Those are implementation details. There exists at least one case where my lock might have gone away: timeout. The timeout may have fired, and my notification is on the way. As long as there exists at least one reason for the error code, then clients need to be able to handle the error. At that point, it is moot *why* the error occurred. The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message. From stpeter at stpeter.im Wed Oct 15 17:23:16 2008 From: stpeter at stpeter.im (Peter Saint-Andre) Date: Wed, 15 Oct 2008 16:23:16 -0600 Subject: [PubSub] RESENDING: two pubsub proposals In-Reply-To: <4EEF8291677CFF47ACC618F0559E9E0403FC905125@NLCLUEXM11.connect1.local> References: <48CEBB43.4030402@stpeter.im><25359.1221518046.326659@peirce.dav e.cridland.net><48EBA79E.5080305@stpeter.im><0AA3A2C546487C4A93AE60213FEFF9 7F01080EE6@SFO00MX1.stentor.com><48EBB990.5050603@stpeter.im><4EEF8291677CF F47ACC618F0559E9E0403FC842CB4@NLCLUEXM11.connect1.local><4EEF8291677CFF47ACC618F0559E9E0403FC842D05@NLCLUEXM11.connect1.local> <7CF7CDE4-7149-4DA1-9010-04674DED2DFA@gmail.com> <4EEF8291677CFF47ACC618F0559E9E0403FC905125@NLCLUEXM11.connect1.local> Message-ID: <48F66D54.6040609@stpeter.im> Craciun, Ovidiu wrote: > This is getting clearer, for me at least: > > An user creates the queue pub-sub node, he's automatically the owner. > Because it is a pub-sub node all the pub-sub rules apply (owner can > manage affiliations, subscribing modes setup for the node are > supported, etc.) Any user other than the owner that has sufficient > rights/privileges can publish items to the queue pub-sub node > (generic pub-sub behavior). Any user that has the sufficient > rights/privileges to subscribe can subscribe to the queue pub-sub > node (generic pub-sub behavior). > > Up to here everything is clear. From here is getting interesting... > > Any user that has subscribed successfully *can* receive notifications > when items are published to the queue pub-sub nodes; this is in > contrast to the pub-sub non-queue nodes where *all* the subscribers > get notifications of new published items in the node (conditioned by > the off-line presence and if the presence mode is set for the node); > so who gets the notification and who doesn't? this is not clear. Because it's application-specific. How you use pubsub queueing is up to you. This mechanism simply gives you the ability to queue the items, it doesn't tell you what algorithm you use in deciding who gets any given item. > There can be many ways in which this can be specified, who will be > notified, and choosing only one (not matter which) will make happy > some use cases and unhappy others. If you offer the options I said > below, you allow all the subscribers to get notification for new > published items, then first come wins, gets the item out of the queue > and the next subscriber will get the next item from the queue and so > on until there's no more items in the queue or no more subscribers > available (non-busy) to read items from queue. Sure, "push to all and first retriever locks" the item is one model. That's what we had in version 0.0.1, but Joe and I had a disagreement about that and in fact what he had in mind is "push to one". > I see now there's no need to explicitly lock. Right. Locking either happens on retrieval (in "push to all") or on publish (in "push to one"). > Once an item delivered to a subscriber, the item is out of the queue, > so next getitem request from a subscriber will return the next > available item in the queue if any. The getitem request with itemID, > for queue is not applicable then...? Right. There's no need for that in the "push to one" model (although I need to add an example of that to XEP-0060 for completeness). /psa