[Members] interop testing

bear bear42 at gmail.com
Tue Nov 2 06:11:24 CST 2010

On Tue, Nov 2, 2010 at 07:59, Matthew Wild <mwild1 at gmail.com> wrote:
> On 2 November 2010 11:45, Dave Cridland <dave at cridland.net> wrote:
>> On Sat Oct 30 14:17:05 2010, Peter Saint-Andre wrote:
>>> I think it would be good to start preparing better for interop testing
>>> at XSF events. I'm fine at leading discussions about specs, but I'm not
>>> a developer so I don't know quite what we need here. What do you folks
>>> think? How can the XSF help in terms of process and infrastructure?
>> On this note...
>> I'd like to propose that the XSF arranges regular interop events to test and
>> publicly demonstrate interoperability across a range of functionality.
>> Various other organizations are doing the same, at the moment, and these are
>> only involving a small cross-section of the implementations - generally
>> those focussed on a particular industry sector. This might cause a somewhat
>> skewed viewpoint on just how good interoperability *is* in XMPP - I've
>> always assumed it to be excellent, but involvement with some of these
>> specialist events has shaken my confidence, and not done XMPP itself any
>> favours.
>> Therefore, I'm proposing that the XSF should organize an event prior to
>> this; this suggests rather strongly that the event should be conducted
>> online. The things I'd like to concentrate on are the basics - S2S and C2S
>> authentication/security, and MUC. (Yes, this does imply existing
>> implementations failed to interop even here, much to my surprise).
>> In particular:
>> - TLS/X.509 authentication for S2S.
>> - SCRAM (including, ideally, -PLUS) for C2S.
>> - Joining local and remote chatrooms.
>> I also think we could do more, but I'd ideally like to conentrate on just
>> this. Other technologies seem to be easier to do interop testing in somewhat
>> less formal ways, in part because the implementation pools are often
>> smaller.
>> I'm suggesting the first week of December - that is, Monday 6th through to
>> Saturday 11th, inclusive.
>> The way I see it operating is as follows:
>> 1) The XSF sets up a test DNS zone, containing domains for each
>> implementation taking part (this might well be more than one - in Isode's
>> case, I think we'd be likely to put in our current release as well as the
>> forthcoming one).
> We've been here before, but yes :)
>> 2) Participants can arrange times to perform testing. Participants will
>> provide each other with transcripts as needed. Server implementations would
>> be online continuously for the week; the arrangement of times is for the
>> "staff".
> What kind of medium are you thinking...? MUC? Wiki?

i'm thinking low-tech - a combination of wiki (via a moderator) and
mailing list.  A MUC would be useful for any coordination chats, but I
prefer a mailing list as we do have various timezones involved.

>> 3) Two temporary teams need to be formed by the XSF. One administrative, to
>> organize resources such as the DNS, chatrooms, mailing lists, etc. One
>> technical, for arbitration if we actually get an interop failure (and
>> writing up what the correct reading and/or fix the specifications need).
> Administrative would probably be the existing iteam - already handling
> DNS, servers and mailing lists. A technical team for arbitration and
> documentation would have to be led by Peter, I'm sure that's probably
> in the bylaws by now :)

Infra team would definitely be who I would look to for creating the
initial services, but I think whoever is on the Interop Admin team
will need to be able to add/change configs for the services to respond
to issues.

Any Interop issues raised would, IMO, already be handled by the
existing XEP technical discussion folks in jdev - this way questions
about interpretations can be fielded by active participants.

>> 4) The XSF will need to operate a CA for the event. Whilst I've not
>> confirmed this, I think Isode can provide (at least one) copy of our CA
>> software to the XSF for this purpose, which would give somebody a simple
>> (ie, GUI clicky) way of generating X.509 certificates with various
>> combinations of Subject RDNs and Subject Alternative Names.
> Fancy :)

nice! A CA is definitely needed - we found that out at the last OSCon
Summit when we were doing BOSH interop and discovered that we had to
also verify S2S and C2S interop because it just wasn't happening

>> So, questions:
>> a) Are sufficient numbers of members interested in this?
> I am, very much so.
>> b) Are sufficient number of implementors able to commit? (I know many aren't
>> members, but if we can generate a critical mass here, we should be fine in
>> attracting more).
> I am, very much so.

I am also

>> c) Is the Board (presumably) interested in overseeing the organization?
> If I were on the Board, I would be, very much so.

I am now and I would love to help make this happen


bear at xmpp.org (email)
bear42 at gmail.com (xmpp, email)
bear at code-bear.com (xmpp, email)
http://code-bear.com/bearlog (weblog)

PGP Fingerprint = 9996 719F 973D B11B E111  D770 9331 E822 40B3 CD29

More information about the Members mailing list