[interop] outcomes

[Gaston Dombiak]

Hey Peter,

I like your ideas. :) 

I was thinking that having a battery of automated test cases to run
would be a good way to optimize the time. We have like 140 automated
test cases in Smack but most of them are related to JEPs testing while
30 aprox. are related to XMPP Core and IM & Presence spec. I created a
list (excel file) with the 30 automated test cases and sent it to Peter.
If anyone of you have more automated test cases it would be nice to
complete the list so we can quickly organize them and learn more about
the current coverage.

Once we have the list of automated test cases we can continue filling
out the excel file with manual test cases. In time we can create
automated test cases for those manual test cases. The final output of
this process would be a list of automated (if possible) test cases that
each server should pass to be considered XMPP compliant.

Just an idea.

Regards,

  -- Gato


> -----Original Message-----
> From: interop-bounces at jabber.org [mailto:interop-bounces at jabber.org]
On
> Behalf Of Peter Saint-Andre
> Sent: Wednesday, July 19, 2006 7:22 PM
> To: interop at jabber.org
> Subject: [interop] outcomes
> 
> It's clear to me that we won't get the chance to test every last
little
> use case and protocol scenario in Portland, even for RFCs 3920 and
3921
> (e.g., we haven't issued X.509 certificates for test servers, let
alone
> for clients). While we'll try to test as much as we can, I'd also like
> to start planning how we can automate and improve some of our testing
in
> the future. That might include:
> 
> 1. Public test servers pointed to via DNS (ejabberd.xmpp.com and so
on).
> 
> 2. CA-issued X.509 certificates for both servers and end users (I am
> working to set up the JSF as an intermediate certification authority).
> 
> 3. A repository of use cases and perhaps even test scripts.
> 
> 4. The beginnings of a test harness.
> 
> 5. A schedule for future "virtual" test events or test periods (e.g.,
> revisit the use cases in October).
> 
> Some of these will help me write the Internet-Draft describing our
> interoperability experience (not limited to our experience at the
> interop event), which we'll use as input for advancing the XMPP RFCs
to
> Draft Standard. Others will help us in the future as we do more
testing
> (which I hope will become more natural and straightforward as a result
> of our meeting and discussions in Portland).
> 
> I'd also like to discuss some of the proposed changes to and best
> practices related to the RFCs, since they have an impact on our RFC
> revision process. That would include:
> 
> JEP-0157: Contact Addresses for XMPP Services
>           http://www.jabber.org/jeps/jep-0157.html
> 
> JEP-0170: Recommended Order of Stream Feature Negotiation
>           http://www.jabber.org/jeps/jep-0170.html
> 
> JEP-0175: Best Practices for Use of SASL ANONYMOUS
>           http://www.jabber.org/jeps/jep-0175.html
> 
> JEP-0178: Best Practices for Use of SASL EXTERNAL
>           http://www.jabber.org/jeps/jep-0178.html
> 
> JEP-0185: Dialback Key Generation and Validation
>           http://www.jabber.org/jeps/jep-0185.html
> 
> Plus of course meeting each other in person and getting to know each
> other better.
> 
> Finally, this is a bit of a dry run for holding more official
developer
> conferences (XMPPCon?), which I hope to start doing next year. Given
how
> disorganized I've been about this one, I conclude that we may need
some
> serious help with holding developer conferences. :-)
> 
> Anything else?
> 
> Peter
> 
> --
> Peter Saint-Andre
> Jabber Software Foundation
> http://www.jabber.org/people/stpeter.shtml