[standards-jig] DTCP again

[Justin Karneges]

Hi all,

Alright, let me consolidate the relevant ideas over the past day.  So far this 
seems to be an extremely hot topic, and everyone has come out of the woodwork 
finally.  This is good, as it is what "Last Call" is for, right?

The big topics at hand seem to be:
  1) Why do we need a custom stream protocol when we have HTTP?
  2) Why a proprietary handshake when we could be using HTTP? 
  3) What about proxy use?

Let me try and address them:

1) Why do we need a custom stream protocol when we have HTTP?
--------------------------
This is likely the most important argument to resolve, since the viewpoint 
locks out the use of DTCP, JOBS, IBB, or anything anyone can ever conjure up 
that is not HTTP.

First, there are other uses of DTCP besides File Transfer, as described by 
various people in yesterday's discussion.  That alone should warrant a custom 
stream, but let us discuss FT for now.

I will quote Matthias Wimmer:
"Most implementors that want to invent something else don't know 
how easy http can be implemented."

On the contrary, I know that HTTP is not very hard to implement.  As a client 
author, I have written an HTTP server but never made it official, simply 
because I felt there was a better way to do things.  I have since scrapped it 
and written a DTCP server.  Other clients, like Tkabber, implement both DTCP 
and HTTP file transfer.

I think HTTP is still a useful FT method, when you consider the description 
found here:  http://www.jabber.org/protocol/filetransfer.html

That is, a client doesn't implement a server.  It merely needs a way to upload 
using HTTP PUT and to pass a suitable URL to a target client.  Arguably, 
target clients don't even need a download facility in these cases, as the URL 
could simply be passed to an external program for handling.  Minimal effort, 
works all the time.  So where is the problem?

The problem is that this is often a waste of resources, as the file could have 
been sent P2P.  Let me describe the problems with using iq:oob for P2P.

a) iq:oob is not enough of a defined spec to be a useful FT.  There is barely 
a handshake on the Jabber side, and often clients have different behavior.  
HTTP is assumed to be the transfer method, but it doesn't need to be (this 
further emphasizes that iq:oob URLs should be sent directly to external 
programs).  When sending the URL, generally in the format 
http://my_ip_address/resource, only one IP address can be used when you may 
have more than one.  There is no protocol in-place, nor a JEP, describing a 
better iq:oob negotiation.  Why not?

b) HTTP does not have a provision for situations where you are behind NAT but 
your friend is not.  A reverse-connection is needed.

Of course, (a) can be resolved with a JEP, yet no one has stepped up to the 
plate to do so.  Certainly I could have done this, as I take a great interest 
in FT, but I felt it was important to also resolve (b).  What resulted were 
four JEPs (41,46,47,52), described by Peter as a "tetralogy", detailing an 
entirely new File Transfer system that would work in all cases, using the 
most efficient means possible, without any additional changes to the server.  
Especially with DTCP's reverse-connects and IBB, this was to bring Jabber FT 
beyond anything the big boys have.

I'm not out to replace the HTTP transfer method.  As noted above, with the use 
of WebDAV, libs, and external programs, it can be implemented with minimal 
effort.  Thus, I'd like to end this war of HTTP vs non-HTTP.  Let HTTP be 
what it is, and advanced clients can use a more solid solution.

In short, HTTP file transfer is good in a lot of cases, but until we have an 
alternative more suitable for P2P, our transfers will not be as robust as 
that of proprietary IM systems (I know, that was cold).


2) Why a proprietary handshake when we could be using HTTP? 
--------------------------
This issue was brought up be temas, and I will quote him:
"Even if we had to handwrite a basic HTTP library the semantics are more well 
defined in the HTTP headers, and the method system is obvious."

And Matthew Miller:
"... I was asked to ponder converting JOBS' out-of-band
communications from it's "proprietary" format to HTTP.  With some advice
from Thomas Muldowney and Ben Schumacher, here is a quick write-up of
what the OOB would look like: ..."

It is clear, at least from Matthew's idea for JOBS, that using an HTTP 
handshake to authenticate an arbitrary bytestream is indeed possible.  Of 
course, the procedure would only work with a JOBS-aware Jabber client and 
nothing else, but its reliance on standard HTTP headers and responses means 
that existing library code can be used.

While I personally view this usage as unnecessary since I don't think DTCP 
will ever be expanded, I see no reason DTCP could not use a similar handshake 
procedure.  Assuming the council is beyond (1) but stuck on (2), I will 
change DTCP as necessary.


What about proxy use?
--------------------------
Let me quote dizzyd:
"With a little slight of hand we can make it transparent to the Target as to 
whether or not they are actually connecting directly to the Initiator (p2p) 
or to a proxy."

and Ben:
"I have yet to hear a technical reason why JOBS is insufficent, and what DTCP 
gives us that is better. The only complaint I've ever heard from you (or 
anybody) is that its too complex, yet nobody has proposed how to make it 
simpler and still maintain the high level of functionality it provides."

Practically speaking, DTCP works well with no such proxy support.  Client 
implementations surfaced even without my assistance.  Only now am I trying 
hard to seek standardization, as these clients need to remain interoperable.  
Now I have my own implementation as well, and it is solid.  All of my TCP 
worries are over.  I can link two clients together with just a few lines of 
code.

The fact is, no one wanted to implement DSPS and it was rejected by the 
council.  Now we have JOBS, which does near the same thing and I fear may 
face a similar fate with the council for the very same reasons.  No one has 
even tried to implement it in a client, and no server components exist.  Even 
the totally lame IBB has implementations, simply because it is easy and it 
works today!

This is not to say that JOBS is not useful, or that proxying is not needed.  
It's just that client developers have no interest.  Instead, they want 
something that works today.  Is there any other reason DTCP would be at the 
Last Call stage?

Sorry to weasel out of this last question, but IMO, the issue of proxy is 
still a separate matter, and I would like to defer the issue.  DTCP was never 
meant to address it.

-Justin