[standards-jig] No Subject....

mlin at mlin.net mlin at mlin.net
Wed Jan 30 23:21:11 UTC 2002

DW: Right - however, the framing implies that the XML is not also being checked 
for well-formedness. Right now, if you send badly-formed XML, the stream protocol
closes its document to you, indicating that there was an error on data it
received, and closes the socket so that data cannot be sent to it anymore.
With the data not being checked for well-formedness by the server (and just
being routed), the data may not get properly interpreted until some point
later in the system.

ML: The purpose of byte length framing is not to entirely remove all XML parsing. Well-formedness checking is easier than building a DOM. Furthermore, byte length framing allows well-formedness checking to be done separately from I/O, in another thread or on another CPU.  Thus, I'll agree that well-formedness checking is necessary on all nodes, which is not stated in the JEP; however, there are still performance benefits to be gained, and there are further code simplicity advantages...

DW: So, at some point later in the route, this non-well-formed data may hit a
node which does not support framing, and instead uses a normal XML parser.
This parser will stop parsing on the error. So, I could frame invalid packets
and send them over the wire to disrupt any clients or components using a
compliant XML parser. Because of this, client and server connections cannot
be trusted to provide well-formed XML, and the XML must be fully parsed whether
or not framing information is present.

ML: ...the critical difference is that it is much easier to fully parse the packet with framing data available.

Most XML parsing suites are not good at handling partially-received information. Any SAX implementation I have used, where they support partially received data at all, is based on a "pull" model, where the parser reads data from a stream. This means that you have to block if data hasn't come in over the network yet, which in actual Jabber implementations has resulted in all sorts of nasty hacks to properly process the stream. Most of these really just come down to checking for well-formedness before sending data to the parser.

With framing information available you can buffer the element in a statically-sized buffer and then parse (or check well-formedness) in one go. In actual implementation, this "push" model is much easier and more efficient; it saves memory reallocation and copying, and is not dependent on how thoughtful the authors of your XML parsing suite were. You furthermore get at least the possibility of knowing the size of packets ahead of time and rejecting them if they are too large.

DW:  So, the data within a frame will need to be manipulated when sent from clients, 
which probably negates any benefit from having clients frame the data.

ML: There are still real performance benefits. The server has to rewrite one attribute of one tag. It can do this without building a DOM for the whole packet.

In the end I'll agree the performance difference for instant messaging applications is not going to matter because the payload is so small and flat. I think there will be a big difference when we start transporting large, complex, and server-opaque XML payloads, like those for web services or peer-to-peer applications. 

DW: I'm just indicating that for data to be routed correctly in a namespace-correct 
server, each packet may need to be rewritten on each hop.

ML: I am not directly familiar with the component and S2S protocols so I will claim ignorance on those. As for rewriting the packet, that is fine. As long as the default namespace for the XML stream is jabber:client, the server will only ever have to rewrite the attributes of the start tag of the element, which is a simple problem that does not require parsing the entire payload or even knowing where the end tag is. If the S2S and component protocols do not use jabber:client as the default namespace, then perhaps they should.

Also, it is important to remember that even if you do end up building a DOM for the whole element anyway, which is undoubtedly how initial implementations would work, you still get code simplicity advantages because of the new framed XML parsing semantics.

One other note. I have probably just slightly misinterpreted, so please excuse me if that is the case. But I would just clarify that the xmlns="jabber:client" attribute, or xmlns="jabber:iq:roster" or whatever, is not just an opaque attribute. It specifies a default namespace that the element in question and its child elements are to adopt if another namespace prefix is not explicitly specified.

DW: The solution I see for this are to make one document inputted to the system
actually represent a document received by another endpoint in the system.
The two ways I can think of doing (as I said previously) are:
- to send self-contained documents between points, rather than first-level
child elements of the root. You would then frame each document.
- to send a document between endpoints, framed by something like BEEP.

ML: JEP-0017 is definitely a compromise solution. The key motivations besides efficiency are that it is backwards compatible and easy to implement. This is not the case with sending self-contained documents, or with switching everything to BEEP. Either of these I agree would be a cleaner and probably better solution, but they either make a lot of people rewrite a lot of code, or require servers to be able to speak more protocols. I don't think either is desirable in the near term when there is a simpler (if hackish) solution available.



More information about the Standards mailing list