Encrypted peer-to-peer XML channel
Hi folks, I've been brainstorming a mechanism for offline history transfer (i.e. one client fetches the offline message history from another client). For that, an encrypted direct client-to-client XML channel would be neat. 1. Are direct client-to-client XML channels a thing? I found XEP-0247 [1], which looks alright on first read-through but is deferred since 2010. Does someone more familiar with the topic know why the XEP was abandoned and whether it could realistically be revived or if there is a better alternative now? 2. Are encrypted direct client-to-client channels a thing? There is JET [2], but it seems to focus on key negotiation (which I would do differently) and file transfers, which I don't need either. I don't see how a bidirectional data channel/stream would be encrypted using JET. There's also a XEP called jingle-xtls [3] in the Inbox, but it's even more abandoned than XEP-0247 and also seems to focus mostly on the key negotiation, which again I would do differently. Next to these questions I would be interested in the general sentiment and ideas for workarounds (e.g. "p2p is cursed, just send encrypted stanzas via the server as usual" or "just use a simple binary format and don't bother with a fully fledged XML stream"). Thanks :D Tim [1] https://xmpp.org/extensions/xep-0247.html [2] https://xmpp.org/extensions/xep-0391.html [3] https://xmpp.org/extensions/inbox/jingle-xtls.html
We're not currently doing it but considering to do in the future. Very much likely we'll just do an upload of an encrypted archive via cloud storage. This way it can be interoperable with the likes of Dropbox / gdrive / whatever. On Tue, Jul 16, 2024, 00:51 Tim Henkes <syndace@web.de> wrote:
Hi folks,
I've been brainstorming a mechanism for offline history transfer (i.e. one client fetches the offline message history from another client). For that, an encrypted direct client-to-client XML channel would be neat.
1. Are direct client-to-client XML channels a thing?
I found XEP-0247 [1], which looks alright on first read-through but is deferred since 2010. Does someone more familiar with the topic know why the XEP was abandoned and whether it could realistically be revived or if there is a better alternative now?
2. Are encrypted direct client-to-client channels a thing?
There is JET [2], but it seems to focus on key negotiation (which I would do differently) and file transfers, which I don't need either. I don't see how a bidirectional data channel/stream would be encrypted using JET.
There's also a XEP called jingle-xtls [3] in the Inbox, but it's even more abandoned than XEP-0247 and also seems to focus mostly on the key negotiation, which again I would do differently.
Next to these questions I would be interested in the general sentiment and ideas for workarounds (e.g. "p2p is cursed, just send encrypted stanzas via the server as usual" or "just use a simple binary format and don't bother with a fully fledged XML stream").
Thanks :D
Tim
[1] https://xmpp.org/extensions/xep-0247.html
[2] https://xmpp.org/extensions/xep-0391.html
[3] https://xmpp.org/extensions/inbox/jingle-xtls.html
_______________________________________________ Standards mailing list -- standards@xmpp.org To unsubscribe send an email to standards-leave@xmpp.org
Hi, Your constraints seem rather far fetched. You are offline? What does this mean? in a local network, not connected to the internet? How do you find and connect to another device without server? Why would you need to transfer your history specifically in these conditions? Simply upload to http server encrypted. XML Stream via Server is not a good way to transfer hundreds of megabytes. Is this about the transport method? Or the format? Because interoperable format seems like a big task, transport seems easy. I think defining the transport and where to store is useful short term, then every client can offer their own backup. Its useful independent of the format. im less optimistic for a standardized exchange format. Regards Philipp On Mon, Jul 15, 2024, at 21:55, Andrew Nenakhov wrote:
We're not currently doing it but considering to do in the future. Very much likely we'll just do an upload of an encrypted archive via cloud storage. This way it can be interoperable with the likes of Dropbox / gdrive / whatever.
On Tue, Jul 16, 2024, 00:51 Tim Henkes <syndace@web.de> wrote:
Hi folks,
I've been brainstorming a mechanism for offline history transfer (i.e. one client fetches the offline message history from another client). For that, an encrypted direct client-to-client XML channel would be neat.
1. Are direct client-to-client XML channels a thing?
I found XEP-0247 [1], which looks alright on first read-through but is deferred since 2010. Does someone more familiar with the topic know why the XEP was abandoned and whether it could realistically be revived or if there is a better alternative now?
2. Are encrypted direct client-to-client channels a thing?
There is JET [2], but it seems to focus on key negotiation (which I would do differently) and file transfers, which I don't need either. I don't see how a bidirectional data channel/stream would be encrypted using JET.
There's also a XEP called jingle-xtls [3] in the Inbox, but it's even more abandoned than XEP-0247 and also seems to focus mostly on the key negotiation, which again I would do differently.
Next to these questions I would be interested in the general sentiment and ideas for workarounds (e.g. "p2p is cursed, just send encrypted stanzas via the server as usual" or "just use a simple binary format and don't bother with a fully fledged XML stream").
Thanks :D
Tim
[1] https://xmpp.org/extensions/xep-0247.html
[2] https://xmpp.org/extensions/xep-0391.html
[3] https://xmpp.org/extensions/inbox/jingle-xtls.html
_______________________________________________ Standards mailing list -- standards@xmpp.org To unsubscribe send an email to standards-leave@xmpp.org
Standards mailing list -- standards@xmpp.org To unsubscribe send an email to standards-leave@xmpp.org
Hey,
You are offline? What does this mean? in a local network, not connected to the internet?
No, I'm not offline. I'm fully online and connected to my server as usual. I just think to synchronize offline history, which could be a lot of data and include confidential messages that were initially encrypted, an encrypted direct client-to-client channel would be cool, both for speed and security.
Simply upload to http server encrypted.
That seems inflexible. Would work for the simplest use-case (completely new client wants the full history), but if realistically possible I'd like to be a bit more flexible. I'd also like to avoid that because it would mean that your whole history is now available online in one nice package secured with a single encryption key, so all the effort you put in beforehand with forward-secrecy etc. would be gone.
XML Stream via Server is not a good way to transfer hundreds of megabytes. That's one of the reasons I don't want to go over the server but use a peer-to-peer connection instead :)
Is this about the transport method? Or the format? Because interoperable format seems like a big task, transport seems easy. Hah, I would have expected it the other way around. For the format I could use e.g. MAM, which would give me nice message archive queries and is supported by the clients I would want to target. The transport seems difficult though. I think defining the transport and where to store is useful short term, then every client can offer their own backup. It's not really about a "backup" in that sense. For a backup I would probably do what Andrew suggested and just create a password-protected zip file with my data and upload that somewhere. The use-case I am brainstorming is when you get a new client, but you can't get the history from the server because it either doesn't have complete archives or some stuff was OMEMO-encrypted. So you quickly scan a QR-code on some of your other clients and it transfers the local, plaintext messages over.
Regards Philipp
On Mon, Jul 15, 2024, at 21:55, Andrew Nenakhov wrote:
We're not currently doing it but considering to do in the future. Very much likely we'll just do an upload of an encrypted archive via cloud storage. This way it can be interoperable with the likes of Dropbox / gdrive / whatever.
On Tue, Jul 16, 2024, 00:51 Tim Henkes <syndace@web.de> wrote:
Hi folks,
I've been brainstorming a mechanism for offline history transfer (i.e. one client fetches the offline message history from another client). For that, an encrypted direct client-to-client XML channel would be neat.
1. Are direct client-to-client XML channels a thing?
I found XEP-0247 [1], which looks alright on first read-through but is deferred since 2010. Does someone more familiar with the topic know why the XEP was abandoned and whether it could realistically be revived or if there is a better alternative now?
2. Are encrypted direct client-to-client channels a thing?
There is JET [2], but it seems to focus on key negotiation (which I would do differently) and file transfers, which I don't need either. I don't see how a bidirectional data channel/stream would be encrypted using JET.
There's also a XEP called jingle-xtls [3] in the Inbox, but it's even more abandoned than XEP-0247 and also seems to focus mostly on the key negotiation, which again I would do differently.
Next to these questions I would be interested in the general sentiment and ideas for workarounds (e.g. "p2p is cursed, just send encrypted stanzas via the server as usual" or "just use a simple binary format and don't bother with a fully fledged XML stream").
Thanks :D
Tim
[1] https://xmpp.org/extensions/xep-0247.html
[2] https://xmpp.org/extensions/xep-0391.html
[3] https://xmpp.org/extensions/inbox/jingle-xtls.html
_______________________________________________ Standards mailing list -- standards@xmpp.org To unsubscribe send an email to standards-leave@xmpp.org
_______________________________________________ Standards mailing list -- standards@xmpp.org To unsubscribe send an email to standards-leave@xmpp.org
_______________________________________________ Standards mailing list --standards@xmpp.org To unsubscribe send an email tostandards-leave@xmpp.org
On Tue, 16 Jul 2024 at 02:37, Tim Henkes <syndace@web.de> wrote:
Hey,
You are offline? What does this mean? in a local network, not connected to the internet?
No, I'm not offline. I'm fully online and connected to my server as usual. I just think to synchronize offline history, which could be a lot of data and include confidential messages that were initially encrypted, an encrypted direct client-to-client channel would be cool, both for speed and security.
If I understand you correctly, you are suggesting that both clients should be online while synchronising their history. If we are talking about a protocol that is supposed to work on all kinds of devices, this is a flawed approach: keeping iOS (and, increasingly, Android) applications online would be very inconvenient, and you wouldn't be able to create a decent user experience with such protocol.
If I understand you correctly, you are suggesting that both clients should be online while synchronising their history. If we are talking about a protocol that is supposed to work on all kinds of devices, this is a flawed approach: keeping iOS (and, increasingly, Android) applications online would be very inconvenient, and you wouldn't be able to create a decent user experience with such protocol.
I think this can be compared to calls or A/V in that both devices are expected to be actively used while the transfer is happening. I think this should be possible even on aggressive platforms like iOS.
It is actually not necessary. On Tue, 16 Jul 2024 at 14:15, Tim Henkes <syndace@web.de> wrote:
I think this can be compared to calls or A/V in that both devices are expected to be actively used while the transfer is happening. I think this should be possible even on aggressive platforms like iOS.
Aggressive or not, it might actually be unnecessary, at least according to our vision of multi-device encryption. We have discovered that whenever there are encryption session that are encrypted for one of your devices, and not the other, this leads to a horrible user experience that can't be recovered in no way. Grabbing the wrong device and seeing 'unable to decrypt message' which is decryptable on another of your devices is really, really bad and inconvenient, and we treat such incidents as a severe error. So our solution is to require the sender to encrypt messages for all valid devices of the message recipient, warning the user really aggressively if one of those devices is not trusted (via fingerprint matching or our other method of verification via code), nudging the user to perform device trust procedure. This way, the only history you need to synchronize conversations that happened prior to addition of this new device just once, at device initialization. For this, you don't need to create some complex protocols that would continuously sync history.
To solve the keys problem you may also be interested in - XEP-0434: Trust Messages (TM) - XEP-0450: Automatic Trust Management (ATM) Which Kaidan has implemented here -> https://www.kaidan.im/2022/08/31/e2ee-trust-management/ It serves as a nice solution to the problem. Andrew Nenakhov kirjoitti 16.7.2024 klo 12.40:
It is actually not necessary.
On Tue, 16 Jul 2024 at 14:15, Tim Henkes <syndace@web.de> wrote:
I think this can be compared to calls or A/V in that both devices are expected to be actively used while the transfer is happening. I think this should be possible even on aggressive platforms like iOS.
Aggressive or not, it might actually be unnecessary, at least according to our vision of multi-device encryption.
We have discovered that whenever there are encryption session that are encrypted for one of your devices, and not the other, this leads to a horrible user experience that can't be recovered in no way. Grabbing the wrong device and seeing 'unable to decrypt message' which is decryptable on another of your devices is really, really bad and inconvenient, and we treat such incidents as a severe error.
So our solution is to require the sender to encrypt messages for all valid devices of the message recipient, warning the user really aggressively if one of those devices is not trusted (via fingerprint matching or our other method of verification via code), nudging the user to perform device trust procedure. This way, the only history you need to synchronize conversations that happened prior to addition of this new device just once, at device initialization. For this, you don't need to create some complex protocols that would continuously sync history.
_______________________________________________ Standards mailing list --standards@xmpp.org To unsubscribe send an email tostandards-leave@xmpp.org
Yes, we are aware of them, but found them insufficient, and went with our own implementation. I'll try to briefly explain the difference in approach: Basically, there are two types of trust information you need to share: your own devices and those of your contacts whom you trust. Own devices are published on PEP with signature (so that if one of your devices, or contact's devices, sees that an already trusted device signed yet another device, it starts trusting it automatically. With contact's devices trusted by your own devices we opted for trust messages, but exchange it via special notifications service. (after many experiments of sending messages to your own xmpp id and implementations of it in various servers, we decided that a service is a more reliable option). Oh, and we added a third state: trust/distrust/revoked and timestamps. Eventually we'll publish it all some time after we're happy with how everything works (this particular section works well, but some others aren't, yet) and we release our apps. On Tue, 16 Jul 2024 at 14:45, MSavoritias <email@msavoritias.me> wrote:
To solve the keys problem you may also be interested in
- XEP-0434: Trust Messages (TM) - XEP-0450: Automatic Trust Management (ATM)
Which Kaidan has implemented here -> https://www.kaidan.im/2022/08/31/e2ee-trust-management/
It serves as a nice solution to the problem.
Andrew Nenakhov kirjoitti 16.7.2024 klo 12.40:
It is actually not necessary.
On Tue, 16 Jul 2024 at 14:15, Tim Henkes <syndace@web.de> wrote:
I think this can be compared to calls or A/V in that both devices are expected to be actively used while the transfer is happening. I think this should be possible even on aggressive platforms like iOS.
Aggressive or not, it might actually be unnecessary, at least according to our vision of multi-device encryption.
We have discovered that whenever there are encryption session that are encrypted for one of your devices, and not the other, this leads to a horrible user experience that can't be recovered in no way. Grabbing the wrong device and seeing 'unable to decrypt message' which is decryptable on another of your devices is really, really bad and inconvenient, and we treat such incidents as a severe error.
So our solution is to require the sender to encrypt messages for all valid devices of the message recipient, warning the user really aggressively if one of those devices is not trusted (via fingerprint matching or our other method of verification via code), nudging the user to perform device trust procedure. This way, the only history you need to synchronize conversations that happened prior to addition of this new device just once, at device initialization. For this, you don't need to create some complex protocols that would continuously sync history.
_______________________________________________ Standards mailing list -- standards@xmpp.org To unsubscribe send an email to standards-leave@xmpp.org
_______________________________________________ Standards mailing list -- standards@xmpp.org To unsubscribe send an email to standards-leave@xmpp.org
On 15/07/2024 21.51, Tim Henkes wrote:
2. Are encrypted direct client-to-client channels a thing?
There is JET [2], but it seems to focus on key negotiation (which I would do differently) […] There's also a XEP called jingle-xtls [3] in the Inbox, but it's even more abandoned than XEP-0247 and also seems to focus mostly on the key negotiation, which again I would do differently.
Could you elaborate on how you would the key negotiation. While it's always interesting to hear the how others would make things different, or maybe their design is based on different assumptions/starting points, I think it's also relevant to this discussion. In any case, I think most specifications are simply abandoned due the lack of implementation(s). Many probably never ever had a prototype implementation, let alone two interoperable implementations. Isn't "all we need"™ an encryption/authentication layer over (bidirectional) streams potentially negotiated by Jingle? And for the latter, there is the <security/> element which JET (xep391) / jingle-xtls also uses. - Florian
Could you elaborate on how you would the key negotiation. While it's always interesting to hear the how others would make things different, or maybe their design is based on different assumptions/starting points, I think it's also relevant to this discussion. Basically I see this as one of the highest security use-cases we have in XMPP. Since this protocol will give access to plain offline messages, even those that have been encrypted with OMEMO, OX or any other technology before, there is absolutely no room for security compromise here. That's why I'm planning a two-step negotiation mechanism that is performed with as little server involvement as possible. The first step would be opening a peer-to-peer channel and agreeing on the crypto primitives used (in a downgrade-protected manner), the second step would be confirming the absence of a MITM and enabling encryption of the p2p channel. This might sound horrible for usability, but the way I'm envisioning it, it would be fully automated and enabled by quick and comfortable QR-code scans. I'll create a quick and dirty draft so that we have a better basis for discussion.
In any case, I think most specifications are simply abandoned due the lack of implementation(s). Many probably never ever had a prototype implementation, let alone two interoperable implementations. Okay that's good to know and matches my impression, thanks :)
Isn't "all we need"™ an encryption/authentication layer over (bidirectional) streams potentially negotiated by Jingle? Yes exactly, just that ^^ And for the latter, there is the <security/> element which JET (xep391) / jingle-xtls also uses.
- Florian
That's why I'm planning a two-step negotiation mechanism that is performed with as little server involvement as possible. [...] I'll create a quick and dirty draft so that we have a better basis for discussion.
Scratch that, I just learned about WebRTC and DTLS-SRTP, which there are already XEPs for including integration with Jingle. Seems like everything is already specified and "just" needs to be connected correctly, which would better be done by someone who has hands-on experience with Jingle, WebRTC and the likes. Thanks everybody!
On Mon, Jul 15, 2024 at 9:52 PM Tim Henkes <syndace@web.de> wrote:
I've been brainstorming a mechanism for offline history transfer (i.e. one client fetches the offline message history from another client). For that, an encrypted direct client-to-client XML channel would be neat.
I’ve been planning to implement his for a very long time and do in fact have the funding to do so I’m just trying to find the time. I was planning on using "XEP-0247: Jingle XML Streams". I don’t think the fact that the XEP is Deferred has any significant meaning. Many of the A/V call related XEPs were deferred as well before we started using them again. Whether or not the XEP needs modification we will see while implementing it but from where I’m standing now I don’t see anything wrong with it. My plan was to first and foremost use WebRTC Datachannels as transports which neatly solves the encryption issue. (This is also the primary reason I've implemented file transfer over datachannels to get familiar with the technology). In a later iteration one could potentially implement jingle-xtls to also get Socks5 and IBB working but Datachannels should provide enough to get started (and cover 95% of use cases) cheers Daniel
I’ve been planning to implement his for a very long time and do in fact have the funding to do so I’m just trying to find the time. Oh okay, maybe we should coordinate then to make sure I don't void your funding possibility. I was planning on using "XEP-0247: Jingle XML Streams". I don’t think the fact that the XEP is Deferred has any significant meaning. Many of the A/V call related XEPs were deferred as well before we started using them again. Whether or not the XEP needs modification we will see while implementing it but from where I’m standing now I don’t see anything wrong with it. Nice, that's the sentiment I also got + seems to match Florian's view.
My plan was to first and foremost use WebRTC Datachannels as transports which neatly solves the encryption issue. (This is also the primary reason I've implemented file transfer over datachannels to get familiar with the technology). In a later iteration one could potentially implement jingle-xtls to also get Socks5 and IBB working but Datachannels should provide enough to get started (and cover 95% of use cases) Great, you seem to have a good idea of the transfer part while I have a solid plan for the key negotiation part, I'll dm you :D
cheers Daniel
participants (6)
-
Andrew Nenakhov -
Daniel Gultsch -
Florian Schmaus -
MSavoritias -
Philipp Hörist -
Tim Henkes