15 Jul
2024
15 Jul
'24
1:51 p.m.
Hi folks,
I've been brainstorming a mechanism for offline history transfer (i.e.
one client fetches the offline message history from another client). For
that, an encrypted direct client-to-client XML channel would be neat.
1. Are direct client-to-client XML channels a thing?
I found XEP-0247 [1], which looks alright on first read-through but is
deferred since 2010. Does someone more familiar with the topic know why
the XEP was abandoned and whether it could realistically be revived or
if there is a better alternative now?
2. Are encrypted direct client-to-client channels a thing?
There is JET [2], but it seems to focus on key negotiation (which I
would do differently) and file transfers, which I don't need either. I
don't see how a bidirectional data channel/stream would be encrypted
using JET.
There's also a XEP called jingle-xtls [3] in the Inbox, but it's even
more abandoned than XEP-0247 and also seems to focus mostly on the key
negotiation, which again I would do differently.
Next to these questions I would be interested in the general sentiment
and ideas for workarounds (e.g. "p2p is cursed, just send encrypted
stanzas via the server as usual" or "just use a simple binary format and
don't bother with a fully fledged XML stream").
Thanks :D
Tim
[1] https://xmpp.org/extensions/xep-0247.html
[2] https://xmpp.org/extensions/xep-0391.html
[3] https://xmpp.org/extensions/inbox/jingle-xtls.html
15 Jul
15 Jul
1:55 p.m.
We're not currently doing it but considering to do in the future. Very much
likely we'll just do an upload of an encrypted archive via cloud storage.
This way it can be interoperable with the likes of Dropbox / gdrive /
whatever.
On Tue, Jul 16, 2024, 00:51 Tim Henkes <syndace(a)web.de> wrote:
Hi folks,
I've been brainstorming a mechanism for offline history transfer (i.e.
one client fetches the offline message history from another client). For
that, an encrypted direct client-to-client XML channel would be neat.
1. Are direct client-to-client XML channels a thing?
I found XEP-0247 [1], which looks alright on first read-through but is
deferred since 2010. Does someone more familiar with the topic know why
the XEP was abandoned and whether it could realistically be revived or
if there is a better alternative now?
2. Are encrypted direct client-to-client channels a thing?
There is JET [2], but it seems to focus on key negotiation (which I
would do differently) and file transfers, which I don't need either. I
don't see how a bidirectional data channel/stream would be encrypted
using JET.
There's also a XEP called jingle-xtls [3] in the Inbox, but it's even
more abandoned than XEP-0247 and also seems to focus mostly on the key
negotiation, which again I would do differently.
Next to these questions I would be interested in the general sentiment
and ideas for workarounds (e.g. "p2p is cursed, just send encrypted
stanzas via the server as usual" or "just use a simple binary format and
don't bother with a fully fledged XML stream").
Thanks :D
Tim
[1] https://xmpp.org/extensions/xep-0247.html
[2] https://xmpp.org/extensions/xep-0391.html
[3] https://xmpp.org/extensions/inbox/jingle-xtls.html
_______________________________________________
Standards mailing list -- standards(a)xmpp.org
To unsubscribe send an email to standards-leave(a)xmpp.org
3:17 p.m.
Hi,
Your constraints seem rather far fetched.
You are offline? What does this mean? in a local network, not connected to the internet?
How do you find and connect to another device without server?
Why would you need to transfer your history specifically in these conditions?
Simply upload to http server encrypted.
XML Stream via Server is not a good way to transfer hundreds of megabytes.
Is this about the transport method? Or the format?
Because interoperable format seems like a big task, transport seems easy.
I think defining the transport and where to store is useful short term, then every client
can offer their own backup.
Its useful independent of the format.
im less optimistic for a standardized exchange format.
Regards
Philipp
On Mon, Jul 15, 2024, at 21:55, Andrew Nenakhov wrote:
We're not currently doing it but considering to do
in the future. Very much likely we'll just do an upload of an encrypted archive via
cloud storage. This way it can be interoperable with the likes of Dropbox / gdrive /
whatever.
On Tue, Jul 16, 2024, 00:51 Tim Henkes <syndace(a)web.de> wrote:
Hi folks,
I've been brainstorming a mechanism for offline history transfer (i.e.
one client fetches the offline message history from another client). For
that, an encrypted direct client-to-client XML channel would be neat.
1. Are direct client-to-client XML channels a thing?
I found XEP-0247 [1], which looks alright on first read-through but is
deferred since 2010. Does someone more familiar with the topic know why
the XEP was abandoned and whether it could realistically be revived or
if there is a better alternative now?
2. Are encrypted direct client-to-client channels a thing?
There is JET [2], but it seems to focus on key negotiation (which I
would do differently) and file transfers, which I don't need either. I
don't see how a bidirectional data channel/stream would be encrypted
using JET.
There's also a XEP called jingle-xtls [3] in the Inbox, but it's even
more abandoned than XEP-0247 and also seems to focus mostly on the key
negotiation, which again I would do differently.
Next to these questions I would be interested in the general sentiment
and ideas for workarounds (e.g. "p2p is cursed, just send encrypted
stanzas via the server as usual" or "just use a simple binary format and
don't bother with a fully fledged XML stream").
Thanks :D
Tim
[1] https://xmpp.org/extensions/xep-0247.html
[2] https://xmpp.org/extensions/xep-0391.html
[3] https://xmpp.org/extensions/inbox/jingle-xtls.html
_______________________________________________
Standards mailing list -- standards(a)xmpp.org
To unsubscribe send an email to standards-leave(a)xmpp.org
_______________________________________________
Standards mailing list -- standards(a)xmpp.org
To unsubscribe send an email to standards-leave(a)xmpp.org
3:37 p.m.
Hey,
You are offline? What does this mean? in a local network, not
connected to the internet?
No, I'm not offline. I'm fully online and
connected to my server as
usual. I just think to synchronize offline history, which could be a lot
of data and include confidential messages that were initially encrypted,
an encrypted direct client-to-client channel would be cool, both for
speed and security.
Simply upload to http server encrypted.
That seems inflexible. Would work for the
simplest use-case (completely
new client wants the full history), but if realistically possible I'd
like to be a bit more flexible. I'd also like to avoid that because it
would mean that your whole history is now available online in one nice
package secured with a single encryption key, so all the effort you put
in beforehand with forward-secrecy etc. would be gone.
XML Stream via Server is not a good way to transfer
hundreds of megabytes.
That's one of the reasons I don't want to go over the
server but use a
peer-to-peer connection instead :)
Is this about the transport method? Or the format?
Because interoperable format seems like a big task, transport seems easy.
Hah, I
would have expected it the other way around. For the format I
could use e.g. MAM, which would give me nice message archive queries and
is supported by the clients I would want to target. The transport seems
difficult though.
I think defining the transport and where to store is
useful short
term, then every client can offer their own backup.
It's not really about a
"backup" in that sense. For a backup I would
probably do what Andrew suggested and just create a password-protected
zip file with my data and upload that somewhere. The use-case I am
brainstorming is when you get a new client, but you can't get the
history from the server because it either doesn't have complete archives
or some stuff was OMEMO-encrypted. So you quickly scan a QR-code on some
of your other clients and it transfers the local, plaintext messages over.
Regards
Philipp
On Mon, Jul 15, 2024, at 21:55, Andrew Nenakhov wrote:
We're not currently doing it but considering
to do in the future.
Very much likely we'll just do an upload of an encrypted archive via
cloud storage. This way it can be interoperable with the likes of
Dropbox / gdrive / whatever.
On Tue, Jul 16, 2024, 00:51 Tim Henkes <syndace(a)web.de> wrote:
Hi folks,
I've been brainstorming a mechanism for offline history transfer
(i.e.
one client fetches the offline message history from another
client). For
that, an encrypted direct client-to-client XML channel would be neat.
1. Are direct client-to-client XML channels a thing?
I found XEP-0247 [1], which looks alright on first read-through
but is
deferred since 2010. Does someone more familiar with the topic
know why
the XEP was abandoned and whether it could realistically be
revived or
if there is a better alternative now?
2. Are encrypted direct client-to-client channels a thing?
There is JET [2], but it seems to focus on key negotiation (which I
would do differently) and file transfers, which I don't need
either. I
don't see how a bidirectional data channel/stream would be encrypted
using JET.
There's also a XEP called jingle-xtls [3] in the Inbox, but it's even
more abandoned than XEP-0247 and also seems to focus mostly on
the key
negotiation, which again I would do differently.
Next to these questions I would be interested in the general
sentiment
and ideas for workarounds (e.g. "p2p is cursed, just send encrypted
stanzas via the server as usual" or "just use a simple binary
format and
don't bother with a fully fledged XML stream").
Thanks :D
Tim
[1] https://xmpp.org/extensions/xep-0247.html
[2] https://xmpp.org/extensions/xep-0391.html
[3] https://xmpp.org/extensions/inbox/jingle-xtls.html
_______________________________________________
Standards mailing list -- standards(a)xmpp.org
To unsubscribe send an email to standards-leave(a)xmpp.org
_______________________________________________
Standards mailing list -- standards(a)xmpp.org
To unsubscribe send an email to standards-leave(a)xmpp.org
_______________________________________________
Standards mailing list --standards(a)xmpp.org
To unsubscribe send an email tostandards-leave(a)xmpp.org
16 Jul
16 Jul
12:45 a.m.
On Tue, 16 Jul 2024 at 02:37, Tim Henkes <syndace(a)web.de> wrote:
Hey,
You are offline? What does this mean? in a local network, not connected to
the internet?
No, I'm not offline. I'm fully online and connected to my server as usual.
I just think to synchronize offline history, which could be a lot of data
and include confidential messages that were initially encrypted, an
encrypted direct client-to-client channel would be cool, both for speed and
security.
If I understand you correctly, you are suggesting that both clients should
be online while synchronising their history. If we are talking about a
protocol that is supposed to work on all kinds of devices, this is a flawed
approach: keeping iOS (and, increasingly, Android) applications online
would be very inconvenient, and you wouldn't be able to create a decent
user experience with such protocol.
3:15 a.m.
If I understand you correctly, you are suggesting that
both clients
should be online while synchronising their history. If we are talking
about a protocol that is supposed to work on all kinds of devices,
this is a flawed approach: keeping iOS (and, increasingly, Android)
applications online would be very inconvenient, and you wouldn't be
able to create a decent user experience with such protocol.
I think this can be compared to calls or A/V in that both devices are
expected to be actively used while the transfer is happening. I think
this should be possible even on aggressive platforms like iOS.
3:40 a.m.
It is actually not necessary.
On Tue, 16 Jul 2024 at 14:15, Tim Henkes <syndace(a)web.de> wrote:
I think this can be compared to calls or A/V in that
both devices are
expected to be actively used while the transfer is happening. I think this
should be possible even on aggressive platforms like iOS.
Aggressive or not, it might actually be unnecessary, at least according to
our vision of multi-device encryption.
We have discovered that whenever there are encryption session that are
encrypted for one of your devices, and not the other, this leads to a
horrible user experience that can't be recovered in no way. Grabbing the
wrong device and seeing 'unable to decrypt message' which is decryptable on
another of your devices is really, really bad and inconvenient, and we
treat such incidents as a severe error.
So our solution is to require the sender to encrypt messages for all valid
devices of the message recipient, warning the user really aggressively if
one of those devices is not trusted (via fingerprint matching or our other
method of verification via code), nudging the user to perform device trust
procedure. This way, the only history you need to synchronize conversations
that happened prior to addition of this new device just once, at device
initialization. For this, you don't need to create some complex protocols
that would continuously sync history.
3:45 a.m.
To solve the keys problem you may also be interested in
- XEP-0434: Trust Messages (TM)
- XEP-0450: Automatic Trust Management (ATM)
Which Kaidan has implemented here ->
https://www.kaidan.im/2022/08/31/e2ee-trust-management/
It serves as a nice solution to the problem.
Andrew Nenakhov kirjoitti 16.7.2024 klo 12.40:
It is actually not necessary.
On Tue, 16 Jul 2024 at 14:15, Tim Henkes <syndace(a)web.de> wrote:
I think this can be compared to calls or A/V in that both devices
are expected to be actively used while the transfer is happening.
I think this should be possible even on aggressive platforms like iOS.
Aggressive or not, it might actually be unnecessary, at least
according to our vision of multi-device encryption.
We have discovered that whenever there are encryption session that are
encrypted for one of your devices, and not the other, this leads to a
horrible user experience that can't be recovered in no way. Grabbing
the wrong device and seeing 'unable to decrypt message' which is
decryptable on another of your devices is really, really bad and
inconvenient, and we treat such incidents as a severe error.
So our solution is to require the sender to encrypt messages for all
valid devices of the message recipient, warning the user really
aggressively if one of those devices is not trusted (via fingerprint
matching or our other method of verification via code), nudging the
user to perform device trust procedure. This way, the only history you
need to synchronize conversations that happened prior to addition of
this new device just once, at device initialization. For this, you
don't need to create some complex protocols that would continuously
sync history.
_______________________________________________
Standards mailing list --standards(a)xmpp.org
To unsubscribe send an email tostandards-leave(a)xmpp.org
5:43 a.m.
Yes, we are aware of them, but found them insufficient, and went with our
own implementation. I'll try to briefly explain the difference in approach:
Basically, there are two types of trust information you need to share: your
own devices and those of your contacts whom you trust. Own devices are
published on PEP with signature (so that if one of your devices, or
contact's devices, sees that an already trusted device signed yet another
device, it starts trusting it automatically. With contact's devices trusted
by your own devices we opted for trust messages, but exchange it via
special notifications service. (after many experiments of sending messages
to your own xmpp id and implementations of it in various servers, we
decided that a service is a more reliable option).
Oh, and we added a third state: trust/distrust/revoked and timestamps.
Eventually we'll publish it all some time after we're happy with how
everything works (this particular section works well, but some others
aren't, yet) and we release our apps.
On Tue, 16 Jul 2024 at 14:45, MSavoritias <email(a)msavoritias.me> wrote:
To solve the keys problem you may also be interested
in
- XEP-0434: Trust Messages (TM)
- XEP-0450: Automatic Trust Management (ATM)
Which Kaidan has implemented here ->
https://www.kaidan.im/2022/08/31/e2ee-trust-management/
It serves as a nice solution to the problem.
Andrew Nenakhov kirjoitti 16.7.2024 klo 12.40:
It is actually not necessary.
On Tue, 16 Jul 2024 at 14:15, Tim Henkes <syndace(a)web.de> wrote:
I think this can be compared to calls or A/V in
that both devices are
expected to be actively used while the transfer is happening. I think this
should be possible even on aggressive platforms like iOS.
Aggressive or not, it might actually be unnecessary, at least according to
our vision of multi-device encryption.
We have discovered that whenever there are encryption session that are
encrypted for one of your devices, and not the other, this leads to a
horrible user experience that can't be recovered in no way. Grabbing the
wrong device and seeing 'unable to decrypt message' which is decryptable on
another of your devices is really, really bad and inconvenient, and we
treat such incidents as a severe error.
So our solution is to require the sender to encrypt messages for all valid
devices of the message recipient, warning the user really aggressively if
one of those devices is not trusted (via fingerprint matching or our other
method of verification via code), nudging the user to perform device trust
procedure. This way, the only history you need to synchronize conversations
that happened prior to addition of this new device just once, at device
initialization. For this, you don't need to create some complex protocols
that would continuously sync history.
_______________________________________________
Standards mailing list -- standards(a)xmpp.org
To unsubscribe send an email to standards-leave(a)xmpp.org
_______________________________________________
Standards mailing list -- standards(a)xmpp.org
To unsubscribe send an email to standards-leave(a)xmpp.org
12:38 a.m.
On 15/07/2024 21.51, Tim Henkes wrote:
2. Are encrypted direct client-to-client channels a
thing?
There is JET [2], but it seems to focus on key negotiation (which I
would do differently) […]
There's also a XEP called jingle-xtls [3] in the Inbox, but it's even
more abandoned than XEP-0247 and also seems to focus mostly on the key
negotiation, which again I would do differently.
Could you elaborate on how you would the key negotiation. While it's
always interesting to hear the how others would make things different,
or maybe their design is based on different assumptions/starting points,
I think it's also relevant to this discussion.
In any case, I think most specifications are simply abandoned due the
lack of implementation(s). Many probably never ever had a prototype
implementation, let alone two interoperable implementations.
Isn't "all we need"™ an encryption/authentication layer over
(bidirectional) streams potentially negotiated by Jingle? And for the
latter, there is the <security/> element which JET (xep391) /
jingle-xtls also uses.
- Florian
3:14 a.m.
Could you elaborate on how you would the key
negotiation. While it's
always interesting to hear the how others would make things different,
or maybe their design is based on different assumptions/starting
points, I think it's also relevant to this discussion.
Basically I see this as
one of the highest security use-cases we have in
XMPP. Since this protocol will give access to plain offline messages,
even those that have been encrypted with OMEMO, OX or any other
technology before, there is absolutely no room for security compromise
here. That's why I'm planning a two-step negotiation mechanism that is
performed with as little server involvement as possible. The first step
would be opening a peer-to-peer channel and agreeing on the crypto
primitives used (in a downgrade-protected manner), the second step would
be confirming the absence of a MITM and enabling encryption of the p2p
channel. This might sound horrible for usability, but the way I'm
envisioning it, it would be fully automated and enabled by quick and
comfortable QR-code scans. I'll create a quick and dirty draft so that
we have a better basis for discussion.
In any case, I think most specifications are simply abandoned due the
lack of implementation(s). Many probably never ever had a prototype
implementation, let alone two interoperable implementations.
Okay that's good
to know and matches my impression, thanks :)
Isn't "all we need"™ an encryption/authentication layer over
(bidirectional) streams potentially negotiated by Jingle?
Yes exactly, just that
^^
And for the latter, there is the <security/>
element which JET
(xep391) / jingle-xtls also uses.
- Florian
4:42 a.m.
That's why I'm planning a two-step negotiation
mechanism that is
performed with as little server involvement as possible. [...] I'll
create a quick and dirty draft so that
we have a better basis for discussion.
Scratch that, I just learned about WebRTC and DTLS-SRTP, which there are
already XEPs for including integration with Jingle. Seems like
everything is already specified and "just" needs to be connected
correctly, which would better be done by someone who has hands-on
experience with Jingle, WebRTC and the likes.
Thanks everybody!
2:36 a.m.
On Mon, Jul 15, 2024 at 9:52 PM Tim Henkes <syndace(a)web.de> wrote:
I've been brainstorming a mechanism for offline
history transfer (i.e.
one client fetches the offline message history from another client). For
that, an encrypted direct client-to-client XML channel would be neat.
I’ve been planning to implement his for a very long time and do in
fact have the funding to do so I’m just trying to find the time.
I was planning on using "XEP-0247: Jingle XML Streams". I don’t think
the fact that the XEP is Deferred has any significant meaning. Many of
the A/V call related XEPs were deferred as well before we started
using them again. Whether or not the XEP needs modification we will
see while implementing it but from where I’m standing now I don’t see
anything wrong with it.
My plan was to first and foremost use WebRTC Datachannels as
transports which neatly solves the encryption issue. (This is also the
primary reason I've implemented file transfer over datachannels to get
familiar with the technology). In a later iteration one could
potentially implement jingle-xtls to also get Socks5 and IBB working
but Datachannels should provide enough to get started (and cover 95%
of use cases)
cheers
Daniel
3:20 a.m.
I’ve been planning to implement his for a very long
time and do in
fact have the funding to do so I’m just trying to find the time.
Oh okay, maybe we
should coordinate then to make sure I don't void your
funding possibility.
I was planning on using "XEP-0247: Jingle XML
Streams". I don’t think
the fact that the XEP is Deferred has any significant meaning. Many of
the A/V call related XEPs were deferred as well before we started
using them again. Whether or not the XEP needs modification we will
see while implementing it but from where I’m standing now I don’t see
anything wrong with it.
Nice, that's the sentiment I also got + seems to match
Florian's view.
My plan was to first and foremost use WebRTC Datachannels as
transports which neatly solves the encryption issue. (This is also the
primary reason I've implemented file transfer over datachannels to get
familiar with the technology). In a later iteration one could
potentially implement jingle-xtls to also get Socks5 and IBB working
but Datachannels should provide enough to get started (and cover 95%
of use cases)
Great, you seem to have a good idea of the transfer part while I have
a
solid plan for the key negotiation part, I'll dm you :D
cheers
Daniel
131
days inactive
132
days old
13 comments
6 participants
participants (6)
-
Andrew Nenakhov -
Daniel Gultsch -
Florian Schmaus -
MSavoritias -
Philipp Hörist -
Tim Henkes