Private overlay networks

Ecosystem
1

Hi all,

I’ve been thinking about using IPFS for some things that require keeping some data private (like sharing files with a friend). It seems that currently, the experimental Private Networks are the best or only way to go, but the way they work has two issues that are inherent in the design:

  1. Need to explicitly specify bootstrap nodes, which means having to communicate their public IP, which is of course a usability mess - just try to use this with a NAT when your provider keeps changing your public IP address…
  2. For use cases like hosted JS applications, we really have a mix of public and private data, and it would be good if our node could still participate in Bitswap for the public data.

Hence the topic: private overlay networks.

Could we have an IPFS node that participates in the full default swarm, but restricts the exchange of certain private blocks to a subset of peers that have authenticated with a private network key? These peers would in effect form an overlay network.

Obviously this isn’t possible today, but it seems desirable to me. Has this ever been considered/discussed? I’m new to IPFS, but this is the kind of thing where I could see myself contributing to making it better.

Oh, and most-likely-FAQ: Why not just exchange encrypted blocks? Simple: Because I don’t want to allow three-letter agencies to be able to vacuum up people’s encrypted blocks and then decrypt them all years later when the used encryption method inevitably gets broken or somebody accidentally but also kind-of inevitably leaks an old key.

Cheers,
Nicolai

1 Like
2

I think something somewhat similar has been discussed here. But this is an old enough discussion that it probably pre-dates private IPFS networking functionality.

FWIW I think this is a great idea, though I’m having trouble envisioning an ideal UI for separating private content and public content on a single node. I think it would also be cool to be able to have multiple private networks on a single node and be able to share files with one or some subset of private networks that your node belongs to.

One thing that I think might be tricky to get around is your point #1 since the nodes in your private network still need some way to identify each other among the much larger group of nodes that aren’t part of your private network. Taking advantage of public nodes for bootstrapping private networks doesn’t really help the public network in any way and would necessarily add some overhead to them. The public bootstrapping nodes would need to keep track of public peers along with peers for all of the private networks hitting it that it doesn’t even belong to.

An existing solution (I haven’t tested this but I don’t see why it wouldn’t work) to dynamic IP addresses for bootstrap nodes would be to use a dynamic DNS provider and specify the bootstrap nodes for the private network similar to how the bootstrap.libp2p.io nodes are – no static IPs needed.

/dnsaddr/bootstrap.libp2p.io/ipfs/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN
/dnsaddr/bootstrap.libp2p.io/ipfs/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa
/dnsaddr/bootstrap.libp2p.io/ipfs/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb
/dnsaddr/bootstrap.libp2p.io/ipfs/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt
1 Like
3

Thank you for your answer! It’s been a while, but I’ve still been thinking about this on and off.

Here’s the simplest thing I could come up with that possibly could provide something useful, although not without problems. Let’s talk UI:

  1. Create a new private networtk:
    $ ipfs privnet create NAME1
    –> generates a SECRET code
  2. Add files to the private network:
    $ ipfs add --privnet=NAME1 FILE
    –> Shows the usual Qm… hash, but the file is internally marked as only available in the private network. Bitswap will refuse to send it to any peer which hasn’t proven that it knows SECRET. If the file is a directory, or a file split into multiple blocks, then all the recursively added files and blocks are marked in this way.
  3. Send the SECRET code to somebody you know. They will register with the private network:
    $ ipfs privnet add NAME2 SECRET
    (The NAME here is just a mnemonic similar to key names, so NAME1 and NAME2 can be different)
  4. Retrieve the file:
    $ ipfs cat /ipns/Qm…
    –>Bitswap will internally communicate that the file is only available in the private network; the nodes will mutually authenticate using some variant of Socialist Millionaire, and then exchange the file.

I think this is about as good as it gets in terms of UI within the current IPFS ecosystem.

The approach does have some problems:

  • The networks are identified by a shared secret, so it’s not possible to revoke access to a network after the fact. But I understand this is similar to the existing private swarm functionality, so may not be such a big deal.
  • Nodes that don’t belong to the private network can still see the hashes of files that are supposed to be secret. In some particularly sensitive cases, even leaking the hash of a file is too much.

The second problem could be fixed by salting the hash with the network’s shared SECRET.

What do you think?

On your point about taking advantage of public nodes for bootstrapping: That’s true, but I think it’s the price to pay for having a peer-to-peer network, and it’s not even exclusive to this kind of private network. Whenever there is some data that only few nodes are interested in, all nodes potentially participate in routing this data via the DHT. This is true whether or not the data is public. So what those private networks would be doing really isn’t that different from any other application of IPFS.

Besides, the more useful IPFS becomes, the more users it’ll have, which should improve the network overall.

1 Like
4

Would ipfs just try every secret you’ve added until one works to authenticate for the file desired?

I really like this idea, or something like it.