Proposal: Peer Hint URI scheme

1

From what I understand how IPFS works, adding files on IPFS works like this:

  • We put a file to IPFS on computer A, it computes [hash]
  • We enter ipfs://[hash] on computer B
  • Computer B starts the search on DHT where [hash] is located, this takes a while
  • When it does, it connects to the peer containing the [hash] and starts to download it (assuming it’s a single chunk file)

OK, but the thing is that when we first enter the IPFS address on the first other computer, i.e. computer B in this case, it takes 10-20 min for smaller files to several hours for bigger files to be quickly available to computer B, because of the slow DHT search. After that, it is quickly available to computer C, D, …

This I think is unacceptable for quickly sharing files with someone or dynamic applications.

Instead, a more rational way for this would be to optionally add to the URI a hint where the file is available, i.e. which peer is the originator. This would hasten the download while the file isn’t popular yet.

For example instead of ipfs://[hash]/ we could have ipfs://[hash]?peer=1.2.3.4/4001 or something similar. Which would mean that the peer 1.2.3.4:4001 has the [hash]. That way IPFS could do a DHT search + automatically add the 1.2.3.4 peer as having the [hash] and thereby speed up the whole process of initially puting files on IPFS.

What do you think?
Am I misunderstanding how IPFS works?
Is this a bad idea?

I bounced the idea on #ipfs@freenode and the ones that responded seemed to like it.

6 Likes
2

I like this idea, and I think there might be some things to learn here from the magnet: URIs that bittorrent uses.

1 Like
3

16 posts were split to a new topic: Debugging slow content discovery

4

Yes, magnet has the alternative download location and tracker url. This would be similar.

20

I just want to note that this is exactly how matrix handles extra discoverability, which is kinda interesting.

21

Interesting idea.

Analog to magnet links is obvious, but in IPFS there are two types of hints we could think about:

Just PeerID(s)

QmSoLnSGccFuZQJzRadHn95W2CrSFmZuTdDWP8HXaHca9z
  • :green_apple: speeds up content discovery by providing a hint for connecting to specified peers
  • :green_apple: short
  • :anger: requires additional lookup to discover peer’s multiaddrs

Peer Multiaddrs

/ip4/104.236.176.52/tcp/4001/ipfs/QmSoLnSGccFuZQJzRadHn95W2CrSFmZuTdDWP8HXaHca9z
  • :green_apple: speeds up content discovery by providing a hint to try to connect to specified peers using specific route/protocol
  • :green_apple: does not require DHT call if provided multiaddr works
  • :green_apple: includes PeerID, so in case specific multiaddr can’t be used, alternative ones can be retrieved from DHT
  • :anger: URI gets very very long

Universal Provider Hint: “provs=”

Given the fact multiaddr will always start with /, we could support both PeerIDs and multiaddrs under a single URI parameter:

ipfs://{cid}?provs={multiaddr1},{peerId2},{peerId3}`

Open Questions / Concerns

There are some unknowns, so if we introduce support for this type of hint to tools like IPFS Companion, we should make it an opt-in experiment (ipfs-companion/issues/722)

Privacy

While I see the appeal of this speeding up things like sharing apps, I am bit worried that this type of hint becomes a p2p version of ping attribute used for tracking. Imagine link-specific peers that are listed as provs but do not host data, just log IDs of peers connecting to it

Unintended DDoS

Link to popular content could cause DDoS of peers listed in URI hint.
Similar concerns were raised in this discussion about seed services.

22

@lidel @sebaseba
Recommend to add this solution to the newly created meta issue below.

23

IPFS team is considering to implement similar feature in IPFS gateway.
So after we ipfs add some data in node A, then we can fetch it in IPFS gateway, with the IP address of node A specified, so as to help the gateway find the data faster.

2 Likes