Is Bitswap discovery step worth the wait?

Protocol
, , ,
1

My intuition tells me that the success rate of Bitswap as a content discovery mechanism is pretty low, perhaps close to zero. The way Bitswap tries to discover content in neighbouring/immediately-connected peers would make a lot of sense if used within the remit of an application, for instance, but it seems pretty optimistic to be able to find anything in the entire IPFS network.

At the same time, the Bitswap timeout is adding 1sec to the discovery process, i.e., before it proceeds to resolve content through the DHT. Our measurements show that in at least half of the content lookup attempts from several geographic locations, the DHT lookup finishes in less than 1sec (first fig)! :clap:t3::clap:t3::clap:t3: In turn, this means that (at least in these cases) Bitswap increases the discovery latency by 100%, or more (second fig).

I would argue that if the Bitswap discovery success rate is minimal, then it’s clearly not worth the wait.

Has there been discussion on disabling the Bitswap discovery completely, or at least making it run in parallel to the DHT lookup?

Screenshot 2022-03-10 at 14.39.05
Screenshot 2022-03-10 at 14.39.05898×926 82 KB

Screenshot 2022-03-10 at 14.38.52
Screenshot 2022-03-10 at 14.38.52900×926 80.5 KB

Screenshot 2022-03-10 at 14.39.28
Screenshot 2022-03-10 at 14.39.282020×72 7.99 KB

1 Like
2

You are making the assumptions that content is in the DHT.

Some pinning service (pinata) does not use the DHT and only rely on that, I guess it’s working fine enough for them.

Aren’t we doing them concurrently ? If not, we should do it. Why bother choosing when you can do both ?

3

I do, indeed :slight_smile: Ultimately, it should be up to operators/users should have the option to choose.

I would see four options:

  1. Bitswap, then DHT (current)
  2. Bitswap in parallel to the DHT (suggested default)
  3. Bitswap only (could work for application-specific cases, e.g., pinata)
  4. DHT only (could be the case for end-users)

There are tradeoffs to be taken into account. Traffic overhead will clearly increase in case of parallel queries, but this would be a good first step IMO.

2 Likes
4

Why make it one or the other.? Every situation is going to be different and constantly changing. Seems like a good candidate for it to be a learned parameter.

5

Which ones do you refer to by one and the other? :slight_smile: Would starting queries both through Bitswap and through the DHT cover your concerns?

6

This would go towards addressing, “… tradeoffs to be taken into account.”