Optimistic Provide: How We Made IPFS Content Publishing 10x Faster

News
, ,
1

Hi folks,

Publishing content in Distributed Hash Tables (DHTs) has traditionally been a slow operation, especially when

  1. the network is large, and
  2. the nodes participating in the network are churning frequently.

The case is no different for IPFS’s Amino DHT, which meets both of these conditions. We, the ProbeLab team, proposed an optimisation that was shown to improve performance, i.e., reduce the content publication time by over one order of magnitude while simultaneously reducing the network overhead by 40%.

The optimisation was named Optimistic Provide and it only recently shipped as a default in IPFS’s Kubo 0.39.0! The basic ideas behind Optimistic Provide are the following:

  1. While walking the DHT, immediately store records with peers that are likely among the 20 network-wide closest peers.
  2. Terminate the DHT walk immediately when the set of the discovered 20 closest peers likely constitute the network-wide closest peers.
  3. Return control back to the user after most (not all) of the PUT RPCs have succeeded and continue with the remaining ones in the background.

Optimistic Provide decreases significantly the “upload” latency, from around 15 seconds to less than 1 second. We believe that Optimistic Provide together with Reprovide Sweep fundamentally redefine the experience of publishing content on IPFS.

Check out our blog post for the details: Optimistic Provide: How We Made IPFS Content Publishing 10x Faster - Blog | ProbeLab Analytics

If you have feedback, we would love to hear from you!

Cheers

1 Like