DHT discussion and contribution opportunities in 2023Q4

The PL EngRes IPFS Stewards team has several ongoing streams of work related to the improvement and optimisation of the Public IPFS DHT, henceforth called Amino (see: Amino is the proposed name for the Public IPFS DHT).

We would like to open up the discussion to the wider community to:

  1. raise awareness of our current and future plans, but also,
  2. seek ideas, help and contributions on the longer-term targets and vision for this important component of Web3 networks.

With regard to point 1. above, in the coming days, we will send a link to a blogpost summarising our ongoing work.

With regard to point 2., we plan to form a DHT Working Group (WG), which can meet online regularly (e.g., weekly/biweekly) and physically when possible, alongside bigger events. A first good venue for a physical meeting is LabWeek taking place alongside DevConnect, in November (in Istanbul). We will announce intentions and meeting times in this thread, so make sure to subscribe/follow if you are interested.

5 Likes

It took a bit longer than initially thought, here is the blogpost: Amino (the Public IPFS DHT) is getting a facelift | IPFS Blog & News

As always, feedback and active involvement more than welcome. Feel free to reach out in the #probe-lab channel in the IPFS Discord or FIL slack (bridged channel).

Sounds great - as far as I understand. Two questions:

1. It does sound like mainly an improvement for “big” providers, and I’m wondering when “big” starts?

I seem to run into “Falling behing reprovides” warning in kubo already with a few thousand CIDs, and the only options seems to be using AcceleratedDHTClient, which is more heavy on resources. (and unclear to me how much).

For a Content Provider providing 100K CIDs, and 25K DHT Servers the expected improvement is ~80x.

Is this ReprovideSweep a different option related to this problem?

2. The timeline was described as:

The […] team is currently working to define the spec for ReprovideSweep, which we hope to have ready in the beginning of October, and we anticipate rolling out this enhancement during Q4’23.

Is there a new timeline? (No pressure, just curious)

Thanks for everything :slight_smile:

We don’t have a concrete timeline for this, but we want to address some of the challenges around slow reprovides.

We also have a tracking issue for Reprovide Sweep Reprovide Sweep · Issue #824 · libp2p/go-libp2p-kad-dht · GitHub

1 Like