TL;DR
In light of recent findings regarding the performance gain provided by Hydra peers in the IPFS network, which indicate that the gain is in the area of 10% (or less) and considering the substantial cost that Hydra’s operation incurs, Protocol Labs has decided to carry out a “Hydra Dial Down” experiment. We plan to turn off certain functionality from Hydras, which we believe will allow all current critical operations that Hydras serve to continue as normal, while at the same time reducing the cost of their operation.
Pointers
-
Watch this recording from IPFS Camp 2022, which gives the summary of the results:
https://youtu.be/zhzxJGoLTg0 -
Follow developments on the project investigating the reliability and effectiveness of Hydra nodes: Hydras: Reliability and Effectiveness.
Details
Hydras serve three primary purposes:
-
Faster access to the Provider Records, i.e., reduced latency.
-
Availability of Provider Records, given that Hydras are generally stable and online, whereas normal DHT nodes churn (i.e., if all 20 peers where the Provider Record is stored go offline, the content becomes unreachable).
-
A “bridge” to services such as Indexers like cid.contact which advertise content from large providers like web3.storage and Filecoin nodes.
Our results show that Hydras provide at most 10% reduction in the latency to access provider records. The impact on the availability of Provider Records is expected to be minimal, given that they are published to 20 peers in total. Check this detailed report on the availability of provider records in the IPFS network over time and how Hydras affect the records’ liveness. In summary, the first two points above should not be of concern for the operation of the network in the absence of Hydra nodes.
The Hydra’s “bridging” component to other parts of IPFS and Filecoin infrastructure will not be affected. Hydras will maintain connections to cid.contact Indexer nodes and will return results from them.
When, where, what and monitoring logistics
When: The dial down will take place on the 1st of December 2022 at 3pm UTC. We will send an update to confirm, or report a date change (if the need arises) in this forum at the latest 24hrs before the above date, ideally earlier. We will post a message here once the dial down deployment has taken place.
What: We will first disable the database part of Hydras, so provider records won’t be served. Hydras will act as normal DHT Server nodes and will return the “closer peers” they know of in the DHT. The bridging to cid.contact Indexer nodes will continue to operate as normal. Further comms will be done if and when things change with respect to that part of the Hydra’s functionality.
Monitoring: We will be monitoring the performance of Hydras and plan to report updates and performance results in this thread. We will post updates to this thread and expect to have the first results report within the first hour after dialling down. We will report results for several hours on an hourly basis following the event and a summarising report for the first day 24 hours later. From then on we will be posting daily reports for up to 7 days, but will continue to monitor the situation continuously afterwards. There will be a team of engineers on-call, monitoring developments and ready to take action as needed.
Where: The best points of contact to get in touch, or report issues are: i) this thread, ii) the #hydra-dial-down channel in the Filecoin Slack Workspace [invite link], iii) through email at: hydra-dial-down@protocol.ai. Please report issues from your vantage point (node, application, infrastructure), or questions you might have before, during, or after the dial down event.
Update on results from the IPFS Network after the Hydra Dial Down
