IPFS-cluster as a drop in replacement

,
1

Hi,

We are trying to replace ipfs-cluster as a drop in replacement for existing single ipfs node.
Hence, we have configured the proxy endpoint so that switch happens seamlessly.
We have noticed that files added to this endpoint are pinned in the cluster as mentioned in the proxy documentation, whereas the DAG blocks added using the ipfs dag put API are not getting pinned in the cluster eventhough passed with pin flag. (Kubo RPC API | IPFS Docs).
Is there any configuration change that can be made to ensure dag blocks are pinned in the cluster when added via proxy endpoint?
Hope i am not missing something obvious.

2

hey @chaitanyaprem .

So the IPFS proxy in Cluster does not handle the dag/put command and this is forwarded directly to the underlying node.

How are you using this endpoint? Do you want a recursive pin for every added dag object? Even though the option is there, I find it a bit weird to dag/put and pin, so I would like to hear more about how/why it is used.

For the record, I don’t think dag/import nor block/put are handled by cluster either.

3

Hey @hector,

Thanks. I have also realized that dag/put is not intercepted after going through the documentation.

We are using IPFS for 2 aspects, to upload json like data snapshots (of data from specific smart contracts) using ipfs add and store IPLD objects which form a DAG-chain via ipfs dag put. As we run our own IPFS node, we want to pin all these objects and files so that they are available and not gc’d.
But, as the data keeps growing we are planning to migrate from a single IPFS node to a cluster to get around IPFS GC limitations.

In order to make it work as a drop-in replacement to single node, we have started using the proxy endpoint for the 2 operations i have mentioned above. If the proxy endpoint doesn’t support pinning dag put in the cluster, then one option is as you have mentioned perform 2 operations (first do dag/put and then pin the CID returned) which would not be optimal and would require specific code changes when using the cluster.

Is there any plan to intercept dag/put also at the proxy endpoint? This would simplify things as cluster and single IPFS nodes can be interchanged.
Is there any other alternative where-in by using a different endpoint we can dag/put and pin the CID in the cluster via single operation?

4

Sure, we can add this. I have added an issue for tracking: ipfsproxy: intercept dag/put and block/put and perform a cluster pin · Issue #1738 · ipfs-cluster/ipfs-cluster · GitHub

1 Like
5

Thanks @hector .

Any idea what would be the timeline for this to be available atleast as an RC candidate?

6

It can prob go in the next release… ~3 weeks?

7

Thanks @hector , will keep tracking the issue you have raised.

8

I have merged support for this to master. I can’t believe I’m on schedule with what I promised.

2 Likes
9

Oh wow, great to hear about this. We will pick it up in the next release.
Thanks a lot for the help @hector