Create a Follower Node that Ignores Pinset


I would like to create a follower node on a resource constrained system, I would like it connected to the cluster so data residing on it can be added to the pinset but not to download and pin anything in the pinset. Basically create an “uploader” node. Is there anyway to achieve this behavior through current configuration settings or would I have to fork the code? If a fork is required is there any suggestions on the best way to implement this feature?


You could run ipfs-cluster-service without running an ipfs node at all. It is going to throw a lot of errors to the logs coming from the ipfshttp facility when it tries to talk to IPFS, but you can silence them with --loglevel ipfshttp:critical (I think). This would only be for clusters that are pinning with replication factor -1.

If you are pinning with a set replication factor, then you can make sure your “follower node” is not allocated any pins by explicitly setting the list of places where things should be pinned, or by configuring MaxStorage in IPFS to be 0 and ensuring other nodes have more free storage.

If you were to fork the code, you could modify ipfsconn/ipfshttp component to no-op on pin/unpin requests and report everything as pinned when asked. Even better would be to add a new ipfsconn/dummy implementation that can be selected in the configuration, similar to how raft/crdt can be selected.

A different possibility would be to add the files pinned to the local IPFS, use ipfs-cluster-ctl to connect to a cluster member which can write to the cluster via --host and add the stuff to the cluster. The assigned cluster nodes for this pin would download the stuff from the node.

After the pinning operation is completed you can unpin the data from the local node again, to make sure the GC can do it’s work if the storage gets full.

And I had the same idea/need a while ago. I created a feature request for this:

Thanks. Setting the MaxStorage to 0 sounds like the quick work around I was hoping for, however I am unsure if that actually prevent the node from downloading the pinset or does it just trigger the GC to constantly run? It looks like there are still some open issues pointing to the latter functionality.

Well, this “trick” would only work if the cluster has not -1 replication (which means replicate everywhere) IIRC.

And yes, you probably the GC should be triggered manually, but not sure about that.