Cluster Auto Pinning after MFS Writes

Help
1

Hello guys,
I have a two node cluster and I’d like to replicate the whole pinning and increase the availability of my public ipfs gateways. Thus in the future I’d like to increase these nodes in the cluster.
My application creates new files upon the user request and then writes to MFS by using js-ipfs API and this application resides locally in each ipfs node.
I know that writing MFS would automatically pin the file on that node. However I’m curious about that writing to MFS of a node would then somehow trigger the pinning in the cluster ?
What is the proper way of doing this kind of flow ?

Thanks in advance !

2

Currently there is no way of automatically following MFS root. You would have to have a helper process on the side that performs a cluster pin-update when the MFS root is updated.

3

Thank you Hector,
Then I’m going to send HTTP requests to cluster rest API in port 9094. I’m looking at the paths here REST API - Pinset orchestration for IPFS
Should I first pin the file written in MFS with /pins/{cid}, then call /pins/{cid}/sync ?

4

Actually, the /sync endpoints don’t exist anymore. I have updated the documentation.

You just need to POST /pins/{new_mfs_root}?pin-update=<old_mfs_root>, followed by an optional DELETE /pins/old_mfs_root (because cluster does not unpin old_mfs_root).

Since MFS DAGs are going to be similar, using pin-update query option will speed up the pinning process by pruning branches that are known to exist. Cluster will also allocate the new pin to the same peers. (see ipfs-cluster-ctl pin update --help and ipfs pin update --help).

5

I understand, thanks.
Just a thought maybe “named pins” would help to identify distinct pins and gives to power to maintainer to choose which pin to update which pin to remove :slight_smile:

6

As pin POST /pin/<hash>?pin-update=mfs ?

Yeah, the problem is non unique names and that everything is keyed by CID.

7

Sorry it was already implemented in the command line ipfs-cluster-ctl
--name value, -n value Sets a name for this pin

I’ve tried to pin the file in MFS but couldn’t get it done, what am I missing here ?
curl -X POST http://localhost:5001/api/v0/files/stat?arg=/main_directory/file-1.json gives the file data successfully.
But cluster api
curl -X POST http://localhost:9094/pins/main_directory/file-1.json gives me {"code":404,"message":"not found"} response.

8

you need to pin the CID of that MFS folder (not the path).

You can also pin the root cid of MFS though, if you want to have a single pin rather than one per item (as shown on ipfs files stat /).

9

Ok, I understand. I’ve pinned CID and it works like a charm.
One last question a little off-topic. Is there any way to directly return CID after calling ipfs.files.write ?
Currently I’m calling ipfs.files.stat after each write for pinning CID

10

I don’t think so, no.

11

Okay, thank you Hector !