Syncing MFS data between nodes?

Ecosystem
1

Hey folks, I started using MFS for storing encrypted application/dapp data.
I was wondering if it is possible to sync MFS data between different IPFS nodes?

This ipfs-cluster issue hints at the idea:

Sidenote, I know people use IPNS for application data, but it doesnā€™t feel like a viable option to me.
Reasons being, the performance and the short lifetime of a IPNS record.

2 Likes
2

We have encrypted data sync between nodes working pretty nicely via Textile: https://github.com/textileio/go-textile. Not quite MFS directly, but same ideaā€¦ structured, encrypted data on IPFS, synced between participating peers. Maybe something useful there?

1 Like
3

I need this too. Itā€™s only for a single user, myself, with a private cluster. Iā€™d like to somehow ā€˜pushā€™ mfs data changes to a node thatā€™s a ā€˜mfs serverā€™, then ā€˜pullā€™ only from that ā€˜mfs serverā€™. For me, a git type of workflow is preferable to syncing.

4

On Node A:

If you run IPFS-Cluster you can modify the MFS on one side, get the hash of the root folder with:

ipfs files stat --hash /foldername

Then replace the old pin in the cluster with the new one:

ipfs-cluster-ctl pin update /ipfs/<old-cid> /ipfs/<new-cid>

This will transfer the data to all cluster members efficiently. But you have to remove the old pin afterwards - if you donā€™t want to keep older versions.

ipfs-cluster-ctl pin rm /ipfs/<old-cid>

On node B:

The cluster follower/cluster daemon will pin the new CID on Node B.

You can either use cluster-ctl to get the new CID or the ipfs daemon itself.

I prefer the cluster-ctl since it shows the name of the pins in the cluster:

ipfs-cluster-ctl pin ls | grep "name of your pin" | cut -d ' ' -f 1

If this outputs two items, Node B havenā€™t catched up yet, you can still use the new CID:

ipfs files rm -r /yourfoldername
ipfs files cp /ipfs/<new-CID> /yourfoldername

Yes, this might not look very user-friendly, if you are very UI focused and just want to run the IPFS-Cluster Daemon in the background.

You can achieve the same in the WebGUI, without any cluster.

Node A: Unpin the folder, cange the folder content, copy the CID, transfer the CID to Node B.
Node B: Unpin the folder, remove the folder, hit ā€˜add from IPFSā€™, enter the new CID, rename it how you want the folder to be named, hit pin-add.

This will transfer all data from Node A to Node B manually.

2 Likes
5

Wow, thanks. The steps look easy enough to script.