Is there any provision to choose ipfs-cluster peer for content replication?

Help IPFS Cluster
#8

I have 3 nodes connected in the cluster.
I did ipfs-cluster-clt pin add --allocations PEERID1 --replication 1

Now the node which had the file in local is offline. So, the third peer is trying to get the contents of the file using

ipfs cat . But the cursor in the terminal just hangs. The cid is pinned to PEERID1 so the other peer should be able to retrieve the content right?

Please let me know if I am missing something or doing wrong.

Thank you

#9

Assuming PEERID1 is well connected to the network, can be dialed-in etc, then yes.

#10

They all are privately connected through the swarm.key and when I run the nodes they show the PEERID1 that they are connected too. I have added the address in the peerstore.

But still it is not able to retrieve the contents after pinning to PEERID1 and when the peer which have the actual file goes offline.

#11

ipfs-cluster-ctl status shows one CID is PINNING and another PIN_ERROR context deadline exceeded.

That’s the reason I am unable to extract data at the third peer

#12

yeah well, that means the content is not available in any of the online-cluster peers if PEERID1 was still pinning.

#13

Yes. Sure thank you.

Can I know if there is any reason why it is taking a lot of time to pin ? I was trying to add a file CID which is of 1kb in size.

I have tried --wait option too. But it was failing with error couldn’t get the pin status after a while.

How do I solve this issue.

Thank you.

#14

Is the peer that is pinning connected to the peer that has the content?

#15

Yes it does show that it is connected. It displays the connected cluster peer id on the node which has the content

#16

Sorry, can you explain again your setup? Is the node that has the content part of the cluster? I’m confused about what is running and what is not running when you pin.

#17

Sure.

Assumption all ports are open.

PEER 1, 2, 3 are the three nodes in the cluster.

Want to make PEER 1 as a central node in the cluster where every other node in the cluster will pin their file hashes to PEER 1.

Commands executed on three nodes are :

  • ipfs init --profile=badgerds
  • created a swarm.key and placed it in .ipfs in all the three nodes
  • ipfs-cluster-service init on all three nodes
  • Copied the cluster-secret key of one node and replaced it in other two nodes so as to be in the same network cluster
  • Added the PEER 2, 3 addresses in PEER 1 peerstore
  • Added the PEER 1 address in PEER 2 and PEER 3 in their respective peerstores
  • Then started ipfs daemon and ipfs-cluster-service daemon on all three machines
  • Saw the PEER 2, 3 peerid’s on PEER 1 command prompt after running ipfs-cluster-service daemon
  • Saw PEER 1 peerid on PEER 2, 3 command prompt after running ipfs-cluster-service daemon
  • Added a file on PEER 2 using : echo “Hello world” | ipfs files write --create --raw-leaves /test.txt
  • Copied the generated hash/CID of the file
  • Then ran ipfs-cluster-ctl pin add --wait --replication 1 --allocations <PEER 1 cluster id>
  • The status showed pinning and failed later with error msg saying context deadline exceeded.

Please let me know if I have to do wrong or if you didn’t get me.

I’m not sure if we have to add addresses in the peerstore of PEER 1.

Thank you.

#18

I think you did things right wrt cluster. This looks like a problem with IPFS daemons not being connected.

What is ipfs swarm peers showing?

They are in a private network so they cannot rely on bootstrappers. You need to let them bootstrap to, for example, the ipfs daemon in peer one. Cluster attempts to connect ipfs daemons among peers, but this depends on what IPs they are reporting to have. Are peers on the same LAN? If not, you need to ensure that IPFS daemons are connected among themselves too.

As a small node, I would re-use the same peerstore file for all 3 cluster peers, containing all 3 addresses. You want Peer2 and Peer3 to be connected too to each other.

1 Like
#19

PEER1 and PEER2 are not on same LAN so as you mentioned I connected them through swarm connect.

Then I was able to pin the file on the PEER1. I could see the status as pinned.

I don’t want to connect any other nodes in the cluster.
The model looks like many to one (n-1 nodes in the cluster which are all connected to the 1 remaining node in the cluster).

When I did from PEER2
ipfs-cluster-ctl pin add --wait <file CID/hash>
The file cid was pinned on the PEER1. I could see the status as PINNED

But when I do
ipfs-cluster-ctl pin add --wait --replication 1 --allocations <file CID/hash>
The status shows as REMOTE.

Not sure on how to pin a cid on to a particular node/peer.

What is the value to be used for --allocations flag ? Is it the ipfs id? or ipfs-cluster-ctl id?
And is it sufficient if we give the id itself of the whole address like /ip4//tcp/9096/p2p/

Thank you.

#20

It’s the cluster peer IDs.

Note if your replication factor is 1, and you already pinned in 1 peer, I think it won’t pin it somewhere else because the replication factor is satisfied. The allocations flag is a priority list of where things should be pinned, but it does not need to make that decision since things are already pinned. So if you want to move the pin from one allocation to other the best way is to unpin and then pin again.

#21

Yes. I think that was the problem, as the replication value is satisfied it wasn’t pinning. So, I changed the replication flag value to 2 and gave one allocation value (PEER 1) and then it pinned. Showing the two peerid values in the allocation.

#22

What does the metadata column in pin add command in cluster-ctl?
How is it useful and what is the data to be provided to it ?

Thank you

#23

You can store arbitrary information with the pin.

#24

Is there any flag or option to add a meta data to ipfs files write command, when we add a file to ipfs?

#25

The hashes that we pin on to the cluster node are shown in ipfs pin ls.

How do we add the file paths in such a way when we do ipfs files ls on the node shows the paths added by the other nodes.

I used ipfs-cluster-ctl add filename.txt
The above command seems to work same as ipfs add as they don’t show on ipfs files ls until and unless explicitly we run ipfs cp hash /filename.txt

So, doing cluster-ctl add command doesn’t show the file paths in ipfs files ls in the node that we are trying to add too.

  1. Is there any way on how we can add file paths when we pin the hashes from local to remote node in the cluster?
  2. ipfs-cluster-ctl add doesn’t show up file names in remote machine when we type ipfs files ls. and I didn’t find any ipfs-cluster-clt cp command so that the hash can be assigned to path and can be listed in ipfs files ls.

Thank you

#26

That is not possible. Cluster is just going to pin or unpin, it does not deal with MFS.

#27

Sure thank you for the help :slight_smile: