While reading into IPFS Cluster, I came across several warnings that trying to ping large fies could potentially result in timeouts,failed/corrupted uploads due to the length in time needed to pin files, as well as for requesting that data over teh IPFS network.
If I were to have three nodes (A,B,C) that form a cluster, all connected via a LAN network. Pin request comes in from the internet to node A to pin a very large file. After that file is pinned, a pubsub message is sent to nodes B,C to pin those files. Once nodes B,C have pinned the files, they trigger an event to the cluster to pin the content, and track it in the pinset.
The question comes down to whether or not this is a viable in-term option to use until any issues with pinning large files to ipfs cluster get resolved. If so, would it be optimal to send the pubsub message to nodes B+C to pin locally first? Or would it be enough to simply pin locally to node A, and then trigger the cluster update event.
Hey @postables,
Bitswap will requests blocks in the most efficient ways (currently) from the available providers so I doubt that the approach improves anything.
I came across several warnings that trying to ping large fies could potentially result in timeouts,failed/corrupted uploads due to the length in time needed to pin files, as well as for requesting that data over teh IPFS network.
Can you point me to the source of those warnings? I don’t remember writing anything about corruption, or about problems in the IPFS network that would prevent it from handling big files. There have been some issues in Cluster with pinning things that take long, but it’s mostly User Interface issues, and we’ve fixed quite a few already. I’m pretty sure this is a misinterpretation.
Oh okay thanks @hector I think you are it definitely was a misinterpretation, I’ll do some additional reading into bitswap then thanks for pointing that out… Would it be safer for me to download the file first and then submit it to the cluster myself if it is a particularly large file or should I just not even bother with that?
Cluster will call pin or refs -r (depending on pinning method). If the file is already available on the local peer it will take less time to pin. But in the end everything is the same. Next release of cluster will have more timeout control when cluster needs to download things which take a long time.
1 Like
Will it be recursive pin’s by default? I know the go-ipfs-api uses recursive pins by default, is that the same? Oh fantastic that’s great to hear, I look forward to having more fine tuned control over cluster pinning 
Btw fantastic work so far keep it up 
Yes, it pins recursively by default.