Hosting a collaborative cluster

1

I am trying to host a collaborative cluster with follower nodes following a domain name. The commands for the follower node is clearly given. I need some help in hosting the collaborative cluster based on a webpage. Basically, the setup involves a webpage receiving updates frequently and the follower nodes in the cluster need to pin the updates regularly in the local storage.
It will be helpful to know how to host a collaborative cluster for this scenario.

2

It seems like what you’re talking about doing is a collaborative cluster, except instead of a follower feed, you’re publishing the updates to the feed on a webpage? If this is what you’re talking about, not sure why you would be doing this, as opposed to a collaborative cluster as is, with the ipfs-cluster-follow command.

3

Thanks for your reply. We have tried hosting a collaborative cluster with a base node, few peer nodes, and follower nodes. The follower nodes just follow a peer node and whenever a new file is pinned to the collab cluster the follower node is updated of the same. This experiment is working fine.
Now we need to make the follower node follow a domain name for which the documentation explains clearly. But there is less documentation to explain how to host a collab cluster when there is a webpage that is receiving updates regularly and follower nodes need to pin all the updates.

We have a website based on IPFS and linked it to the DNSLINK record.

Any help to enable this scenario will be a great help.

4

But there is less documentation to explain how to host a collab cluster when there is a webpage that is receiving updates regularly and follower nodes need to pin all the updates.

Not sure why you would do this. You can use DNSLink to distribute the configuration file used by the followers to join a particular cluster. The trusted nodes in the cluster can then add new data to the cluster (ipfs-cluster-ctl add $HASH) and the followers will automatically start pinning the data you added to the cluster. Abstracting this update process to a web page isn’t really a collaborative cluster, and is more akin to updating an authoritative source for data to be pinned by people interested in the data. If this is what you want, I don’t think collaborative clusters can be used.

This is the flow of collaborative clusters:

  • Trusted peer generates a configuration file
  • Trusted peer distributes this configuration file, via DNSLink or some other means
  • Using ipfs-cluster-follow third-parties will pull in this configuration file and start mirroring the data
  • Trusted peer wants to add new data for the cluster and third-parties to mirror via ipfs-cluster-ctl add or ipfs-cluster-ctl pin add
  • Follower peers automatically pick this up and start mirroring the data

Now we need to make the follower node follow a domain name for which the documentation explains clearly

I think you’re misinterpreting the documentation. This “domain name” is to distribute the configuration file

6

Thank you for your answer. That was helpful. One problem we are facing while hosting the collaborative cluster. When all the nodes (base node, trusted peers and followers) are connected to the same networks, the cluster is running without any issues. But if the follower node is connected to some other network then it is not following the collaborative cluster. How to resolve this issue? Also when we just follow a collaborative cluster hosted by filecoin, there are no issues in following it.

7

Is the trusted peer that the follower node tries to connect to reachable? Have you provided a multiaddress where it can be reached

This is done usually via peer_addresses: https://cluster.ipfs.io/documentation/collaborative/setup/#creating-a-configuration-template-for-followers

On a LAN, the peers may just autodiscover without further configuration using mDNS, but otherwise you need to provide addresses so that they can bootstrap.