Scalability and Highload. Big Private Network and IPFS-CLuster

Help IPFS Cluster
1
  1. Scalability issues and deploying private network/cluster on more than 10s nodes.
  • How IPFS Private Network and IPFS-Cluster would behave with 100s of nodes?
  • What potential issues it could bring?
  • What would be better to have one big cluster or multiple smaller clusters?
  • Could multiple cluster idea solve Scalability issue?
  1. How many files in amount and value(mb,gb,tb,ā€¦) we can store per cluster/peer.
  • Are there any limitations?
  • Can I store millions of files per Peer/Cluster?
  1. How to reference IPFS clusters (e.g. by ID, IP etc.)?
  • Is it possible to have the same Peer ID(cluster) in different Clusters?
  • In case if I have multiple clusters I need identified from which cluster my fileā€™s CID.
  1. Can I replicate files to specific nodes in IPFS-Cluster (prevent replication to all nodes in cluster)?
1 Like
2

Currently, with Raft, badly. Raft cannot scale to 100s of nodes, but we plan to have an alternative by the end of Q1 which will do.

You could use multiple clusters + cluster composition but that becomes harder to manage.

Size limitations come from ipfs, but I think with experimental badgerDS it should be in the terabytes. Cluster is mostly going to set a limit in number of cluster-pinned items. Things like status sync operations will get slower as the number grows into the 100s of thousands and memory usage will grow. We have also some plans to address this rather soon. I am not sure how many pinned items ipfs can take though, but I think pin/unpin operations will degrade badly with really big amounts.

If they are completely isolatedā€¦ but I would not.

  • In case if I have multiple clusters I need identified from which cluster my fileā€™s CID.

If I understand this well, it would be up to you to track. If you use composition, you can use the ā€œover-clusterā€ to track which subcluster has what.

You can set the replication factor to lower than ā€˜everyoneā€™ (-1). We donā€™t support manual peer allocations per pin right now, but it can be done rather easily (open an issue).

2 Likes
3

Thank you Hector for detailed answer! Its helped me a lot.