Scalability and Highload. Big Private Network and IPFS-CLuster

  1. Scalability issues and deploying private network/cluster on more than 10s nodes.
  • How IPFS Private Network and IPFS-Cluster would behave with 100s of nodes?
  • What potential issues it could bring?
  • What would be better to have one big cluster or multiple smaller clusters?
  • Could multiple cluster idea solve Scalability issue?
  1. How many files in amount and value(mb,gb,tb,…) we can store per cluster/peer.
  • Are there any limitations?
  • Can I store millions of files per Peer/Cluster?
  1. How to reference IPFS clusters (e.g. by ID, IP etc.)?
  • Is it possible to have the same Peer ID(cluster) in different Clusters?
  • In case if I have multiple clusters I need identified from which cluster my file’s CID.
  1. Can I replicate files to specific nodes in IPFS-Cluster (prevent replication to all nodes in cluster)?
1 Like

Currently, with Raft, badly. Raft cannot scale to 100s of nodes, but we plan to have an alternative by the end of Q1 which will do.

You could use multiple clusters + cluster composition but that becomes harder to manage.

Size limitations come from ipfs, but I think with experimental badgerDS it should be in the terabytes. Cluster is mostly going to set a limit in number of cluster-pinned items. Things like status sync operations will get slower as the number grows into the 100s of thousands and memory usage will grow. We have also some plans to address this rather soon. I am not sure how many pinned items ipfs can take though, but I think pin/unpin operations will degrade badly with really big amounts.

If they are completely isolated… but I would not.

  • In case if I have multiple clusters I need identified from which cluster my file’s CID.

If I understand this well, it would be up to you to track. If you use composition, you can use the “over-cluster” to track which subcluster has what.

You can set the replication factor to lower than ‘everyone’ (-1). We don’t support manual peer allocations per pin right now, but it can be done rather easily (open an issue).

2 Likes

Thank you Hector for detailed answer! Its helped me a lot.