How big can the badgerdb get? [kubernetes planning]

Help
,
1

I’m running a 3 node ipfs-cluster on kubernetes and so far it’s been great, but I naively followed one example I found online and never thought to ask or investigate further into the /data/ipfs-cluster directory, I had set 5 GB of space for it but never stopped to think how fast would it grow.

After a month of using this setup, those 5 GB are filled up and though the fix is quick and easy, I’d like to know how fast will this grow in order to plan my resources for the future. Or even better, if there’s configuration parameter I’m missing for badger to optimize the size of it.

2

Cluster configuration allows to fine tune all badger-ds settings (v1.6.1). Right now it optimizes for lower memory usage, with no L0 compactions: https://github.com/ipfs/ipfs-cluster/blob/master/datastore/badger/config.go#L32-L40

It may be that playing with those settings helps, but in general the CRDT DAG in cluster grows and grows (how many pins do you have, what is the growth per pin?).

If you just have 3 nodes, you could manually compact it by doing an export+import operation (https://cluster.ipfs.io/documentation/guides/backups/#offline-state-export-and-import) on all the three nodes (or just in 1 of them, while doing a state cleanup in the others).