Docker --enable-gc

Help
1

Hi,

I’ve been running using the Docker commands and after leaving it for a day or two, then entire servers storage has been consumed. It appears that IPFS Docker doesn’t run with the garbage collection enable and the Docker file doesn’t use the ‘–enable-gc’ flag. Running the gc manually freed up 85% of the storage.

Is there any reason IPFS Docker doesn’t have gc enable by default? Looking through the Docker file and the source code of GitHub - ipfs/go-ipfs: IPFS implementation in Go there is no option to enable gc. How do people enable it without forking and adding the option to the Docker file?

2

Is there any reason IPFS Docker doesn’t have gc enable by default?

The GC is slow, it pause the datastore and retrace all blocks starting from your pins making your node unusable from a few seconds to a few minutes depending of your datastore size.

The plan is to change that to refcounting in the future so this could be made partially locking and / or really fast.

3

Is there a way to enable in Docker?

A node with full storage makes the IPFS endpoint suffer worse consequences. As in, once the storage is full, you can’t load any new objects and basic commands like unpinning fail because it can’t write to the storage. Plus the server SSD I’m using runs at 200,000 IOPS (3GB/s), so any block retrace should be quick.

4

Add --enable-gc to the run line in the docker file ?

5

I don’t belive the botleneck are actual just IOPS issues, GC in IPFS isn’t that smart and will still take a few seconds.

6

Add --enable-gc to the run line in the docker file ?

I was trying to avoid that, but if it’s the only way.

I don’t belive the botleneck are actual just IOPS issues, GC in IPFS isn’t that smart and will still take a few seconds.

Still a lot better than the gateway / whole server being unusable until it’s manually cleared.

I tried building the Dockerfile against the 0.9.1 Git tag but the resulting image came back with

ipfs version 0.9.1
Found IPFS fs-repo at /data/ipfs
Error: Unknown Command "/bin/sh"

For now I’ll just run a cron job that runs the gc clean command.

Thanks for your help!

1 Like