# No space left on device writing vlog file

**URL:** <https://discuss.ipfs.tech/t/no-space-left-on-device-writing-vlog-file/14923>\
**Category:** Help\
**Tags:** ipfs-cluster\
**Created:** [August 4, 2022, 8:41pm UTC](https://discuss.ipfs.tech/t/no-space-left-on-device-writing-vlog-file/14923 "2022-08-04T20:41:41Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ldapster](https://avatars.discourse-cdn.com/v4/letter/l/2acd7d/32.png) [@ldapster](https://discuss.ipfs.tech/u/ldapster)\
**Post date:** [August 4, 2022, 8:41pm UTC](https://discuss.ipfs.tech/t/no-space-left-on-device-writing-vlog-file/14923/1 "2022-08-04T20:41:41Z")

</div>

Hi,

We are noticing quite a few errors in our ipfs-cluster logs. The repeat over and over. We have a three node ipfs cluster (kubernetes) and two of the nodes/pods have these errors. The system seems to be running fine we are able to save and retrieve data. The file is: /data/ipfs-cluster/badger/000002.vlog

ERROR bitswap go-bitswap@v0.7.0/bitswap.go:492 Error writing 1 blocks to datastore: Unable to write to value log file: … : no space left on device  
2022-08-04 15:30:22.029 CDT2022-08-04T20:30:22.029Z ERROR badger badger@v1.6.2/logger.go:38 writeRequests: Unable to write to value log file: … : no space left on device  
022-08-04 15:30:22.029 CDT2022-08-04T20:30:22.029Z ERROR badger badger@v1.6.2/logger.go:38 WatchBatch.Cancel error while finishing: Unable to write to value log file:…: no space left on device

---

<div class="post-metadata">

**Author:** ![Jorropo](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/jorropo/32/3230_2.png) [@Jorropo](https://discuss.ipfs.tech/u/Jorropo)\
**Post date:** [August 4, 2022, 8:43pm UTC](https://discuss.ipfs.tech/t/no-space-left-on-device-writing-vlog-file/14923/2 "2022-08-04T20:43:37Z")

</div>

your disk are full.

The badger blockstore does very poorly when the disk is full, I would recomand using flatfs for serious workloads.  
Your data is probably corrupted.

---

<div class="post-metadata">

**Author:** ![ldapster](https://avatars.discourse-cdn.com/v4/letter/l/2acd7d/32.png) [@ldapster](https://discuss.ipfs.tech/u/ldapster)\
**Post date:** [August 4, 2022, 9:15pm UTC](https://discuss.ipfs.tech/t/no-space-left-on-device-writing-vlog-file/14923/3 "2022-08-04T21:15:23Z")

</div>

Thanks. So far the data seems fine. The devops guys seem to think GC will keep the filesystem clean, but it appears to me that GC is struggling with this issue.

---

<div class="post-metadata">

**Author:** ![Jorropo](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/jorropo/32/3230_2.png) [@Jorropo](https://discuss.ipfs.tech/u/Jorropo)\
**Post date:** [August 4, 2022, 9:19pm UTC](https://discuss.ipfs.tech/t/no-space-left-on-device-writing-vlog-file/14923/4 "2022-08-04T21:19:08Z")

</div>

Are you _actually_ running the GC ? It’s off by default, you need to do `ipfs repo gc` (one of run) or `ipfs daemon --enable-gc`.

GC has performance issues for now that why it’s not on by default.  
`ipfs-cluster` can do a robin GC scheduling where it will run it on like 33% of your cluster at once or smth like that. It’s better but more of a workarround than anything.

---

<div class="post-metadata">

**Author:** ![ldapster](https://avatars.discourse-cdn.com/v4/letter/l/2acd7d/32.png) [@ldapster](https://discuss.ipfs.tech/u/ldapster)\
**Post date:** [August 4, 2022, 9:57pm UTC](https://discuss.ipfs.tech/t/no-space-left-on-device-writing-vlog-file/14923/5 "2022-08-04T21:57:13Z")

</div>

Not sure how they are running it. I see these in the logs and assumed it was on:

2022-08-04T21:54:32.183Z INFO badger go-ds-badger@v0.3.0/datastore.go:441 Running GC round  
2022-08-04T21:54:43.592Z INFO badger go-ds-badger@v0.3.0/datastore.go:443 Finished running GC round  
2022-08-04T21:54:43.592Z ERROR badger go-ds-badger@v0.3.0/datastore.go:201 error during a GC cycle: Iteration function. Path=/data/ipfs-cluster/badger/000000.vlog. Error=Unable to write to value log file: “/data/ipfs-cluster/badger/000002.vlog”: write /data/ipfs-cluster/badger/000002.vlog: no space left on device

---

<div class="post-metadata">

**Author:** ![Jorropo](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/jorropo/32/3230_2.png) [@Jorropo](https://discuss.ipfs.tech/u/Jorropo)\
**Post date:** [August 5, 2022, 1:17am UTC](https://discuss.ipfs.tech/t/no-space-left-on-device-writing-vlog-file/14923/6 "2022-08-05T01:17:17Z")

</div>

The GC need to shuffle a few blocks around maybe, but you literally have no space left so it can’t and fail.
