# Constant 100% CPU utilization on AWS EC2 IPFS node **URL:** \ **Category:** Help\ **Tags:** go-ipfs\ **Created:** [November 6, 2023, 12:34pm UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172 "2023-11-06T12:34:55Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![zxriptor](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/zxriptor/32/8094_2.png) [@zxriptor](https://discuss.ipfs.tech/u/zxriptor)\ **Post date:** [November 6, 2023, 12:34pm UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/1 "2023-11-06T12:34:55Z")
We see a constant 100% CPU load on our t3a.large EC2 instance, a machine goes unresponsive sometimes due to this. We use this node to store project files, the team causes a low activity as this is our development environment. Repo is also not huge, less than 4Gb. What I already tried: - upgraded from 0.19.0 to 0.23.0 - applied lowpower profile This did not help at all. Is there any way to narrow down what subsystem causes that load? I enabled debug logs and collected CPU profiling data but those didn’t give me any clue about what is going on. ---
**Author:** ![Jorropo](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/jorropo/32/3230_2.png) [@Jorropo](https://discuss.ipfs.tech/u/Jorropo)\ **Post date:** [November 6, 2023, 12:38pm UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/2 "2023-11-06T12:38:04Z")
Can you run `ipfs diag profile` and post it here ? This will capture a profile of where cpu time is spent. ---
**Author:** ![zxriptor](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/zxriptor/32/8094_2.png) [@zxriptor](https://discuss.ipfs.tech/u/zxriptor)\ **Post date:** [November 6, 2023, 12:52pm UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/3 "2023-11-06T12:52:56Z")
Thank you for your prompt reply. I didn’t find a way to upload zip so here is the dropbox link to the archive [Dropbox - ipfs-profile-2023-11-06T12\_39\_12Z\_00.zip - Simplify your life](https://www.dropbox.com/scl/fi/02or26im65sfhfkf6729r/ipfs-profile-2023-11-06T12_39_12Z_00.zip?rlkey=d9n7e6ekaym5vna8ml5hwgpxs&dl=0) ---
**Author:** ![Jorropo](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/jorropo/32/3230_2.png) [@Jorropo](https://discuss.ipfs.tech/u/Jorropo)\ **Post date:** [November 6, 2023, 1:00pm UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/4 "2023-11-06T13:00:47Z")
Thx this is clearly a bug somewhere. We seems to be doing some weird timer things. ![Screenshot from 2023-11-06 13-55-34](https://us1.discourse-cdn.com/flex020/uploads/ipfs1/original/2X/8/85a3e18216f9e22e6872f1ca9b08a4f69d9578c9.png) I don’t know on the top of my head. ---
**Author:** ![Jorropo](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/jorropo/32/3230_2.png) [@Jorropo](https://discuss.ipfs.tech/u/Jorropo)\ **Post date:** [November 6, 2023, 1:03pm UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/5 "2023-11-06T13:03:19Z")
This could be a `time.Ticker` that is leaked. ---
**Author:** ![Jorropo](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/jorropo/32/3230_2.png) [@Jorropo](https://discuss.ipfs.tech/u/Jorropo)\ **Post date:** [November 6, 2023, 1:12pm UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/6 "2023-11-06T13:12:41Z")
![Screenshot from 2023-11-06 14-04-59](https://us1.discourse-cdn.com/flex020/uploads/ipfs1/original/2X/e/e638ed9da73ef448db0f06e511feaa4e2889357e.png) I pushed a fix here: [swarm: fix timer Leak in the dial loop by Jorropo · Pull Request #2636 · libp2p/go-libp2p · GitHub](https://github.com/libp2p/go-libp2p/pull/2636) If go-libp2p makes a `v0.32.1` backport release I’ll try to include this in Kubo `v0.24.0`. ---
**Author:** ![zxriptor](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/zxriptor/32/8094_2.png) [@zxriptor](https://discuss.ipfs.tech/u/zxriptor)\ **Post date:** [November 6, 2023, 1:24pm UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/7 "2023-11-06T13:24:24Z")
Thank you very much for your help! I’ll continue to watch for that in github. Is there any workaround you can suggest? We are fine to limit swarm functionality temporarily (if it is possible) till the fix makes its way to the release. ---
**Author:** ![Jorropo](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/jorropo/32/3230_2.png) [@Jorropo](https://discuss.ipfs.tech/u/Jorropo)\ **Post date:** [November 6, 2023, 1:45pm UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/8 "2023-11-06T13:45:10Z")
I don’t know of a workaround. ---
**Author:** ![Jorropo](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/jorropo/32/3230_2.png) [@Jorropo](https://discuss.ipfs.tech/u/Jorropo)\ **Post date:** [November 8, 2023, 9:42am UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/9 "2023-11-08T09:42:44Z")
Thx to go-libp2p making a v0.32.1 this should be fixed in Kubo v0.24.0, it should release later today. ---
**Author:** ![zxriptor](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/zxriptor/32/8094_2.png) [@zxriptor](https://discuss.ipfs.tech/u/zxriptor)\ **Post date:** [November 10, 2023, 10:17am UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/10 "2023-11-10T10:17:18Z")
It is better now, thank you for the quick fix. Utilization doesn’t stay 100% all the time however there are periodic prolonged spikes to 80-95% that I hardly can explain due to its cyclical look. ![cpu-utilization](https://us1.discourse-cdn.com/flex020/uploads/ipfs1/original/2X/0/0bd6c0635d368c1cdb756f004c29834df78fdcba.png) ---
**Author:** ![Jorropo](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/jorropo/32/3230_2.png) [@Jorropo](https://discuss.ipfs.tech/u/Jorropo)\ **Post date:** [November 10, 2023, 11:04am UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/11 "2023-11-10T11:04:19Z")
That is still an improvement however it does not surprise me that you have an other issue because the bug in go-libp2p was there for a few releases and yet you are the first one to report it (it would not leak in usual operations). If you could capture a new profile which capture the spike this would be nice, you can increase the duration with `ipfs diag profile --profile-time 1m` if you need something longer to catch the spike. ---
**Author:** ![zxriptor](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/zxriptor/32/8094_2.png) [@zxriptor](https://discuss.ipfs.tech/u/zxriptor)\ **Post date:** [November 10, 2023, 11:14am UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/12 "2023-11-10T11:14:40Z")
Here you go [Dropbox - ipfs-profile-2023-11-10T11\_09\_49Z\_00.zip - Simplify your life](https://www.dropbox.com/scl/fi/ejjj5eujr7xi7tjlsswsz/ipfs-profile-2023-11-10T11_09_49Z_00.zip?rlkey=oo7tzsfwmr7luxssz6059cfro&dl=0) ---
**Author:** ![zxriptor](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/zxriptor/32/8094_2.png) [@zxriptor](https://discuss.ipfs.tech/u/zxriptor)\ **Post date:** [November 15, 2023, 5:40pm UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/13 "2023-11-15T17:40:56Z")
@Jorropo Is that an issue also caused by the timers thing? I am wondering if maybe we can downgrade to one of the previous stable versions where this bug does not exist. This issue is really major for us as it has a high impact on our system performance now. ---
**Author:** ![Jorropo](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/jorropo/32/3230_2.png) [@Jorropo](https://discuss.ipfs.tech/u/Jorropo)\ **Post date:** [November 18, 2023, 8:10am UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/14 "2023-11-18T08:10:07Z")
I’m looking into it, after discussing with some libp2p maintainers the only possible way for this bug to trigger the way it did is for you to do lots of dial attempts on nodes that have zero addresses. Do you know if anything in your workflow could make you dial unreachable peers ? Or peers we can’t find addresses for ? ---
**Author:** ![zxriptor](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/zxriptor/32/8094_2.png) [@zxriptor](https://discuss.ipfs.tech/u/zxriptor)\ **Post date:** [November 22, 2023, 7:00pm UTC](https://discuss.ipfs.tech/t/constant-100-cpu-utilization-on-aws-ec2-ipfs-node/17172/15 "2023-11-22T19:00:47Z")
It is hard to answer this as this is too low-level stuff for me. We do nothing beyond basics: interacting with MFS, and publishing IPNS names. If you have any specific requests, I am happy to get this information for you. Should I look at bootstrap addresses or anywhere else?