IPFS Daemon keeps getting killed

Help
1

I’m using go-ipfs version 0.4.22.

I’m running nohup ipfs daemon & to keep the node up on a server. But somehow it’s getting killed frequently.

A strange thing I noticed in the logs was this:

Initializing daemon...
go-ipfs version: 0.4.22-
Repo version: 7
System version: amd64/linux
Golang version: go1.12.7
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/172.31.40.167/tcp/4001
Swarm listening on /ip6/::1/tcp/4001
Swarm listening on /p2p-circuit
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/172.31.40.167/tcp/4001
Swarm announcing /ip6/::1/tcp/4001
API server listening on /ip4/127.0.0.1/tcp/5001
WebUI: http://127.0.0.1:5001/webui
Gateway (readonly) server listening on /ip4/0.0.0.0/tcp/8080
Daemon is ready

Received interrupt signal, shutting down...
(Hit ctrl-c again to force-shutdown the daemon.)
19:59:56.490 ERROR       core: failure on stop:  context canceled builder.go:47

The interrupt part usually comes 5-6hr after I start the daemon.
But I haven’t hit CTRL+C, also no one else has access to the server.
Any idea how this could happen?

2

It seems your daemon gets killed right away? I can’t reproduce this, nohup seems to work for me.

I guess you could hack a way to know where the signal is coming from, but otherwise the canonical way of doing what you want to do is a systemd service: https://github.com/ipfs/go-ipfs/tree/164dd287bef18ba3a4a684d52cfbeb41446c3b54/misc/systemd

1 Like
3

Thanks for this. I will use systemd instead.:slight_smile:

P.S. I updated the logs in the question for more clarity.

4

I just tried running ipfs daemon with systemd and got the following error

Jan 22 20:17:08 ip-172-31-40-167 ipfs[27219]: WebUI: http://127.0.0.1:5001/webui
Jan 22 20:17:08 ip-172-31-40-167 ipfs[27219]: Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Jan 22 20:17:08 ip-172-31-40-167 ipfs[27219]: Daemon is ready
Jan 22 20:18:37 ip-172-31-40-167 systemd[1]: ipfs.service: Start operation timed out. Terminating.
Jan 22 20:18:37 ip-172-31-40-167 ipfs[27219]: Received interrupt signal, shutting down...
Jan 22 20:18:37 ip-172-31-40-167 ipfs[27219]: (Hit ctrl-c again to force-shutdown the daemon.)
Jan 22 20:18:37 ip-172-31-40-167 ipfs[27219]: 20:18:37.060 ERROR       core: failure on stop:  context canceled builder.go:47
Jan 22 20:18:37 ip-172-31-40-167 systemd[1]: Failed to start IPFS Daemon.
Jan 22 20:18:37 ip-172-31-40-167 systemd[1]: ipfs.service: Unit entered failed state.
Jan 22 20:18:37 ip-172-31-40-167 systemd[1]: ipfs.service: Failed with result 'timeout'.

But this time it got killed in a minute(see timestamps in the log). I will dig deeper into this. Maybe I’m just doing something silly here.

5

What’s your systemd service file definition? I see you’re using IPFS 0.4.22 but I’m guessing you may be using the systemd service definition from go-ipfs master.

The service definition in go-ipfs master expects the daemon to report a successful start via systemd’s notification system (Type=notify). If it doesn’t, systemd will think the unit has failed to start and will kill it.

Removing the Type=notify line from the service definition should fix this.

2 Likes
6

Thanks @stebalien!

That worked. Although, I needed to execute sudo systemctl daemon-reload as the system was picking the old ExecStart.

Here is the working systemd file:

[Unit]
Description=IPFS Daemon

[Service]
ExecStart=/usr/local/bin/ipfs daemon
KillSignal=SIGINT

[Install]
WantedBy=default.target
2 Likes