I think probably someone else has mentioned this already and I’d be surprised if the developers haven’t already got plans to fix it:
Adding large amounts of data to IPFS seems to cause the I/O on the drive to go absolutely ape and crashes the computer.
This would be especially problematic on older PCs, and low end devices like phones that have slow storage etc.
Not to mention I can’t use IPFS for anything useful to me until I can add larger amounts of data.
It seems like the obvious solution for the time being, is to stop the system crashes, by setting a sensible limit on the I/O operations per second that IPFS makes. Better yet, do a test on the storage speed first before overloading it with too many read write operations.
Ideally though, adding the files wouldn’t be so demanding on the storage, I’m not sure what IPFS could be doing that’s so demanding on the disk I/O.
What OS are you using? Without information on why your computer crashes, it’s not immediately obvious to me why a lot of disk IO would cause it.
On Linux I’d be curious what syslog shows leading up to the “crash”.
When you add content using default settings, all of that content must be read from disk, broken up into chunks with valid multihashes, and then written into the IPFS repository as chunks.
There are ways to avoid most of the writes, but it requires the files that were added being mostly left as is after adding.
Hi, thanks for the prompt reply!
Unfortunately it’s taken me a long time to reply, as my system needed to be wiped after filesystem corruption.
I suspect this could have happened because of too many system crashes while the PC was at max I/O lol…
I re-ran the IPFS desktop appimage from github once I got Ubuntu reinstalled, but same problem, system crashed again.
I’m running Ubuntu Budgie 18.04.2
My system updates are current, and I’m now trying out a program called Akasha, which I found in the Ubuntu app store, and is supposed to be using IPFS.
Actaully wait. There’s tons of messages in that file.
I deleted it because there’s just tons of junk in there, but I ran the IPFS Desktop appimage again, and it didn’t generate any messages in /var/log/syslog when I started it. I added a few files and that went ok. But when I added a 5GB file, it really monopolised the SSD, and I don’t really want to add any folders full of files because it’ll just crash my PC again I think.
Maybe if I copied the files to the backup drive, and added them from there, it’d be ok, because the system wouldn’t become unstable then probably, but it’d still make my backup drive unusable while it’s processing the files.
Akasha downloaded the data it needed to download on startup, but then encountered some kind of problem and wouldn’t progress past the initial download screen.
Akasha uses more than just IPFS. None of your syslog output so far for Akasha appears to be related to IPFS itself.
syslog has logging for the entire system, so definitely don’t post it all.
When a crash happens, do you know what time it happens? If so, I’d suggest maybe looking at syslog entries leading up to the event. Filtering down to lines with error might be a good way to start.
Do you see any errors from the IPFS daemon leading up to the crash? If you’re not already, I would suggest logging it somewhere so it can be referenced if another crash happens.
FWIW, I think you’re likely encountering an issue with your machine – rather than an issue with IPFS specifically (and IPFS is just revealing it due to higher load). I run IPFS on a cheap laptop with Ubuntu as the OS – and I’ve never experienced a crash (with IPFS or in general).
The system becomes unresponsive, then eventually hangs. It’s just what normally happens when OS isn’t able to access the system files due to lack of available drive operations.