Add file slow As the number of documents increases

1

Hi everyone,

ipfs is go-ipfs v0.5.1, run in docker, there are no resource constraints to this container.

As the number of documents increases, the speed becomes slower and slower when add file to ipfs


# ipfs is idle, top show
~ top
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
14014 root      20   0 1529848 245212  19016 S   9.0  0.7   0:38.86 icfs

~ ls -lh myfile
-rw-r--r-- 1 root root 15 Dec  1 17:33 myfile

# add a small file to icfs cost about 2 seconds
~ curl -v -X POST -F file=@myfile "http://localhost:5001/api/v0/add"

# when add file to ipfs, top show
~ top
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
14014 root      20   0 1529592 251312  19012 S 183.2  0.8   0:31.77 icfs

~ du -h -d 1
106G    ./blocks
124M    ./datastore
0       ./keystore
106G    .

but use param
curl -v -X POST -F file=@myfile “http://localhost:5001/api/v0/add?only-hash=true”
curl -v -X POST -F file=@myfile “http://localhost:5001/api/v0/add?pin=false”
return soon

when can i do? i will upload pprof later

2

Upgrade to v0.10.0 before anything.

3

Upgrade to v0.10.0

[root@localhost .ipfs]# time echo "123" | /data/golang/source/go-ipfs/cmd/ipfs/ipfs add
added QmTEzo7FYzUCd5aq7bGKoMLfsbbsebpfARRZd4Znejb25R QmTEzo7FYzUCd5aq7bGKoMLfsbbsebpfARRZd4Znejb25R
 4 B / 4 B [=======================================================================================================] 100.00%
real    0m1.699s
user    0m1.943s
sys     0m0.156s

[root@localhost .ipfs]# /data/golang/source/go-ipfs/cmd/ipfs/ipfs version
ipfs version 0.10.0

[root@localhost .ipfs]# pwd
/root/.ipfs

[root@localhost .ipfs]# du -h -d 1
110G    ./blocks
8.0M    ./datastore
0       ./keystore
110G    .

[root@localhost .ipfs]# time dd if=/dev/zero of=bigfile bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 1.47131 s, 713 MB/s
real    0m1.480s
user    0m0.006s
sys     0m1.467s