Adding a large amount of files to the cluster behind nginx ends up with an "unexpected EOF" error

Help IPFS Cluster
1

I set up a cluster and nginx proxy. When it comes to a small number of files, everything is fine both when loading with rest api and when using ipfs-cluster-ctl. But in the case of loading a large number of files - an error occurs:

2022-11-18T17:58:07.859Z	ERROR	apiclient	client/request.go:149	unexpected EOF
2022-11-18T17:58:07.859Z	ERROR	apiclient	client/lbclient.go:95	could not cast error into api.Error
error : unexpected EOF

The whole command looks like this:

ipfs-cluster-ctl --host /dns4/ipfs.my.site/tcp/443 --basic-auth "user:pass"  add -r --cid-version 1 --name some name .

I am trying to solve the problem for the second day, and have tried every possible configuration for nginx. At the same time, if I send a command to the address locally (/ip4/127.0.0.1/tcp/9094), everything goes fine.

I read that in the case of nginx there can be problems and should use “stream-channels=false”. Except that ctl doesn’t seem to have similar arguments and this parameter is always set to true… This is what it looks like in the nginx logs:

"POST /add?chunker=size-262144&cid-version=1&format=unixfs&hash=sha2-256&hidden=false&layout=&local=false&mode=recursive&name=TEST&no-pin=false&nocopy=false&progress=false&raw-leaves=true&recursive=true&replication-max=0&replication-min=0&shard=false&shard-size=104857600&stream-channels=true&user-allocations=&wrap-with-directory=false HTTP/2.0"

p.s
ipfs-cluster-ctl version 1.0.4

2

See the warning about /add and stream-channels here: REST API - Pinset orchestration for IPFS

3

This is exactly what I wrote about, but how to solve this problem in the case of ipfs-cluster-ctl? There is no such flag to change this parameter…

4

Sorry!

Yeah, there is no way to turn stream-channels off with ipfs-cluster-ctl.

Can you curl -X POST... the files instead, or is there something in ipfs-cluster-ctl you need? (curl will be more performant).