Go-ds-s3 configuration

Help
1

Hi guys,

I am currently configuring my IPFS to use S3 as its datastore.
I used these cmds

ipfs init --profile server

ipfs config --json Datastore.Spec.mounts "[{\"child\":{\"type\":\"s3ds\",\"region\":\"$S3_REGION\",\"bucket\":\"$S3_BUCKET\",\"rootDirectory\":\"$S3_DIR\",\"accessKey\":\"$S3_ACCESS_KEY\",\"secretKey\":\"$S3_PRIVATE_KEY\"},\"mountpoint\":\"/blocks\",\"prefix\":\"s3.datastore\",\"type\":\"measure\"},{\"child\":{\"compression\":\"none\",\"path\":\"datastore\",\"type\":\"levelds\"},\"mountpoint\":\"/\",\"prefix\":\"leveldb.datastore\",\"type\":\"measure\"}]"

echo "{\"mounts\":[{\"bucket\":\"$S3_BUCKET\",\"mountpoint\":\"/blocks\",\"region\":\"$S3_REGION\",\"rootDirectory\":\"$S3_DIR\"},{\"mountpoint\":\"/\",\"path\":\"datastore\",\"type\":\"levelds\"}],\"type\":\"mount\"}" > $IPFS_PATH/datastore_spec

ipfs daemon --migrate=true --agent-version-suffix=docker

And encounter the error

Error: datastore configuration of '{"mounts":[{"mountpoint":"/blocks","path":"blocks","shardFunc":"/repo/flatfs/shard/v1/next-to-last/2","type":"flatfs"},{"mountpoint":"/","path":"datastore","type":"levelds"}],"type":"mount"}' does not match what is on disk '{"mounts":[{"bucket":"my-test-bucket","mountpoint":"/blocks","region":"ap-southeast-1","rootDirectory":"my-test-directory"},{"mountpoint":"/","path":"datastore","type":"levelds"}],"type":"mount"}'

I am aware that IPFS has a mechanism to prevent the mismatch config of datastore. So I tried to let IPFS generate the datastore_spec itself by follow the following cmds

ipfs init --profile server

ipfs config --json Datastore.Spec.mounts "[{\"child\":{\"type\":\"s3ds\",\"region\":\"$S3_REGION\",\"bucket\":\"$S3_BUCKET\",\"rootDirectory\":\"$S3_DIR\",\"accessKey\":\"$S3_ACCESS_KEY\",\"secretKey\":\"$S3_PRIVATE_KEY\"},\"mountpoint\":\"/blocks\",\"prefix\":\"s3.datastore\",\"type\":\"measure\"},{\"child\":{\"compression\":\"none\",\"path\":\"datastore\",\"type\":\"levelds\"},\"mountpoint\":\"/\",\"prefix\":\"leveldb.datastore\",\"type\":\"measure\"}]"

// trying to re-init IPFS
mv $IPFS_PATH/config /somewhere/config.temp
rm -rf $IPFS_PATH/datastore_spec
ipfs init /somewhere/config.temp
rm /somewhere/config.temp

// start daemon
ipfs daemon --migrate=true --agent-version-suffix=docker

This works.

But I wonder if this is a correct way to re-init the IPFS to configure datastore.

Please give me your advice.

Thanks in advance.

2

Hello,
I’ve been researching this issue myself. My understanding is that the config file is only read during node initialization. I don’t think it’s possible to re-init the node and have not been able to find any command suggesting you can. The only way is to migrate the datastore as you have.

I’ll keep digging and update you if I find a way to re-init.

3

Hi,

My purpose is just a workaround to let IPFS re-generate the datastore_spec file by backup the config file, run ipfs init again with the backup file.

It works so far. Just wondering this way will cause any harm to my IPFS node.