I have a quick update to share.
We’ve made progress and now the following implementations will yield the same CID for UnixFS given the same input (with the exception of empty folders which we should probably make configurable)
- GitHub - storacha/ipfs-car: 🚘 Convert files to content-addressable archives and back which as of v3.0.0 defaults to 1MiB chunks and 1024 children and raw blocks
- Helia which also defaults to 1MiB chunks and 1024 children, and raw blocks. Source for example
- Kubo, which added a new
test-cid-v1-wideprofile with these defaults of v0.35:ipfs config profile apply test-cid-v1-wideto apply
Here you can see it in action
# Add with Kubo 0.35
➜ helia-101 git:(main) ✗ ipfs --version
ipfs version 0.35.0
➜ helia-101 git:(main) ✗ ipfs config profile apply test-cid-v1-wide > /dev/null
➜ helia-101 git:(main) ✗ ipfs add -r test
added bafkreie4vttikpz5imd5ccfoanozyohpabvxlgovbjr5d4w7y3xep5m6ki test/index.spec.js
added bafybeifrc2vrh76j7dccg2hgihoy66su7jw2vvxoihrswevbdaazlquhpq test/war-and-peace.html
added bafybeiahi2rfez66oxcyeyrwniq7kktzeqf3bkffuyktbpwxlppw3k6b7i test
3.85 MiB / 3.85 MiB [===================================================================================================================================] 100.00%
# Add with helia and @helia/unixfs
➜ helia-101 git:(main) ✗ node helia-add-unixfs.js test '**/*' helia.car
CID(bafkreie4vttikpz5imd5ccfoanozyohpabvxlgovbjr5d4w7y3xep5m6ki) /index.spec.js raw
CID(bafybeifrc2vrh76j7dccg2hgihoy66su7jw2vvxoihrswevbdaazlquhpq) /war-and-peace.html file
CID(bafybeiahi2rfez66oxcyeyrwniq7kktzeqf3bkffuyktbpwxlppw3k6b7i) directory
Wrote car file to helia.car
# Add with ipfs-car v3
➜ helia-101 git:(main) ✗ ipfs-car -v
ipfs-car, 3.0.0
➜ helia-101 git:(main) ✗ ipfs-car pack test > ipfs-car.car
bafybeiahi2rfez66oxcyeyrwniq7kktzeqf3bkffuyktbpwxlppw3k6b7i
➜ helia-101 git:(main) ✗ ipfs-car ls ipfs-car.car --verbose
bafybeiahi2rfez66oxcyeyrwniq7kktzeqf3bkffuyktbpwxlppw3k6b7i - .
bafkreie4vttikpz5imd5ccfoanozyohpabvxlgovbjr5d4w7y3xep5m6ki 1373 ./index.spec.js
bafybeifrc2vrh76j7dccg2hgihoy66su7jw2vvxoihrswevbdaazlquhpq 4039247 ./war-and-peace.html
# Here you can see the blocks
➜ helia-101 git:(main) ✗ ipfs-car blocks ipfs-car.car (
bafkreie4vttikpz5imd5ccfoanozyohpabvxlgovbjr5d4w7y3xep5m6ki
bafkreib4xkrayqwopys7y4zv27baf2ga3yiswfitnfssruqafer6hqbktu
bafkreieu3cnito5c7ebd4ceri3dvhoy5xw73vvf4nut4v2jualhqlftb3q
bafkreibrvwq4goy4dmqviggvxvzty7wuqzwztggl7szkwht2kdek4ngmri
bafkreib433qkwzzkpxa2tt5cst5gb5scfu32t3tptt34mxa22i7ttdc5iy
bafybeifrc2vrh76j7dccg2hgihoy66su7jw2vvxoihrswevbdaazlquhpq
bafybeiahi2rfez66oxcyeyrwniq7kktzeqf3bkffuyktbpwxlppw3k6b7i
This already puts us already in a much better place. Ongoing spec work can be found in the specs repo: IPIP 0499: CID Profiles by mishmosh · Pull Request #499 · ipfs/specs · GitHub