Should we profile CIDs?

Protocol
37

Also, DASL is an incomplete solution the problems described above because it doesn’t address chunking and DAG width for larger files.

So we are still pursuing profiling to get predictable, comparable CIDs.

Here is a summary table of current defaults, thanks to input & clarifications from @danieln @achingbrain @lidel:

Helia default Kubo default Storacha default “test-cid-v1” profile DASL
CID version CIDv1 CIDv1 CIDv1 CIDv1 CIDv1
Hash Algo sha-256 sha-256 sha-256 sha-256 sha-256
Chunk size 1MiB 256KiB 1MiB 1MiB not specified
DAG width 1024 174 (but it’s complicated*) 1024 174 not specified
DAG layout balanced balanced balanced balanced not specified
  • Kubo has 2 different default DAG widths:
    • For HAMT-sharded directories, the DefaultShardWidth here is 256.
    • For files, DefaultLinksPerBlock here is ~174

Kubo currently has no CLI / RPC / Config option to control DAG width in Kubo. ipfs add: parameter to control the width of created UnixFS DAG · Issue #10751 · ipfs/kubo · GitHub is the starting point to add that ability.

Next steps:

  1. Discuss whether 1024 or 174 width is preferred, or if it’s worth having both.
  2. Come up with a better naming system for these profiles (test-cid-v1 isn’t quite right since it covers a lot more than CID version, the test part doesn’t instill confidence, and it doesn’t really work in a series bc it’s not clear whether v1 refers to the cid version or a profile version)
  3. Identify someone to land ipfs add: parameter to control the width of created UnixFS DAG · Issue #10751 · ipfs/kubo · GitHub
3 Likes