Lean IPFS implementations

[bumblefudge]

There is a kind of reciprocal relationship between profiles and “lean” or “subset” implementations of IPFS-- if you only need support for one profile, you might not need all of kubo or helia, which could make all the difference in the implementability of IPFS in your use case!

@Mosh recommended breaking out this thread inside another one and I think it’s a worthwhile mini-research project to do an inventory of lean-client prior art.

@mosh

Consider more minimal implementations - Start a new thread about leaner IPFS implementations with key functions: hash, hash --profile, verify, [others].

• review prior art by community: Debian / Rust Cid Npm · GitLab
• review prior art by community: GitHub - MarshalX/python-libipld: Fast Python library to work with IPLD: DAG-CBOR, CID, CAR, multibase
• incorporate ideas from thread & elsewhere, like: If i have a cbor decode of data with a cid, I get a 42 tag, and then some hex, typically. it would be great if something like cid.ipfs.tech could more easily massage that back into a sane cid, versus the current

1. The most recent commit to the Debian package seems to not build as configured and thus fail CI – but that could be for a lot of reasons, and the last commit is over a year old.
2. One of the underlying rust crates built into this^ package is ipfs-cid, which hit v2.0.0 6 months ago, and is maintained by an unfamiliar face. This crate seems worth tracking independently from the debian package, IMHO, since many architectures require a CID-only library to run in resource-constrained corners just as a checksum/security check for CIDs and referents they get from other services.
3. MarshalX/python-libipld (features list from readme) - seems load-bearing for all of AT Proto
4. David Buchanon’s CBRR library might also qualify

[achingbrain]

@bumblefudge

you might not need all of kubo or helia

For what it’s worth, Helia is already a lot more minimal than Kubo.

The helia module doesn’t ship with UnixFS, or DAG-CBOR, or CAR files or anything.

It only has the hash functions that are provided by the runtime (e.g. the implementation is provided by node crypto or webcrypto), so these do not impact on the size of the application.

If you want the one that just does HTTP retrieval and doesn’t do libp2p or any fancy networking, use @helia/http

The idea is very much that you extend it according to your use-case which seems to align nicely with the thoughts in the OP.

[bumblefudge]

Oh, that’s an entirely valid point-- it’s awesome that helia was architected so modularly already! I guess the goal is what to tell people that need, say, Go or Rust or some other language what the equivalent is. Helia might be a good benchmark here, since it already is architected for lean subsets.