Given a CID, I want to quickly check if it fully exists locally. In the JS implementation, the blockstore in
js-ipfs-repo has a method to check if the CID exist in the store. I’m looking for a similar solution ideally using the
The closest thing I can come up with is using
files.stat with the
withLocal option but I assume that this will attempt to retrieve the blocks from the network if they are not available locally.
After digging a bit further - doesn’t seem like
files.stat would work, even if the
withLocal option was supported in
js-http-client. I’m now thinking that going through the local
refs might be my best bet.
I think you’ve figured it out
ipfs.refs.local will give you a list of all CIDs with blocks in your repo.
Is there a way to check a single block only @achingbrain ?
That would be incredibly useful. In my use case, the nodes may have upwards of 4-10TBs locally and for various reasons (mainly performance), it helps to distinguish local CIDs from remote CIDs to more efficiently decide what operations to perform and in what order.
I’m going to test out pulling the local refs through
ipfs.refs.local later today — I’m guessing it’ll work fine but might become an issue as the list grows.
To clarify, will a CID show up in
ipfs.refs.local if any or all of its blocks exist locally?
That will list ALL local blocks.
I believe it is a reasonable feature request to extend
ipfs refs local [-r] [CID] to take optional CID argument and print it if it is in local repo. If
-r is passed, print all local blocks that belong to the root. This would match existing behavior of
ipfs refs and
ipfs refs -r. Together, they would enable people to calculate how much of data behind CID is stored locally.
Use case: IPFS Companion is bundling webui and importing it to datastore if it is not in it already. To do the check, it needs to list all local CIDs and if there is no match, perform import. Extremely not efficient check if user has a lot of blocks.
Where should I fill issue for this?
Core API is at https://github.com/ipfs/js-ipfs/tree/master/docs/core-api#ipfs-core-api – does it mean js-ipfs repo?
But go-ipfs already supports this with the
That is a good point. We probably just need to improve UX:
I was not aware it is even an option.
--offline is a global flag (not listed in
ipfs refs --help)
$ ipfs refs -r --offline bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq
Error: failed to fetch all nodes
Perhaps we could implement
ipfs refs local [-r] as porcelain on top of
ipfs refs [-r] --offline ?
Digression: I still think we need a dedicated API for this check.
--offline trick is not a silver bullet: it works only in go-ipfs CLI.
AFAIK it is impossible to use it for checking presence in local repo in JS:
it is missing from Core API and is not supported by js-ipfs-http-client.