I hope my question is not that stupid, but i have read a lot of docs and still don’t understand how does ipfs get works.
My installation:
3 devices in local network. Each device has ipfs installed. Each device already discovered each other, so there 2 other peers for each node.
In my mind IPFS works next way:
ipfs add <path_to_file> - add file to ipfs repository. Hash returned. Let it be $HASH.
ipfs pin add $HASH. Since add or pin add file started to be distributed over peers current node can reach. It means, i don’t need to start uploading process on other node manually.
Then i have ipfs refs local command to display local hashes. In my mind if i see created hash on the other node in the list of local refs, it means, file already exists in ipfs local repository. When i say exists i mean, that it was completely uploaded from other nodes.
So question 1 here.
How can i use this file if it hash already in local refs? For example, i was adding binary file and want to run it?
Is there some way to watch how many blocks | bytes already exists in my local repo?
I’m confused, because i have get cmd, which as i see, always try to download file form remote node instead of using local repo even if i have my file in local refs.
Example:
Node 1:
# create file we are going to distribute over peers
fallocate -l 499M large_file
ipfs add large_file
> added QmQacN1u9yygjG3fQn86RuufPdPqGAi7XJmoCUehwdmSRs large_file
ipfs pin add QmQacN1u9yygjG3fQn86RuufPdPqGAi7XJmoCUehwdmSRs
> pinned QmQacN1u9yygjG3fQn86RuufPdPqGAi7XJmoCUehwdmSRs recursively
Node 2:
ipfs refs local | grep QmQacN1u9yygjG3fQn86RuufPdPqGAi7XJmoCUehwdmSRs
> QmQacN1u9yygjG3fQn86RuufPdPqGAi7XJmoCUehwdmSRs # ref found
ipfs get QmQacN1u9yygjG3fQn86RuufPdPqGAi7XJmoCUehwdmSRs
> Saving file(s) to QmQacN1u9yygjG3fQn86RuufPdPqGAi7XJmoCUehwdmSRs
> 176.59 MiB / 499.00 MiB [=======>-----------------------] 35.39% 00m27s
As you can see, when i type get it starts downloading which takes about 30 seconds with my internet bandwidth.
Looks like i have found all answers to my questions.
My confusion was caused by slow ROM read/write operations. So get, actually does not download my file from Node 1 when it already in the local refs list. It just write ipfs hash content to file.
Also if you need to track current uploading progress of file there is a way:
ipfs files stat /ipfs/<hash> --with-local
> <hash>
> Size: 5242880
> CumulativeSize: 5244129
> ChildBlocks: 20
> Type: file
> Local: 5.2 MB of 5.2 MB (100.00%)
I also had an interesting result.
I’ve created 100mb file, added to ipfs and pinned on node 1.
Then i started to track local presence of hash on Node 2.
ipfs files stat /ipfs/Qmca3PNFKuZnYkiVv1FpcV1AfDUm4qCSHoYjPTBqDAsyk8 --with-local
> Qmca3PNFKuZnYkiVv1FpcV1AfDUm4qCSHoYjPTBqDAsyk8
> Size: 104857600
> CumulativeSize: 104882589
> ChildBlocks: 3
> Type: file
> Local: 91 MB of 105 MB (87.00%)
i’ve executed with cmd about 10 times, but result was always equal.
Then i’ve executed
ipfs refs local | grep Qmca3PNFKuZnYkiVv1FpcV1AfDUm4qCSHoYjPTBqDAsyk8
> Qmca3PNFKuZnYkiVv1FpcV1AfDUm4qCSHoYjPTBqDAsyk8
Which told me, that hash already exists locally.
Then i’ve executed ipfs get Qmca3PNFKuZnYkiVv1FpcV1AfDUm4qCSHoYjPTBqDAsyk8, which created my file and then i’ve executed file stat again
ipfs files stat /ipfs/Qmca3PNFKuZnYkiVv1FpcV1AfDUm4qCSHoYjPTBqDAsyk8 --with-local
> Qmca3PNFKuZnYkiVv1FpcV1AfDUm4qCSHoYjPTBqDAsyk8
> Size: 104857600
> CumulativeSize: 104882589
> ChildBlocks: 3
> Type: file
> Local: 105 MB of 105 MB (100.00%)
Now there is 100%.
Question:
why it wasn’t completely downloaded until i manually exec get cmd?
Thank you!
The file is composed of lots of chunks, not just the original CID. When you pin the CID, it will automatically (I believe, but you might need to specify recursive somehow) pin all of the chunks required by that CID. Getting the CID will also do the same, provided that your local node’s cache has enough free space to hold them all.
If you want all of the nodes, or a particular node, to have the whole file and keep having it, then you need to pin (recursively?) the CID to those/that node.
I’m actually kind of surprised that you had ANY presence on node 2 without getting or pinning the file, unless there are component chunks within that CID that were shared between multiple files and those chunks were previously accessed via node 2.
In my understanding, chunks do not move unless they are pinned or get(ed) from a node that doesn’t have them (yet). There is no “push” of chunks in IPFS. They sit where they were added until they have some other motivation to move.
1 Like