How does Files API - `ipfs files` command work

Help Old FAQ
1

From @Kubuxu on Mon Jul 04 2016 18:27:17 GMT+0000 (UTC)

IPFS Files API allows to manipulate unixfs objects (current IPFS files and directories format) in manner similar to normal Unix commands.

How does it work:

IPFS Files API introduces virtual root directory. You can check it content with ipfs files ls /.
You can copy files and directories that were added to IFPS using ipfs files cp /ipfs/Qmc4xcnns2rA4akaWfGdzdaN8My1ES6zh7yddYZ5mo7n2U /greetings.

Then you can view content of that directory using ipfs files ls /greetings and read file in it using ipfs files read /greetings/greetings.md.

You can then modify the structure with different commands: ipfs files mv /greetings/greetings.md /greetings/greetings.txt, note that will change hash of the /greetings directory. You can check new hash using ipfs files stat --hash /greetings.

If you have any more questions, please ask.


Copied from original issue: https://github.com/ipfs/faq/issues/143

2

From @Ghoughpteighbteau on Fri Jul 08 2016 21:45:59 GMT+0000 (UTC)

So hereā€™s me playing around with this feature.

~ $ ipfs files ls
~ $ ipfs files cp /ipfs/QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7 /examples
~ $ ipfs files ls -l 
examples    QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7  0
~ $ ipfs files ls -l /examples | column -t
Makefile           QmVxEtpVTAjneTSC2vZukww5KfLMnHFsBA3XQZnMHNXXDf  157
api                QmavsNYkgNcrNGgcmF6iq1SrkaaFNJVzqEnfVfdixFpPsk  0
basics             QmYEcwYnCYb9CU6ipxRJvkFmBPwgRY4fD8eLLAj3rmetyi  0
bootstrap          QmXRv4gRdokiarChpxF33jr35b6rexADEcdkwoFqKBRhzZ  0
config             QmXhaJNHgzRaWqkdnmGJSPuxc12AueXeLda8F736MqPTbd  0
data               QmTitSFQFQeBMZRDsGEvzdNgPnwx7coBCQxEYvNzgWPkK8  0
git                Qmf6cuHtrDbqoKH2L8y9MtQUmGyNksR5RoKVNVvN23BG4G  0
graphmd            QmfT4cEw9zezKaV6wF9aELFtpuovxFCR8CXSkUb6enAgw8  0
init               QmTSE278Mb1dLXtco1wBbqaUCzD17hxRKGbqhZtUbNL3Vz  0
ipns               QmUTWHFReYWNf6DX2GKy1kGXFecxQPaLTyZjVf2zKr28xu  0
network            QmRPg57F8V46vvAGjdjARPtSMYj6wBicf482MJfen3M4LT  0
pinning            QmXQNjQFBv8DxjgW1artxcXdhReFoaGRxhKqYgiMGPJ9o1  0
published-version  QmPtqzMtd7ZLBWu9sF1xSUDfRZ6kkVnactBMKHdsMC1ZtJ  188
snapshots          QmYuo1vhfDNTAmTPakchttvamQj6aLFjv6J5oeUJZhC99M  0
videos             QmaJ8xTbRqTFBKEfePvL5tS3iAZJUMRWtUX9k8Pm9Sh8pT  0
websites           QmZdWW55p6YTkFnGZHUmdVZjVKaBA745FENzftoYdapU9g  0
~ $ ipfs files cp /ipfs/QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7 /test
~ $ ipfs files ls -l 
examples    QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7  0
test    QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7  0
~ $ # gaaah, no columns....
~ $ ipfs files ls -l | column -t
examples  QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7  0
test      QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7  0
~ $ ipfs files rm -r /test
~ $ ipfs files mkdir /test
~ $ ipfs files cp /ipfs/QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7 /test/otherExamples
~ $ ipfs files ls -l | column -t
examples  QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7  0
test      QmdQ3wVuxPPSjbihiDKndwYBFQDtGWznEdmnjLEpdEphas  0
~ $ ipfs files ls -l /test | column -t
otherExamples  QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7  0
~ $ ipfs files stat /
QmTffJW1tzPwikLeCAFw7t4DbZidVGwVM1oMs96PTy25tk
Size: 0
CumulativeSize: 794139
ChildBlocks: 2
Type: directory
~ $ ipfs files cp ./pic/dukesOfNyan.gif /test/dukesOfNyan.gif
Error: Paths must start with a leading slash.
~ $ # I expected that to work...
~ $ ipfs add ./pic/dukesOfNyan.gif 
added QmeFBEVocqUpuob62umabRuSXERGFwF6TVK3ZqBAdv3VjL dukesOfNyan.gif
~ $ ipfs files cp /ipfs/QmeFBEVocqUpuob62umabRuSXERGFwF6TVK3ZqBAdv3VjL /test/dukesOfNyan.gif
~ $ ipfs files ls -l /test | column -t
dukesOfNyan.gif  QmeFBEVocqUpuob62umabRuSXERGFwF6TVK3ZqBAdv3VjL  19411092
otherExamples    QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7  0
~ $ ipfs files ls -l / | column -t
examples  QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7  0
test      QmX8NbuNEEqqxDNU1GuXyQST83EpqRt3gQweZeAGzpJFGc  0

So a few questions. Is this meant as a convenient editor for adding IPFS directory objects?

Does this have an advantage over just editing a directory in the local file system and adding that?

I see root has a hash as well. If someone requests that hash, will I give it to them?

When I changed /test from being QmdQ3wVuxPPSjbihiDKndwYBFQDtGWznEdmnjLEpdEphas to QmX8NbuNEEqqxDNU1GuXyQST83EpqRt3gQweZeAGzpJFGc I noticed that the old versions of /test was still stored

~ $ ipfs object get /ipfs/QmdQ3wVuxPPSjbihiDKndwYBFQDtGWznEdmnjLEpdEphas
{"Links":[{"Name":"otherExamples","Hash":"QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7","Size":396987}],"Data":"\u0008\u0001"}
~ $ ipfs object get /ipfs/QmX8NbuNEEqqxDNU1GuXyQST83EpqRt3gQweZeAGzpJFGc
{"Links":[{"Name":"dukesOfNyan.gif","Hash":"QmeFBEVocqUpuob62umabRuSXERGFwF6TVK3ZqBAdv3VjL","Size":19415747},{"Name":"otherExamples","Hash":"QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7","Size":396987}],"Data":"\u0008\u0001"}

So each modification done in ipfs files is added to ipfs. Is that what the ipfs files flush command is for? To avoid constantly adding changes to ipfs?

I notice that entries into the files API are not pinned, should they be?

Finally Iā€™m trying to get an overall sense of what this command is intended for long term. This kinda seems like a porcelain command, more than a plumbing. Long term, are we all going to have an IPFS ā€˜driveā€™, that our file browsers understand, where we organize the contents of IPFS?

3

From @whyrusleeping on Fri Jul 08 2016 22:05:03 GMT+0000 (UTC)

> Is this meant as a convenient editor for adding IPFS directory objects?

Yes, among other things

Does this have an advantage over just editing a directory in the local file system and adding that?

depends on what your usecase is, ipfs add actually uses this code under the hood, so performance should be identical

So each modification done in ipfs files is added to ipfs. Is that what the ipfs files flush command is for?

Each modification is stored unless you use the --flush=false option. If you set flush to false, it will not write intermediate objects out until you run a command on that branch with flush true, or you run ipfs files flush on the directory/object in question

I notice that entries into the files API are not pinned, should they be?

This one is tricky, and something we havent quite finished. Stuff in the files API are not currently pinned, and they should not be pinned recursively. The reason for this is because one use case we want to explicitly support is ipfs files cp /ipfs/Qm20TBDataSet /somewhere, and be able to operate on that dataset without pulling it entirely local. Towards this, we are going to implement a new pin type called ā€˜best effortā€™ that will keep what you have local, local. Until then, be wary running a gc if youre playing with the files api.

Long term, are we all going to have an IPFS ā€˜driveā€™, that our file browsers understand, where we organize the contents of IPFS?

Thats one idea :slight_smile: And one i very much support

4

Hi everyone,
Iā€™m still cannot get the purpose of files API. I try to read spec and searching around, but I couldnā€™t have any idea to persuade me. So I have several questions about files API and hope someone can help me clear my confusion.

IPFS Files API introduces virtual root directory. You can check it content with ipfs files ls /.
You can copy files and directories that were added to IFPS using ipfs files cp /ipfs/Qmc4xcnns2rA4akaWfGdzdaN8My1ES6zh7yddYZ5mo7n2U /greetings.

  1. Does it mean it creates a symlink from IPFS to our local root directory or it just copy data from IPFS to our local directory? If itā€™s a symlink, how can I find corresponding file in IPFS?
  2. When we call some function like :
    ipfs files mkdir
    ipfs files write
    what will happen ? Does it affect IPFS storage ?
  3. Why we need a virtual directory instead of storing in a real directory ? As I understand, IPFS client can be browser, so I think people might want to create virtual directory to manage generated data. Does it only a reason or am I missing something?
  4. And final question, I can read file though API (https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesread), so basicly does anyone can access to it or they have my private key? Reason I ask this question because, if I put some file in to ā€˜/somepath/myfileā€™, if any one can access file through that path, it would be nicer than very long meaningless hash name and I can update file content without using IPNS, which is very slow to update?

Thanks for your clarification.

5

Hey @tuan3w, the idea of the files API (you might also see it referred to as ā€œMFSā€ or Mutable FileSystem) is generally to make it easier to edit and organize files in IPFS. Since nothing can be changed after itā€™s added to IPFS (or: changing a file/directory changeā€™s its hash, which basically means youā€™re making a new file), doing something that might simple on your hard drive, like renaming a file, can be pretty complicated in IPFS.

For example, if you had a directory structure like:

/documents
    /ipfs-introduction
        - overview.txt
        - use-cases.txt

And you renamed overview.txt to introduction.txt, youā€™d have to:

  1. Make a copy of the ipfs-introduction directory and change the name of the file in it.
  2. Add the new directory to IPFS.
  3. Make a copy of the documents directory and change the hash for the ipfs-introduction directory in it to point to the new one.
  4. Add that new directory to IPFS.
  5. Get the new hash for documents so you can share it.

To make that easier, the files API gives you some basic Unix-style commands (cp, ls, mkdir, mv, rm) to manipulate files. When you use them, IPFS takes care of managing all the different hashes and links that have to change. So, to do the above operation, you can just:

  1. Move the file:

    > ipfs files mv /documents/ipfs-introduction/overview.txt /documents/ipfs-introduction/introduction.txt

  2. Get the hash:

    > ipfs files stat /documents

Does that make sense? Hopefully that helps clarify some of your original questions:

ipfs files cp doesnā€™t make a copy; it just adds a pointer to the hash of the file you are copying. So in your example, you added a pointer named greetings in the IPFS object that represents the root directory whose value is /ipfs/Qmc4xcnns2rA4akaWfGdzdaN8My1ES6zh7yddYZ5mo7n2U.

When we call some function like ipfs files mkdir and ipfs files write

On the other hand, ipfs files write adds the data to IPFS. Itā€™s kind of like ipfs add [filename], except it doesnā€™t just add it to the IPFS network with a hash ā€” it also puts it in the MFS root directory with a nice name. After that, youā€™ll have:

  • The original file on your computerā€™s filesystem
  • A copy of the file in IPFS
  • Two pointers to the file in IPFS: one as just a regular hash and one from MFS

does anyone can access to it or they have my private key?

Right now, the only way to access MFS content is through the files API. That means someone has to have access to your IPFS nodeā€™s API in order to read. The API is normally private (only your computer can access it). You can change that, but I donā€™t recommend it.

6

Oh, thank @Mr0grog for your clarification. Itā€™s very insteresting.
Before this, in order to modify structure of a folder, I have to use ipfs object patch and store hash somewhere for retrieval purpose. Now, using files API can make everything much easier :smiley: .