Decode dag get API

Help IPFS
1

Hello

I’m learning IPFS and trying to understand its concepts.

I created a “file” with this command

echo 'abc' | ipfs add
added Qme4u9HfFqYUhH4i34ZFBKi1ZsW7z4MYHtLxScQGndhgKE Qme4u9HfFqYUhH4i34ZFBKi1ZsW7z4MYHtLxScQGndhgKE

Then I can access it via dat get API

ipfs dag get Qme4u9HfFqYUhH4i34ZFBKi1ZsW7z4MYHtLxScQGndhgKE | jq 
{
  "Data": {
    "/": {
      "bytes": "CAISBGFiYwoYBA"
    }
  },
  "Links": []
}

This dag get API returns CAISBGFiYwoYBA, that I can decode from base64

This value decoded results in

8 2 18 4 97 98 99 10 24 4

This result includes my data: 97 98 99 10, but there is some extra data around it.

Which format is this, and how do I parse/decode it?

1 Like
2

Doing rev

8
2
18
4 <- payload size
97 98 99 10 <- payload
24
4 <- payload size
3

This is protobuf:

$ echo -n CAISBGFiYwoYBA== | base64 -d | protoc --decode_raw
1: 2
2: "abc\n"
3: 4

If you look at the unixfs spec you can fidn the schema to get the names:
https://github.com/ipfs/specs/blob/main/UNIXFS.md#data-format

$ echo -n CAISBGFiYwoYBA== | base64 -d | protoc --decode=Data unixfs.pb
Type: File
Data: "abc\n"
filesize: 4
2 Likes
4

How do you know that?
Can we add this info to the docs?

5

You’re looking at the file with two different abstractions. It would be sort of like adding a file to your file system and then viewing the file with a hex editor. Sure, it’s the same file but it’s going to look a little wonky and you might see a bunch of stuff that you didn’t otherwise.

You’re adding the file as UnixFS (ipfs add) but then viewing it as IPLD (ipfs dag get) so you’re seeing the plumbing.

I personally find some of the CLI commands a little confusing regarding files. There’s the leftover ipfs file with a single ls command. Three top level commands that deal with UnixFS files add, get, and cat. Then there’s the files group which has more to do with MFS but is related to UnixFS because that’s where the files come from. (it’s disappointing there’s no way to directly add a file to MFS). Then there’s the filestore command that is referring to files referenced on your local filestore rather than the block store which can be a little confusing because the block store is on your file store too but just as chopped up blocks. It’s a lot of different uses of the word file.

Hopefully that helps.

6

This is specs not docs domain.
The only thing specs should do is point to spec if you want to know more.

I’m already doing a lifting of the unixfs specs: Update UnixFS specification · Issue #316 · ipfs/specs · GitHub