IPFS binaries on IPFS?

We have an android application that runs an internal IPFS node. Currently it contains hardcoded binaries of go-ips for all the architectures we want to support (arm and atom/x86).

It would be useful to have the IPFS binaries hosted on IPFS, so you can download them using curl https://ipfs.io/ipfs/QmABCD instead of having to download and unpack them from https://dist.ipfs.io.

Are the binaries already being published? If not, could this easily be added to the ipfs build process?

Ideally there should be an IPNS name pointing to a manifest with a map of version/architecture -> hash.

Dists page is already hosted on IPFS: https://ipfs.io/ipns/dist.ipfs.io, and there is a version file for each package https://ipfs.io/ipns/dist.ipfs.io/go-ipfs/versions

1 Like

Cool. That’s almost what we’re looking for.

We would like to have a machine-readable manifest that allows you to get the go binary (not the .tar.gz archive) of ipfs for a specific version and architecture. That would allow us to just download the file when installing the apk.

Kind of how it is done in https://github.com/ligi/IPFSDroid/blob/master/app/src/main/java/org/ligi/ipfsdroid/IPFSDaemon.kt#L22 , but with an officially supported mechanism.

That would allow us to just download the file when installing the apk.

Is downloading the archive and extracting it not an option? You can:

  1. Get the latest version from the end of https://ipfs.io/ipns/dist.ipfs.io/go-ipfs/versions.
  2. Figure out the right dist archive for your current architecture/os by reading https://ipfs.io/ipns/dist.ipfs.io/go-ipfs/$version/dist.json
  3. Download the archive.
  4. Extact the ipfs binary from the archive (there are java based tar.gz extractors).
1 Like

Sure. That is what we will do if there is no direct link to the binary. I just thought that it would be better to directly get the binary, especially given that the binary is all that you need from the archive.

  1. The archive offers significant compression (42M versus 12M).
  2. We need the license, readme, and install script.
1 Like

That is actually a good point.

OK, I have now implemented downloading and unpacking of the tar.gz file according to the version. I hope that things like file name and location (go-ipfs/ipfs) as well as the format of the dist json are going to remain standard.

p.s. How would one layer compression into ipfs? IPFS could store blocks compressed and serve them compressed when there is an Accept-Encoding: gzip header in the GET request of the API? You would probably not want to compress before hashing, since that would mean that the same data with different compression algos would no longer have the same hash…

We have no plans on changing that layout.

We’ve considered this but there are a lot of tradeoffs. For one, if we auto-compress blocks in transit, we could reveal information to an attacker. For example, let’s say an attacker can:

  1. Monitor the amount of data sent between two peers.
  2. Trick one peer into repeatedly downloading {"secret": "my secret data", "non-secret": "attacker controlled data"} from the other.

The attacker could use this to figure out the secret data by abusing the compression. See: