We have an android application that runs an internal IPFS node. Currently it contains hardcoded binaries of go-ips for all the architectures we want to support (arm and atom/x86).
It would be useful to have the IPFS binaries hosted on IPFS, so you can download them using curl https://ipfs.io/ipfs/QmABCD instead of having to download and unpack them from https://dist.ipfs.io.
Are the binaries already being published? If not, could this easily be added to the ipfs build process?
Ideally there should be an IPNS name pointing to a manifest with a map of version/architecture -> hash.
We would like to have a machine-readable manifest that allows you to get the go binary (not the .tar.gz archive) of ipfs for a specific version and architecture. That would allow us to just download the file when installing the apk.
Sure. That is what we will do if there is no direct link to the binary. I just thought that it would be better to directly get the binary, especially given that the binary is all that you need from the archive.
OK, I have now implemented downloading and unpacking of the tar.gz file according to the version. I hope that things like file name and location (go-ipfs/ipfs) as well as the format of the dist json are going to remain standard.
p.s. How would one layer compression into ipfs? IPFS could store blocks compressed and serve them compressed when there is an Accept-Encoding: gzip header in the GET request of the API? You would probably not want to compress before hashing, since that would mean that the same data with different compression algos would no longer have the same hash…
We’ve considered this but there are a lot of tradeoffs. For one, if we auto-compress blocks in transit, we could reveal information to an attacker. For example, let’s say an attacker can:
Monitor the amount of data sent between two peers.
Trick one peer into repeatedly downloading {"secret": "my secret data", "non-secret": "attacker controlled data"} from the other.
The attacker could use this to figure out the secret data by abusing the compression. See: