IPFS gateway in IPFS cluster configuration

With one IPFS server node, the approach I’d use to go and retrieve content from / download a file is to

  • open port 8080
  • GET to http://<DOMAIN>:8080/ipfs/<CID>

How does that work when we’re dealing with multiple containers with IPFS cluster and IPFS server nodes as the next image shows?

(image source: IPFS Cluster with Docker. happy devOps | by λ.eranga | Rahasak Labs | Medium )

Do we open port 8080 in all of the IPFS server nodes and interact with them directly or we do it through IPFS cluster too?

You open 8080 in all IPFS nodes. Cluster does not give you a proxy to the HTTP gateway.

1 Like

Thank you hector. Does that mean one has to keep track of at least one of the IPFS nodes where a specific CID is in, or not really? (assuming not all of the nodes will have the exact same content / files)

I was going to write a simple answer but when I was thinking about my response it might be somewhat more complex.

If they’re open gateways they should retrieve the content from anywhere it’s available so it could retrieve the file from somewhere other than your cluster. That shouldn’t generally happen but I suppose it could if something was wrong with the cluster. So a Gateway on Node A could retrieve it from any of the other nodes in the cluster, or a node somewhere else.

It might get a little more complicated if you wanted to set up a restricted gateway where you only allow gateway access to files that are on your local node. I’m pretty sure you can do that but since the you want the cluster to be one logical node I’m not quite sure how you’d set that up.

1 Like

Cluster does not intervene in the gateway. The gateway will not fetch something that is available locally, as it will read it from disk.

There’s a noFetch option that allows restricting gateways only to local content.

1 Like

What I was getting at is there isn’t really a noFetch option for the cluster as a whole. If you had a cluster of say 7 nodes with a replication of 3 you’re not going to have a copy at every node so if you turn on noFetch you might get it and you might not depending on which node gateway your accessing. Otherwise you leave off noFetch and you can get the content from any node but it leaves the gateway open to the world.

I very well could be wrong there but if I am I’d be interested and if right would like to know if there are any options for doing something like that. Ie. can you setup a node with gateway that only dials nodes in your Custer and enable noFetch?

1 Like

Ah yes, you’re right. I understand what you mean now.

Perhaps with a combination of AddrFilters, Peerings etc. but I’m not sure.

Is that really necessary?

Being the goal to always get the file with noFetch on, might be worth test using the REST API of the IPFS Cluster on port 9094, namely

  • GET /pins/{cid}
  • GET /allocations/{cid}

If in any of these an IPFS server node is given in the response where the file can be found, then we could use that info to get to that gateway on port 5001 and that way always get the file.