Hello, I am a newbie in IPFS world and have a basic question on it. Is it possible to use this docker-compose file to deploy the IPFS Cluster to Microsoft Azure Container Apps? If so, can you please guide me through the steps or share some info? Thanks.
I wouldn’t try to do that but that is my personal opinion. That would be trying to containerize IPFS nodes ->cluster in Azure object storage
Probably better to not use Azure. But if you must then to use IPFS Daemon rather than Cluster. If you use Cluster then you should use Ansible to configure which is why Azure is problematic. Now you have to use WinRM to configure MS infrastructure. I run an on-premise IPFS cluster that is a private network made up of a handful of machines (desktops and Androids) these nodes are bootstrapped together and only communicate with each other and no new nodes can be added maintaining privacy and security. Using Ansible this can be done following an inventory of network assets. The respective nodes are then in the yaml file and that eliminates uncertainty.
I had to send your docker-compose file to one of my workstations to read it as I’m on my phone atm. Did you write it yourself or are you using someone else’s file? I just ask that because it has a template feel to it and you said you are a newbie so I make the assumption that you got the file from someone else. Anyway I don’t see why you would need IPFS Cluster AND IPFS daemon. There is a hierarchy but the two can do the same job. I looked at how Protocol labs implemented IPFS on their own infrastructure. They have a collaborative cluster which part of it is on the same machine. That machine has multiple nodes running on it as either containers or VMs that machine uses IPFS Daemon to manage them and present a singularity to IPFS cluster. IPFS cluster is needed to orchestrate pin sets across discrete remote nodes. It uses libp2p extensively which makes it so powerful but that must be used properly. That’s why I question the efficacy of using it in an Azure ecosystem as they are quite different and I’m sure it will work most of the time (maybe?) it will not scale without considerable pains. I would recommend a different hypervisor and public cloud infrastructure. I like to use Proxmox VE as hypervisor for my homelab/studio it is very efficient and free. It plays well with most of the major Linux distros as well as VMware and Hyper-V images. A close alternate to Proxmox is XCP-NG it is a fork of Citrix Xen Server. Canonical (Ubuntu) uses CloudInit/ MaaS and JuJu to manage Kubernetes. Proxmox and XCP-NG can use OVN and eBPF tools like Cilium and Prometheus to monitor kubernetes clusters and docker swarms. That is why I think Azure is poor choice as it lacks that kind of granularity and compartmentalization. I think a lot of MS products start out easy to draw users to them. But then when you scale up it requires major expenditure to make sure it works as intended. Typically they try to lock you in to “Enterprise level solutions” meaning $$$ and more complexity that would necessitate dedicated staff or “Managed Service Provider”. And all you wanted to do is now this ginormous thing. Don’t let them fool you. I nearly went down that path when I received a Dell poweredge server that had MS server 2016 on it. I started down the Azure path and next thing I’m making AD DC and so forth
1 Like
Thanks for the reply. I will look into the alternatives you enlisted in your comments. Back to your question on the YAML file, I got it from here: https://raw.githubusercontent.com/ipfs/ipfs-cluster/master/docker-compose.yml
Ok I’m glad you aren’t locked in to Azure. That nearly happened to me.
Frankly I prefer to avoid setting up an IPFS Cluster and just go ahead with bootstrapping IPFS nodes together. The only issue is that how I can provide pinning capability without the cluster?!
Cluster is for orchestrating. Pinning should be easy as adding a file at the IPNS gateway. There it is committed to IPFS getting a CID and broken into blocks. A good way to think of it is like a private torrent That torrent file (not what it is made of) or the magnet file are like pins. When something is pinned it shows intent to store in IPFS by someone. This is why it becomes immutable and can’t be Garbage Collected. Pinning files are just a way to communicate intent to store
Awesome, thanks for clarifying the path. I will give a try to an infrastructure without a cluster.
I’m glad to have been some help 
1 Like
Indeed it was a great help
You can deploy your private nodes bootstrapped into each other. Use some firewall/iptables rules to block public access for all of them. Then open only 1 node to public internet by bootstrapping that into public network and allowing specific firewall/iptables rules.
Is this that what you are looking for?
What I have failed so far is to run a docker-compose push to Azure Container App. All images i.e. ipfs0, ipfs1, …, cluster0 are skipping when running that command. The problem that I have is more fundamental than configuring the firewall rules. At this moment, all I want is to see if I can host everything on Azure Container Apps which was inconclusive so far.
I haven’t seen any official documentation regarding clusters at containers. And I don’t think there is any out of the box tool available for that.
You might need to manage individual container as it is single ipfs node and then cluster it manually.
Or, deploy some custom ansible/terraform scripts (which may be very complex one) to manage those containers as cluster.
That’s by far I think helpful for you.
1 Like
Ok my bad that I haven’t been keeping up on this topic since last time. I think I went over the documentation for IPFS cluster and found that it does lead people down some wrong paths. I don’t know why it pushes docker as that is a fundamental error. I followed the path on an Ubuntu workstation and it specifically warned me to not install by snapcraft (snaps) and stated that using deb was experimental but approved solution. Snaps install to their own chroot (sandbox) containment outside of the root file system. Because of that it clashes with IPFS because it isn’t root and abstracted. I noticed that in Ubuntu it guides user to install docker as a snap as well. Therefore my read on this is that abstraction or federation of an IPFS node is disallowed and throws errors. So If it has been working as a docker image then that’s lucky for you. You probably have it (docker) installed directly to your OS so it has or was installed with root privilege. But when it goes to Azure it is definitely federated and now nested in a further abstraction (Azure container app) no doubt it will continually fail.
Also I think there are some semantic conflict as far as “cluster” is defined. Best rule is to use UNIX/BSD conventions. IPFS Cluster is for a cluster of nodes with the expectation that they aren’t on the same machine and are physically distributed. Nodes are machines and if they are virtualized on the same machine (server/workstation) then IPFS Daemon(Service on Windows Server/Workstation) is the best use.
@Galactus is correct that what is needed is Ansible, Chef or Terraform/Consul/Nomad to make it happen in Azure. But it isn’t complex or difficult unless one makes it so. Hector Sanjuan is the curator for IPFS cluster and I see and heard he uses Ansible. If you look up IPFS Cluster in GitHub he has a simple Ansible file as an example. Basically it handles bootstrapping your nodes that are defined in the file. I think I’ve used Terraform->Consul->Nomad to deploy a cluster of nodes but not IPFS cluster. I am unsure how well they would work together as they use “raft” rather than CRDT consensus.
I think this should give some clarity to the foundation and explain errors.
Keep me apprised of your progress I would like to see you have some success. 
Thank you for your continuous support and care. I will carry on exploring this route and keep you posted as I progress. As a side note, I was able to run an IPFS node on Azure Container Instance with mounted storage (FileShare by Azure Storage Account). I composed a set of bicep files to provision this infrastructure. I think if we spawn multiple nodes of this kind, and create a swarm of them, then they can support a simple private swarm. Do you think this can serve a private production environment? Having said that, that’s not my ideal way to go on, and having a real IPFS Cluster is the ultimate goal. I will give Ansible a try once I have a chance after this long weekend.
1 Like
I’m unfamiliar with “bicep files”. 
. Sounds cool though. I initially thought it was like “flex time” 
.
I think that a docker swarm is something I want to try out on Azure but I don’t think I would use IPFS nodes. There was something that was discovered from the dismantling and dissection of the “Avalanche” criminal network in 2014. They used a very resilient or persistent “fast flux” network that would generate new domains programmatically that seemed plausible Human generated. There is a project called “cloud box “ https://cloudbox.works that has some DNA from Avalanche. I think you will like it.
I used the ACI and bicep as a POC to create an IPFS node. As I stated before, a true IPFS cluster is a safe and reliable goal down the road. I deleted the Azure resources to prevent unwanted CPU and storage charges. Ansible is something that sounds promising and I need to spend some time first learning it and then trying to leverage it. I wish there was a working full-blown Kubernetes solution too.
1 Like