Since about a week ago, I took some old hard drives and connected it to a raspberry Pi and ran the latest version of go-IPFS. I wanted to pin files, and later probably serve websites from it.
But a week later, when ever I serve the net on my home network. I would get 429 server responses whenever I visit IPFS sites. For example on the nifty ink website, it will be blank, and in the console there will be 429 errors trying to connect to https://ipfs.io.
The return error is : 429 Too Many Requests: openresty
I’m not sure what is going on… And since the owner of the websites, like nifty.ink probably hard coded the IPFS gateway inside the site, there is no way for me to change it to point to my local node. And hence some IPFS sites are now really broken for me.
You should think of the “ipfs.io” gateway as a reference/demo instance for doing small tests for people getting started. I don’t think they’re gonna just host large amounts of data or traffic on there for the whole world. That’s not what it’s for. If you want to do a large number of transactions, or something commercial, you probably will need your own gateway.
Probably what’s happening is that since IPFS is distributed and worldwide it has to have some peers to seed with, and connect to when it starts up. I did a quick search and I think maybe that’s called the “bootstrap list”?
So you can maybe either set it to your own custom list or remove ipfs.io from the default list if it’s there and your machine can’t reach it for some reason. Also remember if you’re lost just download the source as a zip file, extract, and search for “ipfs.io” and you should be able to see where it’s getting it from.
you provide conflicting statements on whether this affects multiple sites or just Nifty Ink. Assuming the latter, I think using IPFS Companion should correctly redirect requests to your local node. It would seem to me that they are running their own gateways though, now.
ipfs.io will rate limit requests for popular content so that they do not overwhelm the service.
Hi thanks all the kind devs for the replies, here are my thoughts.
The Owner of nifty.ink has hard coded ipfs.io as the ipfs gateway by default. After visiting that site, I realised my IP address got blocked by the gateway. And for the rest of the day, visiting some other sites that run on IPFS breaks as well. After a while that resets itself.
We are going beyond the stage of tinkering for the decentralised web, and we should be thinking UI/UX issues for the mass public right now. The Masses are coming.
And no, we do not expect the masses to download another browser extension to try to fix a console bug.
Since this is a decentralised service, why not ipfs.io just route requests to other decentralised nodes instead of banning IPs??? Yes i know it is still a single point of failure. But how does the tor network works? Maybe we can borrow something from that.
By the way, i tried IPFS companion, and it gives me HTTPS error. It redirected to HTTPS site, while my own gateway is running on HTTP.
all in all, IPFS is a great software, but with all these UI/UX issues, mass adoption will be very painful.
And lastly, if you don’t want devs to actually use that gateway, just write it so on the documentations in big red letters: ONLY FOR TESTING, DO NOT USE IN PRODUCTION.
I don’t disagree with the general feeling, but the world is complex:
It would seem this isn’t an ipfs-hosted site in the first place, no dnslink record and just leeches of the public gateways as if they were a free hosting service. The gateways are a production service and have very reasonable limits for what you’d expect. If opening the website triggers dozens of requests to the gateways for every user it is normal that certain content or IPs become rate-limited.
I don’t disagree, but this is the reason gateways have limits. The masses come and abuse your service in all ways possible. Without limits it is impossible to provide a reliable service in the first place.
Brave browser includes ipfs by default. Opera too (I think). Chrome and Firefox added the possibility of handling ipfs:// and js-ipfs can run inside the browser. This is not enough, but I can tell you that you just cannot knock on Mozilla’s door and say “please put native IPFS into Firefox”. Adoption at that level is very slow, way slower than I’d like, but it’s what we have to deal with.
Because that would make the service suck in average way more that it sucks for you individually. We cannot be responsible to how others run infrastructure like this. Because gateways are gateways at the end of the day there is a component of trust involved too. You can however use IPFS directly and natively and/or generally use any gateway you want with the browser extension.
Check your settings, I am not sure how you configured your gateway, but the urls in the settings specify http/https individually.
working on it!
We want them to use them, but beyond certain limits they should setup their infrastructure because it is not a free unlimited hosting service as much as we’d like it to be.
Apparently my own IP has been rate limited by all the IPFS public gateways since running a node…
I’m sorry but this answer nothing.
Why ?
I’m running 3 nodes on my home network, (which is more than one and thus should eat any rate limit faster), and I’m not rate limited by anyone.
I would be interested to hear why you think that your IP is under rate limiting ? (Any experiment while running the IPFS node and while not ?)
Secondly, I’m pretty sure the rate limiting of ipfs.io doesn’t even look at if you host an IPFS node or not, because most out of the box solutions either work on the webserver and then are capable to do things such as returning 429 but thoses only care about the webserver and don’t know about the outside world or in the firewall / ip routing layer and then you would receive a Connection refused (or other) ICMP packet or just no reply at all (the server acting like the port is closed) and thoses are capable to rate limit across multiple ports.
To me it just seems whatever app you are using use ipfs.io as it’s exclusive gateway to download too much data and this is unrelated to you running a node or not.