Queries being declined when pulling NFT data from IPFS

Help
1

Hi,

I am here on behalf of my dev to try and resolve an issue we are having when pulling NFT data from IPFS storage.

This is his comment:
" A small fraction of them go through but most get rejected which I’m assuming is because they’re rate limiting by IP
Keep in mind that I’m sending like 1k requests in parallel, which seems like a lot but is necessary for scraping all this data.
And arweave and others handle it fine.
Only IPFS hosted metadata is giving me trouble.

image
image2562×1442 284 KB

2

Run your own gateway/daemon. It will natively fetch the things from the IPFS network and not be rate-limited.

You can also try out the nft.storage gateway (NFT.Storage Docs) at nftstorage.link which may be less rate-limited and uses aggressive caching.

3

Thanks Hector we will give that a try