Content Retrieval is too slow

Ecosystem
1

I worked a lot with IPFS and tried out many of its features. I want to say that I am still in love with this network, because of its simplicity and its usability in so many contexts. However, it is sometimes an unhappy love! My main problem with the pride is content resolution. Any content that I add to a specific node, just takes way too long to be available “on the other end” of the public network.

I understand that this is a very hard problem to solve, even for the brightest developers. And I am sure there are very smart people working on this.

Please give me some good reasons for more optimism! Will it ever be solved/improved? How will it be solved? — Make IPFS quicker, please !!!

3 Likes
2

Which usecase do you have? Just two nodes you own? Then you can connect them together to speed this up.

You know that you can start your daemon with --enable-pubsub-experiment and use IPNS names, to speed the resolution up as well? :slight_smile:

3

I would like to use public ipfs gateways to serve my single page applications. So if I add the page to my node, there is too much delay until it is found by the gateways. So if my node is up and running, it would be absolutely fantastic if any public gateway could serve my page in no time.

4

Which public gateway do you use?

5

I think @lyrx is referring to the official ipfs gateway.

6

I’m not sure how the status of the gateway is considered in terms of performance. But at my location the Cloudflare gateway is better in terms of performance.

@lyrx do you use ipns with pubsub?

Edit: Just checked with a freshly generated CID, the Cloudflare gateway loads the page within 2 seconds.

7

I am not thinking about specific gateways. My usecase is cloudflare as a gateway and pinata for pinning – and my own node for adding. Anyway, if we had k nodes for pinning [P_1 … P_k] and m gateways [G_1 … G_m] If you randomly choose a gateway and a pinning node, than it should work even if all the other nodes were down. That would be real censorship resistance. m * k combinations would have to work for that, and that would be fantastic.

Also, if I used pubsub, maybe that would help me, I do not known enough about it. But if I would just like to be part of the swarm, then it would only help me if everybody had it. Look at a swarm of birds in the sky: There is no leader, but the information which direction to take propagates almost instantanously from one bird to the whole swarm. That would be exciting: Be a node and add any information, and in a minute it is available everywhere …

1 Like
8

Pubsub is basically direct communication, instead of talking thru the DHT, which is much faster.

You asked how you get the gateway access faster: That’s how.

And birds can’t propagate an information world wide in real time either :wink:

9

Maybe this can help:

Maybe @obo20 can help you out more on this :slight_smile:

UPDATE: As you don’t own the nodes at Pinata & Cloudflare, you can setup a node yourself and connect it to the nodes at pinata & Cloudflare. This would ensure atleast 2nd degree connection, if not 1st.

10

Hi, you give me valuable information, thanks! Sorry to ask for even more: I think I would need the ipfs multiaddresses for the cloudflare/ipfs gateways, and then I could connect my own node with them.

For the pubsub thingy: How to use pubsub and ipns to speed up content publication to gateways: Is there any doc out there that could be linked related to this?

EDIT: I found this: https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#ipns-pubsub

2 Likes
11

I think I would need the ipfs multiaddresses for the cloudflare/ipfs gateways, and then I could connect my own node with them.

You may need to contact them. I’m not sure if they(cloudflare guys) will be in for this.

12

@lyrx Let us know if you are still stuck.

13

You don’t have to excuse. This board is meant to be used for discussions and exchange of knowledge.

Feel free to extend the currently available information on the docs if you think they are not clear enough, or there are gaps.

This webpage is available via CloudFlares gateway and stored on three nodes by the author on IPFS.

It’s pretty fast.

The author explains in a linked howto how he set up this webpage.

Best regards

Ruben

1 Like
14

Very well! For me this P2P stuff is not only about technology, it is also social – social learning. I paid some money to have my own IPFS node on a hosting platform, and I even started my own private dedicated IPFS network, because I thought it could be a good idea. It wasn’t! I learned that P2P is about sharing and participating. That is why I am back in the public IPFS network, and I want to have my node on each of my hardware pieces, and maybe have other people who share my data on their nodes. There are still problems: NAT, latency and others, but they seem to be on the decline, these problems! I think that cloudflare does a good job with their IPFS integration, and it might be a beginning only …

I already took over one charity site from some people, and I think I can save them mire than a 100 dollars a year for hosting, because I have put it on IPFS+cloudflare. Underprivileged might soon be able to share their content, because it is much cheaper to do so (less hosting costs), and it might be censorship resistent !!

4 Likes
15

Rest assure there are a lot of efforts being put into making content resolution great again, with many things in progress at the moment, and many others in testing. These should start landing in a few weeks.

Regarding gateways, here are a few you can connect to (but they will rather aggressively prune your connections from their side):

           /ip6/2604:1380:1:b600::3/tcp/4001/ipfs/QmVjLyXj8NtBUt1XoEWLmwwb9NPZJrRUsjppi8a3uw29CZ
                /ip6/2604:1380:4020:1400::3/tcp/4001/ipfs/QmVNb9aEcdFgxGBLCzs9oJziSxbodt6sZMRcoXYHkyzfQx
                /ip6/2604:1380:2000:f700::3/tcp/4001/ipfs/QmXAdzKCUZTg9SLLRK6nrfyhJmK1S43n2y5EBeGQ7zqhfW
                /ip6/2604:1380:40b0:c00::1/tcp/4001/ipfs/QmbUa4f2y1yqAwFBPS61A818xFoqVAPbbJPXti5wjV3Tdd
                /ip6/2604:1380:40c0:e00::3/tcp/4001/ipfs/QmYZmw1PQXtKVeAjRfn8TnKYAurSdezRd1kdDjenzxTF6y
                /ip6/2604:1380:1:b600::1/tcp/4001/ipfs/QmXUEWEWWq6dhKqBwryPSR8jYoa9ByeAaqnz62CRDY7ppL
                /ip6/2604:1380:3000:6f00::3/tcp/4001/ipfs/QmQ41LAQUdH6JQCeSR1qMT8qdPAkZ9XgkvqYHs9HiAdh5Z
                /ip6/2604:1380:1000:ea00::3/tcp/4001/ipfs/QmXYHU1M8t86gRKvUmE42smybPKXqTPEUPDUaCVtLmS6Bw
                /ip6/2604:1380:2000:f700::1/tcp/4001/ipfs/QmRzfM4kLwk6AjnZVmFra9RdMguWyjU4j8tctNz6dzxjxc
                /ip6/2604:1380:4020:1400::1/tcp/4001/ipfs/QmaXN7mAgF82xLwmARPJ3snW3Nni9jb7aAn3BNdCyiw63w
                /ip6/2604:1380:40b0:c00::3/tcp/4001/ipfs/QmYAFesVS5hJSBmGBkHLWG7ou3cLrfBL3nisqFmdpgyYdN
                /ip6/2604:1380:40c0:e00::1/tcp/4001/ipfs/QmSPz3WfZ1xCq6PCFQj3xFHAPBRUudbogcDPSMtwkQzxGC
                /ip6/2604:1380:4080:600::3/tcp/4001/ipfs/QmS4YDJRE5CSST84BowT1WxKRVxGnW7YTrJxMk7bcZZbfV
                /ip6/2604:1380:3000:6f00::1/tcp/4001/ipfs/QmXygeCqKjpcbT9E7jgSbAvcDnPaJp9gQDzEVFZ6zzjVc2
                /ip6/2604:1380:1000:ea00::1/tcp/4001/ipfs/QmVLb8bi8oAmLwrTdjGurtVGN7FaJvhXP5UM3s56uyqfxL
                /ip6/2604:1380:4080:600::1/tcp/4001/ipfs/QmQzT1S3mGoTZFTzCXp67xnxxKPppYfpYDAxLMj4Zsp6kU
16

Did you try a pinning service?

17

I use pinata as a pinning service, and it works very well.

I also tried out the list hector sent to me. Thanks! – You know, there are a lot of aspects in this, and I am not a real expert. But I just feel that this can really become great if nothing depends on a single node (or on a list of nodes). So for example today I have trouble with the cloudflare IPFS gateway that is inside my DNS configuration. So my websites do depend on this specific gateway, and today this gateway is just having a bad day. But my content is on IPFS, so the whole configuration should be more resilient, and it isnt. There should be no reason to configure specific IPFS nodes anywhere.