Trying to understand where the storage space is coming from for all the existing ipfs data . . . is it just a bunch of hobbyists setting up some servers in their basements? People with ipfs nodes on their laptops that pop into and out of existence as they power up and down? Who is providing the disk space for the photos I upload via fleek (just as an example)?
Anyone have some ideas about what the landscape looks like?
Itās a common misconception that IPFS has some kind of magical storage space.
The fact is that you have to run a node for storage, it doesnāt have to be you but someone has to. If you use someone else storage they might give you free space but they donāt have to and most donāt.
For maximum decentralisation everyone would need to run a node.
Itās when the same data on IPFS is shared by many people that the magic happen.
Thanks for confirming my suspicions. With every announcement of a new IPFS application, I always wonder whoās going to be hosting all those photos/web sites/whatevers?
Does Protocol Labs (or someone else) publish any statistics on IPFS storage? One thing that the StorJ folks do well is publishing their storage node stats/details. Is there anywhere to get a picture of how many nodes there are and how much storage is out there? I guess Filecoin integration will help with getting more and persistent storage on line. Is there a timetable for that?
With every announcement of a new IPFS application, I always wonder whoās going to be hosting all those photos/web sites/whatevers?
In theory you, in a perfect world everyone runs their own IPFS node and you host your own content.
Sadly we arenāt in that world yet, so most apps use services like web3.storage, nft.storage, estuary.tech or other paid services like pinata.cloud, fleek.co or infura.io.
Secondly, why do you ask this question ?
Itās not that itās a bad one or anything but many web2 services already found ways to make money hosting other people files for free, why couldnāt we just apply the same thing to web3.
Thanks for your insights. Iām only asking because I was just curious and didnāt understand how it was working. Like I said, I see many new apps/services in the weekly announcements, and I couldnāt figure out where it was all being stored.
Three are lots lots and lots of abandoned apps/services as well, and dead-end ipfs hosted web pages, so maybe thereās not as much out there as I thought!
Iām only asking because I was just curious and didnāt understand how it was working.
Yeah, Iāve shouldnāt have phrased things like that, what I meant is that web2 makes it work so we can too.
Three are lots lots and lots of abandoned apps/services as well, and dead-end ipfs hosted web pages, so maybe thereās not as much out there as I thought!
This is a valid concern for not IPFS. But on IPFS you can self host your files, so if you care for your files just self host them yourself (or pay someone with filecoin or a pinning service). 
Thatās my next area for a bit of research - Filecoin and how that works with IPFS . . . . if you have any sources of info that youāve found helpful, Iād appreciate a pointer!
Iām continually looking for better ways to explain IPFS so hereās my latest attempt. You can think of IPFS as a system that makes it very easy to ask but not tell. You can stand in the middle of a crowd and shout, āHey does anyone have X that they can give me?ā and maybe someone will throw you an X. If you hear someone else asking for X then youāre free to share youāre X with them. What you canāt do is simply walk around and say, āTake my X! You take my X and share it with people, NOW!ā. Thatās not very friendly to go around bossing people around like that.
So when people ask why canāt I just put something on IPFS and expect it to be replicated forever is like just showing up to your neighbors house and putting your junk in their basement because they have the room and expecting them to keep it safe for as long as you feel like putting it there.
You canāt tell people what to do but you can ask them to do things or pay them to do what you want. āIf I give you $5 will you hold on to X and give it to people when they ask for it?ā or you can just ask someone to do it for free but just donāt be surprised if they get distracted and wander off.
Some people noticed that people are willing to pay for a reliable service so they setup a booth and started selling the service of storing your stuff and sharing it when people ask for it.
So whoās providing the storage? Right now itās a mix of random people just being nice and people with booths who are being paid to provide it.
Filecoin is like adding money to our fictional universe. Right now the paid providers are being paid in money that appears from a far off land. You can pay for things and take it back to that far off land and use it there but there is no local currency. Filecoin allows someone to buy and sell their storage and to prove theyāre doing what they say they will.
Your analogy is great and a helpful way to visualize the current situation. What prompted me to ask this was that I keep feeling like Iām missing something - for example, in the latest IPFS weekly newsletter (#164) thereās a link to a video with the teaser: " Discord + IPFS? [Check out this brief demo] of uploading a file to IPFS directly from a Discord channel." Great, a new tool. They talk about IPFS like itās a place that has storage waiting for you to use. This kind of wording is used for almost all the new product announcements. āUpload to IPFS - itās great!ā, so Iāve been wondering - whoās ponying up the disk space for all these files that all these new tools generating. Are there a few large organizations proving āseed spaceā, or are there a bunch of hobbyists and casual/occasional people making some space on their ephemeral nodes available when they happen to be on line? Just a bit of a mystery to me. Paying for storage (with Filecoin) is understandable to me as it sounds similar to the STORJ model. But as far as I know, paying for storage with Filecoin is not available yet. Maybe there is much less storage space than my imagination has me believe and that all these files uploaded with these new tools donāt last long and just fade away after a while . . . .
I can see that. Unfortunately itās a tough choice. Either you have the nice simple, āUpload to IPFS - itās great!ā or you kind of go into a long detailed explanation of how exactly that works.
Itās more like a BYOS, Bring Your Own Storage/Buy Your Own Storage else rely on the charity of others. It doesnāt make for a great sound bite but itās better than the opaque system we have now where youāre not paying Facebook so theyāre going to find some other way of making money from it.
Either you use a centralize system provided by big tech, which u donāt have to pay for it in the beginning, or you can use your own d-internet with own server
I mean you are already paying for your fiber cable
I think the technology is much broader than that. the protocols are different. instead of storing files not needed at nodes, the nodes learn what to store and what not to make things more efficient based on whats being asked for by itās neighbors. there has to be alot of experimentation to figure things out and iron out the kinks. this is why we have the free service so that many users try it so the developers get to figure out whatās working and whats not. this is my understanding. I think this is really much needed and some funding is going to make it happen. this is my understanding, I might be wrong.
Iāve been wondering if we could use a system based on reciprocity (Iāll pin your stuff on my node if you pin my stuff on yours), and how it could be formalized into code the way filecoin has done it. Maybe itās an opportunity to build something (Iāll give it some more thought, Iām a dev for a living, I could possibly come up with something)
Interesting idea!
Do you know if there are any IPFS storage network statistics, like how many nodes there are or how much storage space there is?
This answers some of those questions:
Thank you very much for this! Honestly, I never thought that a talk titled Web3.0 would have this kind of info - I guess I need to get out more!
Thanks again
In a decentralize system, capitalism shows u provide people obligations to do it for you, some use crypto currencies but I will provide one vanilla example :You are in a network of your company, managers set up peopleās ipfs so they pin everything thatās shared with you, because it benefits the entire company if resources like internal resources (binaries, pictures, etc.) are distributed and available in ipfs without paying money to store them in one server.
Or maybe if you are on your own, but app for the future d-internet are hosting a chat room, it would of course be obligated to pin all the chat messages because you want them to be available even after that person turns off devices
To add to the conversation of this topic, I am understanding the storage is provided by your computer thatās running ipfs, but all ipfs can store files in a cache and provide it to others when they run into them, for example your computer might be hosting ipfsās documents now if youāve browsed the document through ipfs protocol which it is available on (install the ipfs companion in your browser to automatically browse websites straight from the ipfs network if the website has dnslink enabled ). but the cache wonāt stay live after garbage collection by the ipfs implementation, unless youāve pinned it.
Just watched this presentation and found it very interesting. It seems that about 14% of the total nodes (2735) are āalways onā and the other 85% are on for relatively short periods of time and then go off line. 2735 is actually more nodes than I was expecting, so, for me, thatās good! My next question now is . . .how much storage space do those nodes represent? Still digging!
Thanks again Yves, for this pointer!
I do stats from time to time, I found that right now about 8~11k nodes are always on.