Social Media Architecture with IPFS

Quanta now is fully able to function as a ā€œPinning Serviceā€ and afaik is the only open source pinning service there is. To clarify, Iā€™m not saying ā€œquanta.wikiā€ itself is the pinning service but that the open source project can be installed and run, to make your own pinning service, with your own branding even.

I may end up running this as a commercial service, but my primary objective here was not to make money, but to create an open source solution to ā€œcommoditizeā€ pinning services, kind of in the mold of how Linux commoditized operating systems. It means to operate a pinning service you donā€™t have to use a proprietary one, but you can just run a Quanta instance instead.

afaik Quanta is also the only Fediverse-capable app that has IPFS uploading as a core feature.

Iā€™ve recorded all new demo videos:

https://quanta.wiki/n/screencast

And hereā€™s the video on IPFS:

I am a noob to this, but commited to mastering it.
Explain this to me like Iā€™m a 1st yr OOP novice w/Python, C# & Java.

Are you saying that the Pinning services within Quanta can provide ā€œcommercialā€ grade IPFS Nodes? So youā€™d still have to ā€˜centralizeā€™ the service with co-located servers across the country/world to offer a distributed modelā€¦ is that correct?

Iā€™m all in, but need to understand the scope of the investment.

Personally, I want to believe in the decentralized model enough to launch my own IPFS node pinning service. Is anyone else moving in that direction?
Iā€™m just now (in the last 30-45 days) learning whatā€™s under the hood of such a ventureā€¦ has someone white-labeled a pinning service yet ? lol

I briefly go over some of those details here:

But to summarize: To create a full commercial pinning service, weā€™d have to add a credit card processing page to allow people to purchase storage, but the platform itself does already have the code in place to enforce a quota for each user, based on how much they have paid.

The other ā€˜expenseā€™ (overhead) to start an actual pinning service would be just to boost up the hard drive space on my Linux instance hosting service/server running the instance, so that it does have the storage (hard drive space) it would be selling.

But as far as any IPFS-related coding, yeah thatā€™s all been done. Quanta runs a docker container internally which is a GO-IPFS gateway. So anyone who installs an instance of Quanta has a pinning service right out of the box, and it is white-label too, although no one has ā€œtriedā€ afaik to run it under their own name (the GitHub gets about 6 to 10 downloads per day, and I never hear back).

Iā€™m looking for investors to continue this platform, but since the entire project will remain open source many investors will realize their competitors can just run Quanta too, and that therefore pinning services may not be a lucrative investment, for the same reason no one is ā€œsellingā€ Linux.

The main business model here would perhaps only be in hosting and support (i.e. a company to specifically host Quanta for organizations for outsourcing both the IPFS features and the Fediverse features), but again from my personal position, no one needs me either because they can just take the code and run. Iā€™m fine with that. Thatā€™s what open source is all about.

(Iā€™ll probably paste the above text into Quanta under the IPFS info node, since itā€™s a good question, and the answer is relevant to potential interested other parties.)

Hi and thanks for this amazing discussion.

I am a junior dev with a background in economics and game theory with a particular crush, so to speak, on the idea of building a distributed social network.

A few months ago I started to think about the issue ā€œhow to distribute a social networkā€. At the time, I did not know about IPFS and I come up with this idea:

  • each node publishes its databases as a static web site exposing JSON
  • each node publishes its contents according to a publicly understood schema
  • each node continuously crawls the network spanned by its nodes list and updates its cache

Thatā€™s it for what concerns the nodes. On the other side, there might be different front-end applications offering different social media experiences on top of the same data.

Finally, given a common data infrastructure and a network of nodes, there is space for an ecosystem of applications that may give rise to completely new social media experiences.

Then I understood that I was just playing around with the idea of the semantic web. This led me to IPFS and IPDL.

I tried to migrate my schema to an IPDL schema and I just produced a (signed) history object which can be traversed to retrieve all the information that is requested by a front-end application to represent the network. This was fun!

Now I am reading this long discussion and I would like to ask you all

  • is there a dedicated place for this kind of discussion? It seems like a hot topic! Maybe there is a dedicated platform that I did not find? I would love to try to give my contribution
  • in previous responses there are some attempts to describe a common data structure. An IPDL schema the community agrees about would be great I think, is there someone working on this?
  • what do you think about the idea of decoupling the front-end applications from the database management? Do you think that front-end applications could compete for having nodes using their services?

(random: Thank you for your time! If you use Twitter and you tweet about this I would like to follow you)

1 Like

hello LucaPanofsky

To discuss the subject in general there is this thread. As far as I know there is no central platform. Everyone does a little bit as he wants.

Around IPLD after reflection on my side it would be interesting as you mentioned to create perhaps a working group to create IPLD schema (link here) which would allow to see several frontends appear. That people interested contact me I will perhaps launch a github repo in a first time and we will see what we can do.

For the decoupling of DatabaseManagement and the frontend is for me a viable solution but to see with other opinion.

1 Like

There are several protocols out there but my money is on ActivityPub as being the ultimate ā€œwinnerā€ in the Social Media Protocol space, and I think currently most people equate it with the ā€œFediverseā€.

You can go here:

https://activitypub.rocks

No one has proposed a ā€œmergedā€ IPFS and ActivityPub spec yet but it does seem possible. Iā€™m still busy finishing the AP support in my own platform, and want to have that before I try to suggest/promote improvements in the AP spec around IPFS.

1 Like

Weā€™ve just released our social media protocol in Peergos. Every social post is just a file stored and shared using the built-in encryption and access control.

https://peergos.org/posts/decentralized-social-media

2 Likes

Ultimately, I believe the hardest part of decentralization will be determining where the balance should lie between sacrificing some of our beliefs of what is right/wrong in the name of privacy and sacrificing some of our autonomy in the name of morality.

What if we could create a form of decentralized, or at least democratic, enforcement of standards. For example, all users could be eligible to volunteer to be a social coordinator of sorts, but they could only restrict or remove content or types of content that had been previously voted upon by the IPFS community as a whole, with each user having one vote.

Thatā€™s the best idea I have at the moment; hopefully it will add some value. :sweat_smile:

I think the problem with ActivityPub right now is that it creates lots of ā€˜fiefdomsā€™ where whatever server you join becomes your local ruler and king. This is not good. What we need is a situation where no one has any authority over anyone else. This doesā€™t mean you cannot block someone. If you have the ability to block someone then you are in charge of your life and they are in charge of theirs.

So Iā€™ve come to realize we just need something like ActivityPub (perhaps almost identical, or even based on 99% of AP) but which is MUCH easier for anyone to setup, and where everyone can even be the ā€˜sole proprietorā€™ of his own ā€œinstanceā€. In other words if everyone in the world could easily create their own Mastodon instance (or Quanta instance, which is my own platform I wrote) somehow, and be their own King that would be ideal, and is consistent with what most call ā€œdecentralizationā€.

The other big competitor to AP is Matrix I think, but we need to get away from the the requirement that to be a ā€œnodeā€ in this network you have to register a domain, get https configured, get a database installed, and run thru the hour long process of setting it up that only a software developer can do. It needs to be an app anyone can click and install and get onto the network just that easily.

Then you can control your own feed too. No more manipulation of feeds. Youā€™d get an ā€œhonestā€ feed of your own that only blocks content based on the rules YOU decided, and importantly you canā€™t block or even cast a vote about anyone else.

1 Like

Hello,

Iā€™m still reading this thread and I feels like thereā€™s a lot of good ideas in here.

Also, Iā€™ve built tools to do decentralized live streaming and live chat. Check it out!

1 Like

Hereā€™s a blog post I wrote showing just how easy it can be to get a minimum viable Social Media Network off the ground using nothing but IPFS PubSub:

The solution has been staring us in the face for about 4 years already, and we just havenā€™t realized how to harness it. Kind of reminds me of how XMLHttpRequest and DOM Updates via Javascript existed for about 5 years before developers realized they can build Web2.0 on that. But this time itā€™s Web3.0, and the ā€œunrealized toolā€ is simply IPFS PubSub.

A lot of people (including me) have been so busy trying to create something full featured that can compete with Mastodon/Twitter, that we forgot to just focus on the MVP first, which imo should be a protocol so simple a 13yo just learning JavaScript can jump in and write a client to at least read and post, in a couple of hours.

Another anecdote from history: RSS/Atom. Think of how simple RSS was, yet it created the podcasting industry. So Iā€™m saying a Social Media Web3.0 can be just that simple too, and for something like a ā€œLevel 1 Complianceā€ all it really needs to be is a JSON format for a cryptographically signed message so anyone can post to a topic, anyone can read on that topic, and NO ONE can censor anyone else.

IPFS PubSub already gives us this!!

1 Like

Responded on discord but I have more thoughts.

GossipSub v1.1 gives us hardened PubSub messaging and we can use IPFS for persistence.

The question is always WHO DECIDE? In this case it would be who decide what is persisted or not?
We could make it live only but then itā€™s not very Twitter like.

CRDTs could be used to links messages together but again who decide? We would need some kind of consensus algorithm and a blockchain. Maybe Microsoft ION?

What about spam? Which is really a question about decentralized identity. The Upala protocol has some good ideas has well has BrightID and Idena.

IPFS & GossipSub are so powerful tools, if only we could natively access IPFS API from the browser.
Soon I hope!

With #IPSM (what Iā€™m calling this ā€“ Social Media over PubSub) anyone can store a permanent record of all the messages from everyone they follow, and/or all the messages they themselves post, using ā€œpinningā€.

There will also be individuals and services recording the IPSM Firehose all day long for later analysis, to enable search services, etc.

Btw, youā€™re probably aware but, Brave Browser does support IPFS protocol already.

With #IPSM (what Iā€™m calling this ā€“ Social Media over PubSub) anyone can store a permanent record of all the messages from everyone they follow, and/or all the messages they themselves post, using ā€œpinningā€.

Thatā€™s almost what I did except in my case I use IPNS to anchor the feed of content.

There will also be individuals and services recording the IPSM Firehose all day long for later analysis, to enable search services, etc.

Thatā€™s the power of Web3 protocols VS Web2 platforms!

Btw, youā€™re probably aware but, Brave Browser does support IPFS protocol already.

It does ā€œsupportā€ it but, no PubSub, no IPNS over PubSub and the only thing you can do is IPFS cat via the browser fetch API.
Thatā€™s not good enough! I would need IPFS; pubsub pub, pubsub sub, dag get, dag put, id and name resolve

To your point about a common format with encryption.
Thatā€™s what IPLD is for. Could be JsON or CBOR, doesnā€™t matter really.

Iā€™m looking at switching from my rickety custom signature thingy to this. IPLD ā™¦ DAG-JOSE Specification

The cool thing about IPLD is that you donā€™t need IPFS itā€™s separate.

Hello,

I wrote pronto with the idea of a decentralized semantic network with no other dependencies than IPFS itself.

Rather than depending on an external blockchain, the chaining is done in RDF graphs. Content goes from JSON-LD/IPLD to RDF via JSON-LD expansion (schemas are stored in IPFS). Synchronization is done peer-to-peer via SparQL.

This results in a giant P2P RDF graph, easily extensible with new schemas ā€¦ i think a social media application is quite easy to build on top of this because of the power of linked data. Pronto already uses decentralized identities (with IPID).

Iā€™ve made docker images so itā€™s easier now to run a node and try pronto.

Actually, although this originally seemed to me like a mainly PubSub thing, and still is I guess, but I ended up realizing that the ā€œlive broadcastā€ of the PubSub events is merely one way to share them, that happens to be as close to realtime as IPFS gets. However these lists of events (either filtered per person, or raw unfiltered) can be saved at IPNS named locations too for asynchronous processing/consumption.

BTW: I made a ton of progress today and did more testing on PubSub. Looks like IPFS can run reliably once enough peers are defined.

Hereā€™s the top-level link that encompasses all of it.

That sounds cool. Iā€™m going to look into everything you mentioned. I havenā€™t fully investigated PublicKey-based identities (i.e. best in class solutions), and the biggest challenge we know of and want to overcome is the spam problem. We were thinking of one option being to require a crypto account to at least exist with money in it, as an ā€œoptionalā€ way people can choose to attempt to filter out spammers, but that was just a brainstorming idea.

The best thing about IPNS is that it can use PubSub so you get Anchoring AND live update.

Super exciting! We separately converged to the same solution, which is a good sign.

@wclayf sir you need to catch up, my defluencer protocol already works like you described, else I will be the one to put Big Tech out of business! :stuck_out_tongue:

@reload Cool Stuff! Is it on Galacteek only for now? How can each node track all chains?