Social Media Architecture with IPFS

[crockwave]

The world needs a distributed social media system, in order to achieve the resilience required in times of turmoil. Social media is a very effective tool for allowing citizens to audit and correct societal issues. The obvious flaws in current social media platforms: exchange privacy for free use, user censorship, single point failure, etc., are driving the use case for developing new social media.

This thread should compile a list of the issues related to making the social media over IPFS use case happen, so that the IPFS team can potentially fold these use cases into the architecture, and so that we can more efficiently define an architecture and get a github project started.

Previous contributors to this subject include @MichaelMure, @musicmatze, @StormFire-Technologi, @rklaehn, @karthik-ir, @MirceaKitsune, @josselinchevalay,

Matthias Beyer (@musicmatze) has given some thought to the issues here and here and locally here

Michael Mure (@MichaelMure) has created Arbore. The Arbore project uses an earlier IPFS release and was last updated in April 2018

Many of the issues have been discussed, and solutions should be defined for all perceived issues, so that the architecture can be defined.

Distributed Social Media Issues

1. Topics (Rooms/Groups) - Allow users to create topics (rooms/groups), users to be able to discover topics, and users to be able to subscribe to and remove themselves from topics. Use ipfs-pubsub-room. Each peer would store all of the data for all of the Topics that it is subscribed to. As the quantity of peers grows for a given Topic, the persistence of the data will improve, and the data will fade from existence based upon peer cache size limitations.

2. Chat (Profile to Profile/Peer to Peer messaging) - Use ipfs-pubsub-room

3. Data Format - Meta-data + User Data in DAG - (From M. Beyer second post)

   {
   “version”: 1,
   “previous”: [ “Qm…1234567890” ],

   “profile”: [ “Qm…098765”, “Qm…54312” ],

   “post”: {
   “mimetype”: “text/plain”,
   “nodes”: [“Qm…abc”],
   “date”: “2018-01-02T03:04:05+0200”,
   “reply”: ,
   },

   “publicfollow”: [ “Qm…efg”, “Qm…hij” ]
   }

4. Temporary Federated Component - Temporary federated components could provide an effective way to make peer connections, while not being required once peer connections are established. Use multiple rendezvous servers running something like p2p-websocket-star to allow peers to easily find each other. For resilience, if rendezvous servers get taken down, peers would maintain and disseminate a dynamic list of N rendezvous servers. Once peers find each other, they would minimize use of rendezvous servers. They might ping rendezvous servers periodically to see if their rendezvous server list needs to be updated and if there are any new Topics available for subscription. Rendezvous servers would try to keep track of other rendezvous servers, ranking them so that the top ranked servers would be able to be disseminated to connecting peers. Rendezvous servers would also try to keep track of subscribable Topics, ranking them by frequency of use so that the top ranked Topics would be disseminated to any peers requesting a list of Topics.

5. Linking Posts and Profiles - IPLD Not only linking posts and profiles, but linking posts and multiple objects within a post, such as emoji, URL, text, GIF, image, and any combinations of these objects

6. Multi-Level Replies - pubsub Refer to both first post link and second post link above

7. Multi-Device Support - Publishing Content - Refer to first post link above and second post link below

8. Multi-Device Support - Subscribing - Refer to first post link above and second post link below

9. Deleting Profile Versions - CRDTs in IPLD. Refer to first post link above and to Profile DAG in the second post link below

10. DAG Structure - Refer to Matthias Beyer’s second post

11. Deleting Content - CRDTs in IPLD. Refer to second post link above and the local discussion

12. Forgetting Content - IPLD. Naturally handled by activity and cache size. Refer to second post link above

13. Subscribing Privately - CRDTs cached locally? Refer to second post link above

14. Discovering Content - Discussed in second post link above (Temporary Federated Link helps get started, but maybe peers notify other peers of topics they subscribe to, so that those peers offer new topics to users

15. Ease of Use - The platform would need to be nearly as easy to use as FB, Twitter, or MeWe

16. Storage Incentive Without Storage Charges - The platform would need to be free to use, to allow third world country use. Current social media provide free use (data storage), in exchange for giving up privacy (targeted ads incentive). The new platform would need to have all accounts in a given topic (social media group) be mutually incentivized to store all of the data for the topic, without being burdened to pay for storage to third party IPFS nodes. That means the entire topic is cached by all subscribers to a topic, and no storage charges are exchanged between those topic subscribers.

I have further notes on how to enhance the UI and user experience on a social media platform, which would just be details in the infrastructure, once the basic infrastructure is proven.

[MirceaKitsune]

First of all, thank you for noting me and for making this excellent thread. I’d like to take a moment to reiterate my own view on the matter and explain how I personally see this issue.

We’re all familiar with centralized social media (Facebook, Twitter, Tumblr, etc). This model had nearly a decade of glory. Unfortunately it’s now reaching the point where it is no longer sustainable: With the rise of authoritarianism worldwide (including occidental countries) the new norm seems to be blaming web hosts and holding them liable for what users do on their platforms. With many governments already passing laws to make tech firms enforce censorship on their behalf, conventional social media is expected to eventually become as strict as television or radio shows.

Decentralized social media platforms are already providing a solution. The latest innovation here seems to be the ActivityPub standard, spearheaded by platforms such as Mastodon or Pleroma. Everyone can host their own server (their very own Twitter clone) and those servers communicate with each other independently. Currently the fediverse provides just the fix we needed, considering those sanctions and expectations only apply to tech companies and not ordinary people at home. But will even that last? How long until governments start demanding upload filters for copyright / hate speech / porn / whatever under threats of knocking at your door if your Mastodon or Peertube or GnuSocial instance isn’t modified to work by their demands?

That’s why to me, the holy grail is not just decentralized but also serverless platforms: A model in which no one has to host a server, and clients using the system handle all processing and hosting on their own. In this case a third party doesn’t even know who the owner of a platform is unless they mention… in fact platforms don’t even need to have owners as we could have profiles acting as independent sites communicating with one another directly. The only way to take any action then is to either criminalize the software (encryption and libraries like IPFS) or go after hundreds of people across the planet, without even knowing who’s the originator or just a viewer who accessed that page and got it cached on their drive, making law enforcement ridiculously hard without some crazy mass persecutions which I doubt even China would try at this scale.

This is where IPFS comes in: With the help of OrbitDB or similar databases, the serverless model can be achieved to a good extent! The secret is to rely on strong security keys so that any browser can read and modify a database if it has access but no one can touch it if they don’t… that is trickier but I’m sure we have brilliant developers already working on it. Of course there will be limitations: You can only do so many things under browser JavaScript so you won’t get all the features of having a server optimized to process stuff, like say compressing images and videos to lower quality before uploading to IPFS (maybe browsers can do that one now). Still I think it’s very worth it to consider building something like this.

[pubswap]

I agree with everything said so far, though the serverless idea does come with limitations as noted. One important one being security concerns.

I have designed and developed a distributed content distribution system (pubswap.com) for video content much like youtube. One thing I have learned in doing so is the need for associated meta data of each piece of content and the need to index that data and to be able to search that data. For instance, I can upload a video and get assigned a hash and then share that out but that is not too practical. There are useful pieces of information, such as title, description, tags, likes, dislikes, comments, etc. none of which are supported by IPFS. This makes sense as IPFS is the protocol and NOT the application.

First and foremost I think we need to keep IPFS as the protocol and don’t attempt to make it do things it was never designed to do. All of these ‘extras’ are server-bound and not decentralized. My intent for my service was to address the two largest issues with video content distribution. Those being storage space and bandwidth requirements. IPFS has solved these two issues quite nicely. Of course, intelligent people can come up with a number of solutions to decentralize and support this additional content but some service would have to manage it and would probably be a centralized service.

I mention all of this to lead up to what I feel is the real question and one I did not consider prior to launching my service and this is, should we be doing this? Given the nature of my service, I can still block and/or remove content from being INDEXED in my service but cannot prevent the content from being published to IPFS. Most of us, including myself, see this as a good thing to avoid the current BS censorship but can this and will this be used in ways not desired? I am thinking it will be.

The last thing I want for my service is to be rife with child porn, snuff videos or have copyright issues that would not be able to be addressed. I want to aid in the free expression of thought but do not wish to facilitate child sex trafficking or the like. If we FULLY decentralize all content and go one step further into complete serverless land, will we regret it? If we don’t do it, will someone else? Has Pandoras box already been opened?

I’d be curious to hear others thoughts on this question of ‘should’. IMO the worst thing we could do to IPFS right now is dirty its reputation as a child porn protocol. Blockchain is already being attacked as a tool for terrorism and whatnot. I could see IPFS having the same issue in the future.

[crockwave]

@pubswap Thanks for the response. I’ll try to respond to your general philosophical question. As a newbie to IPFS, I don’t want to try to discuss technical, but can respond from a philosophical point-of-view.

The idea of any community self-policing is always a good idea, as the reputation of the community is important to the livelihood of that community. That should translate to the IPFS community as well. Just as with today’s web, there are both good and bad forces at work, with the good trying to overcome the bad. That would translate into the good subverse keeps itself cleaned up and the bad subverse stays marginalized.

I am currently using MeWe desktop and mobile apps. It might be a good example of how the content issues are addressed. MeWe is a centralized social media service. Its weaknesses are central point of failure and risk of censorship.

1. Terms of Service - To create an account, agree to terms of service. Maybe IPFS official releases could have a Terms of Service feature, such that any new account must agree to the terms before comms can commence
2. IPFS Seal of Approval - Maybe IPFS could apply a seal of approval to certain applications that use good behavior/reputation management tools. Potential users could look for the IPFS Seal of Approval before using the app
3. Group Owners - Group owners (Topic/Room Owners) have the ability to define their own rules of behavior and can boot accounts that do not abide by the rules
4. Account Reputation - Maybe social media solutions could be engineered to allow account reputations to be adjusted by other accounts, and those reputations get disseminated
5. Peer Blacklists - A peer blacklist could maybe be created such that bad actors would need to create new accounts to not be filtered by peer blacklists

[ststefa]

Very fundamental and inspiring thoughts indeed. I have a feeling that the goodguy/badguy topic cannot be solved on a technical level. It might be helpful to look at how the analog world is tackling this problem.

If I think into that direction for one or two steps then it quickly arrive at the term “identity”. Identity is (imho) one of the most fundamental building blocks in the real world which allows to address/discuss/handle the problem of use vs. misuse. Maybe an ipfs-based social network would benefit if it would be technically somehow intervowen with some kind of trustable identity? The term “publicly trusted blockchain” spontaneously comes to my mind in that context.

This foundation could maybe erect some kind of democrasized structure where the majority is able to drag the platform into the desired direction without any single entity being able to dictate that direction. Sure that would raise privacy concerns on the downside. But maybe then (and I’m really scared to say) we cannot build “desirable platforms” without sacrificing privacy?

I hope that, despite my limited (non-native) knowledge of English, I was able to clarify my thoughts to a certain extent.

[crockwave]

The Account Reputation bullet item in my last post is a way of building “trust” while retaining anonymity, where the peers that you interact with vote your reputation up or down. Those with good reputation values could roughly be assumed to be good guys, causing the entire platform to congregate activity around those accounts with good reputations. Voat uses a up/down voting process, but it is too volatile. Limiting up/down votes to once per day per remote IP address could remove some of that volatility.

[ststefa]

Yes, account reputation sure is a good idea. But I think it’s not of much use regarding “trust building” as long as it’s an isolated solution limited to one or maybe a handful of platforms.

Take the two of us as an example. You and I could be two individuals trying to come up with creative thoughts to build a better social media system. I could as well be a criminal who wants to influence the development in a way that might better serve my criminal intents. And you could be some NSA person who tries to move the development into a direction which is better suited for surveillance.

There is absolutely no way in which either of us could trustfully judge which of these is the case. At least partly because we have no broader “evidence base” than what our specific accounts have been saying on this specific platform. Both of us could even have numerous such accounts for numerous purpose.

I think building trust for something as broad as an “ipsn” (=“interplantary social network”) requires a much broader base. For example, if you had a trustable way to lookup what I (very different from my account) have been posting on the internet recently then that might give you some (trustable) conclusion.

I still think it needs something like an unforgeable and unchangeable 1-to-1 relation between physical and digital identity. Otherwise an account will stay a “throwaway identity” which I can risk to “burn” by undesirable activity anytime. I think digital identity needs to have a value so high that one cannot risk to burn it.

[crockwave]

OK, very good points, but I do not buy the argument that value cannot come from digital identities that are not verified against physical identities. I do agree that you can obtain immediate value by allowing your digital identity with a physical identity, but it should not be mandatory. An “ipsn” (=“interplantary social network”) needs to allow both anonymous and non-anonymous digital identities.

Due to the secure nature of some digital identification systems, such as what ipfs provides, you can safely associate content history with a particular digital identity. There are particular physically anonymous digital identities on 8chan and Twitter, as examples, that have been determined by peers to provide significant value, without the need to doxx those identities.

ipfs ties content to a digital identity. The entirety of the content associated with a digital identity is one form of trust-building and self-censorship, similar to the resume of physical identities, and the reputation of the identity can be established through peer review of the identity’s content history.

When you further tie a physical identity to a digital identity, you can use the combination of the footprints of both the physical and digital identities to establish reputation.

The “ipsn” therefore needs to be able to support a Verified feature similar to Twitter, which indicates that the digital identity has been tied to a physical identity and confirmed.

The Account Reputation feature I originally posted, still adds value in allowing peers to provide feedback to digital identities in a way that could make sorting by reputation easier when choosing who’s content feed to subscribe to.

[josselinchevalay]

Hi all,

First of all I am delighted to see that the topic is generating so much interest. I took the time to read you and think about this topic. I am not an expert in IPFS or any other field that engages only my point of view.

IPFS must absolutely remain a protocol and the fake be thinking as wide as possible. This specialized just to cover the case that interests us: a social media does not have or little impact on the roadmap of Juan and his teams. For that i’m agree with @pubswap. However IPFS is a protocol that allows decentralization the notion Client / Server is supposed to disappear. In the case that concerns us I prefer to talk about nodes or instance (as for mastodon https://joinmastodon.org/) I think it could be a good approach for some meta-data evoked by @pubswap. Many of you talk about trust and group and content management I think the notion of nodes or instance and not bad for that. People with knowledge can create their instances and link to others or join an existing one they trust. I think that technically supported by orbitdb / pubsub feature and web server.

Identity is the second point of my response yes @ststefa identity is good and important point. you have to find a fair balance that allows you to identify a person and to allow certain people to be able to anonymously post things do you know the estonia e-republic https://e-estonia.com/ or https://tse.bitnation.co/ maybe that can be instresting to see that to create a strong identity system.

I’m waiting for your feedback on it’s thank you points

Regards

[ststefa]

For me, the most fundamental question is which decision-making system an “ipsn” should be based on. A question which has been (and still is) learned slowly and painfully over centuries in the real world. In my opinion, democracy currently is the best model we have at our disposal. Although it is certainly not perfect. It is slow, decision-making is complex and time-consuming. However it is the only model that I know of which emphasizes on equality and the voice of “the people”.

So should we try to rebuild the model of democracy in the digital world? Or should we try to invent something entirely new? Imho this carries a severe risk of reintroducing problems that had already been solved. My best guess would be to take democracy as a template and try to apply it to the digital world. And apply careful modifications where it does not fit or where we think can do better using new digital capabilities. Certainly democracy in “digitalia” will not be the same as in “pangea”. My point is only that it could be the best starting point that we can use.

I come back to my point that one of the cornerstones of democracy is the “one voice per creature” concept. This alone is difficult to implement, as anonymity is an important value at the same time. Without anonymity, society is susceptible to manipulation. This can be seen very clearly in cases like Cambridge Analytica where elections can be potentially influenced by micro-targeting. So we need “one voice per creature with anonymity at the same time”. Which happens to be the exact same concept like in “pangea”. Just how do we do that in digital? This question must be answered not only for “ipsn”, but for the digital society as a whole.

Decision making in a democracy is complex, because everyone can (and should) give his opinion on a topic. At the same time, the multitude of opinions makes decision-making confusing and time consuming. We can see this very well, for example, in predatory scientific journals. It has reached the point where quickly written assertions can stand at eye level with elaborately scientifically researched figures. Ordinary citizens like me hardly have a chance to unfold this difference and therefore perceive it as equal opinions. This is the birthplace of alternative facts. I think that in an “ipsn” there must be a concept of “weight” to an opinion.

[josselinchevalay]

hi,

your “weight” to an opinion link reputation right ?

Regards

[crockwave]

I’m starting from the premise that one can have multiple digital representations of themselves in an “ipsn”, with each instance being an opportunity to try to influence others. Each representation can be either anonymous or tied to a physical self, giving it potential “weight”. In addition each representation could have a “reputation” tagged onto it, applied by peers on the "“ipsn”. The audience is motivated to efficiently sift through the multitude of voices, so they will reward those who behave properly by helping to adjust their reputation upwards, by replying to them, by not blocking them, and by forwarding that information.

It is the responsibility of the audience to keep the “village”/group clean. When strangers appear, evaluate them. On the sum, the village thrives.

It behooves the influencer to gain as much “weight” as they can, knowing that their efforts will be wasted otherwise. It behooves the audience to examine anonymous information for potential value, while paying more attention to information coming from influencers that have “weight”.

Each account that an influencer creates would have an account name and avatar, which would carry a bit of weight if the audience recognizes the influencer and remembers any previous valuable information provided by that influencer. As long as the “ipsn” required unique account names, trust can start being developed with anonymous accounts.

The audience would be able to observe account name, avatar, reputation value, and the information posted by the account, to determine whether or not to pay attention to that particular post.

On decision making that affects the entire “village”, e.g. voting, anonymous accounts would carry no weight in that process.

An “ipsn” chat is for general purpose information gathering/sharing, with varying levels of anonymity. An “ipsn” group is for posting articles, with multi-level comment thread available for each post.

One other critical element to managing the “village” is a wiki, where the village can collaborate on managing social issues and information repositories via the wiki. The way I am currently handling these 3 critical elements is using MeWe chat/group and using MediaWiki. The MediaWiki element requires a vetted committee to manage the information repositories. A distributed wiki can be a topic for a future conversation.

Bottom line, there is value to be derived from both anonymous and non-anonymous accounts in an “ipsn”

[ststefa]

I have pondered quite a bit about that question until I read crockwaves reply (above). Sure “reputation” and “weight” are closely related. They can be almost used interchangeably even though they are not conceptually the same. My best attempt to illustrate this thought would be to consider “weight” as something which is gained through reputation. However, there might be other factors which influence weight. Like maybe a verifiable link to a physical identity. Or perhaps a generally accepted proof of competence.

The interesting point here is that “weight” cannot be implemented by an explicit mechanism alone, which may even be tied to a single platform. It also needs to build (as crockwave suggests if I get him right) implicitly through “trails of usage”, ideally gathered throughout the whole digital space. I think weight/reputation is a multidimensional property. That’s just like in the real world. It cannot be described using a single value. Rather, it is a complex woollen clew consisting of various threads.

[josselinchevalay]

hi stefa,

instead use “weight” why talk “credibility” : [the quality of being trusted and believed in.]. reputation is based for exemple on numbers of person follow you or give thumbup but the credibility can be calculate how much a article or post is mention or re-use another comments and other person. Another case when you have more credibility when your stuff are verified by another person. what do you think about that ?

Regards

[ststefa]

Yes, my impression is that all the three of us are thinking into the same direction on this.

[ststefa]

It seems I have missed to explicitly agree that both types of accounts have their justification. I do. Probably even both types are required.

On decision making that affects the entire “village”, e.g. voting, anonymous accounts would carry no weight in that process.

I think this is where our arguments meet.

First I’d like to express your points again in my own words to understand whether I follow your argument properly.

• Reputation and identity are two separate properties of accounts. Reputation is virtually inevitable, because it (potentially among other factors) results implicitly from its activities. Identity is optional.
• In the long run, both types of accounts are required to run the “village”.
• Referring to the real world: Accounts with higher credibility/weight get a louder voice in a discussion. But when it comes to voting for the “village” (or decisions of comparable importance), only non-anonymous accounts are entitled to vote.

Maybe then a publicly known link between an account and a physical identity is even undesirable. Maybe “identity accounts” should be used solely for voting with not even a possibility to use them for anything else. Reputation and weight would thus be accumulated under different (anonymous) accounts. This might mimic the real-world model of anonymous voting and public reputation quite nicely.

The topic of reputation sounds solvable (albeit complex) to me by setting up corresponding rules and relationships. However it is probably important to keep these rules “fluid” so that they can be adapted to new insights and observations using some kind of “village voting process”.

Without having thought it through extensively, I think that the digital identity requires the participation of official institutions. If I may make a wild speculation: Maybe the solution could be a combination of a chain connector like polkadot with (national) services similar to the aforementioned e-estonia.com or tse.bitnation.co. This speculation, however, is no more well-founded than a squirrel’s conclusions on quantum computing (sorry squirrels, no offense).

Maybe then, the question of identity should be tackled in a different discussion. But then again, what would be the foundation of the “village voting process” which is the foundation of the “reputation building rules”?

[crockwave]

In reply to your bullets, fully agree. The definition of identity in your description is the tie between an account and a physical identity. Anonymous accounts have a form of identity, to allow their reputation to build, but they don’t have the tie to a physical identity. Twitter has Verified account method of tying account to physical identity. Maybe “verified identity” is a better term for your definition of identity.

In many cases, publicly known links between an account and a physical identity are undesirable, hence the support for interacting with fully anonymous accounts. In other cases, someone may want to rapidly build their “reputation” for a given account, so they go through the “verified identity” process for that account, which could give instant “reputation” value.

“Reputation” management systems could vary by platform, but displaying reputation value for a given account makes the “ipsn” more valuable, to help users sift through the information flow more effectively. The idea of one reputation upvote/downvote per user per day for a given account is just one possible implementation of a reputation management interface.

Agree on the official institution method of verifying identity, e.g. certification organization. This part of the discussion would be best served on another discussion thread and include sovereign identity protection concepts.

On reputation building rules, “reputation” can follow account, but in today’s world, each social media platform has its own account universe. The “Login using FB” or “Login using Twitter” are some schemes to allow you to carry your account with you to different platforms. Maybe a universal “ipsn” account with associated “reputation” and/or “identity” could provide value when a user traverses different implementations of “ipsn”.

When we start discussing identity, voting, etc, with both anonymity and non-anonymity, the implications become broader than just a social network. Other parts of the village come into play, e.g. contracts, conducting business, in addition to having conversations.

[josselinchevalay]

hi,

yes maybe we can think to create a forum about all this question. Maybe it’s time to create a group of talking about that.

Regards

[carsonfarmer]

Hey all, just getting up to speed with exciting discussion going on here. I represent https://textile.io/ who are developing https://textile.photos on IPFS. We’re very much interested in being involved in this convo and any group(s) that spin out of it. We’ve been forging forward on many facets of the discussion here, so obviously have developed some opinions Hopefully we can share some thoughts etc here!

Carson

[josselinchevalay]

Hi carson,

Sure all persons can we participe are welcome ! ireead your presentation on https://textile.io so your develop web application with another approach based on transperency etc. that’s sound good.

Regards