Social Media Architecture with IPFS

Ecosystem
41

The original idea was to use a DHT to serve(requests for a particular public key) a magnet link(and signature) to a json file containing profile information, including other magnet links which represent a users publications. I experimented with IFPS/IPNS a bit and liked the API, so I decided to make the switch. Now, instead of the DHT responding with a magnet link, Iā€™m using IPNS to respond with an IPFS link which points to the aforementioned json file. This architecture can be used to publish arbitrary content types (videos, blogs in the form of txt/html files, etc) and the UI(electron+angular) will render the media accordingly.

Development has been going well, expect I canā€™t seem to get IPNS to serve the IPFS links outside of my local network. Until recently, Iā€™ve been using the default config. Anyone have any thoughts on a more reliable way to configure? I will probably post this in an OP. So if this seems like the wrong venueā€¦

1 Like
42

For public network, you need DNS Link ?! https://docs.ipfs.io/guides/concepts/dnslink/
But, you need an API access to your DNS and changes propagation takes timeā€¦

Any social / sharing / logging layer above IPFS would be awesome nowā€¦
@iohzrd did you have a look to Textile. It has a good documentation and functionalities can be extended to any formatā€¦
What is your opinion?

43

I wasnā€™t planning on having the identity files be accessible via http/s / regular browsers(except via webtorrent). Rather, have electron acquire them directly through ipfs(which would already work if IPNS was more reliable/ i could figure out how to configure it properly) , so DNSLINK isnā€™t relevant. and considering What Iā€™m going for is a publication / subscription (think youtube minus all the manipulationā€¦) platform, where ones content is, well, public. Iā€™m not sure textile is of much use TBH. I see no need to do anything more complex than serve json files containing magnet/ipfs links. One of the main design goals for this project is censorship resistanceā€¦

44

In fact, webtorrent is so reliable at this point that Iā€™m not even tied to using ipfs for the distribution of the actual content files, and therefore am primarily interested in using IPNS as a means to have a persistent identity pointing to mutable content (the identity file)

1 Like
45

abstract flowā€¦
IPNS / dht.GET(pubkey) -> ipfs_hash/magnet_link -> profile.json = list of ipfs_hash/magnet_link(s) to users publications -> magnet/ipfs link -> content.

example identity fileā€¦

{
    "arbitrary_key": "arbitrary_value",
    "display_name": "iohzrd",
    "email": "iohzrd@protonmail.com",
    "homepage": "https://www.example.com",
    "identity": "QmYHExHtTzFEjdwyQcJkD9gaKKgfwvGswoFb4simrJn5Q6",
    "pgp": "",
    "publications": [
        {
            "desc": "",
            "title": "1200px-Greek_letter_uppercase_Phi.svg.png",
            "file_name": "1200px-Greek_letter_uppercase_Phi.svg.png",
            "magnet": "magnet:?xt=urn:btih:359eadd749a2cbf8009468a77d05809d46bd8448&dn=1200px-Greek_letter_uppercase_Phi.svg.png&tr=udp%3A%2F%2Fexplodie.org%3A6969&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969&tr=udp%3A%2F%2Ftracker.empire-js.us%3A1337&tr=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337&tr=wss%3A%2F%2Ftracker.btorrent.xyz&tr=wss%3A%2F%2Ftracker.fastcast.nz&tr=wss%3A%2F%2Ftracker.openwebtorrent.com",
            "ipfs": "Qmc7j7QsVo7Zi8TMqLuVpGA1vXDTmPZVr7rXVkd6tjaGro",
            "ts": 1556395572915
        }
    ],
    "publications_archives": {
        "1": "ipfs/magnet link to prior content list that got too big(>100 or something)",
        "2": "using this we can do content pagination"
    },
    "some_cryptocurrency_address": "3FkenCiXpSLqD8L79intRNXUgjRoH9sjXa",
    "some_social_media_account": "https://twitter.com/iohzrd",
    "suggested": [
        "friend_identity1",
        "friend_identity2"
    ],
    "timestamp": 1552563530
}
47

Screenshot from 2019-05-21 16-34-01.png1920Ɨ1080 150 KB
Screenshot from 2019-05-21 16-34-14.png1920Ɨ1080 150 KB

48

my approach to this, is simply to not have a reputation / rating system at all and leave all of that to good old human interaction (sharing IRL, mirroring, whatever) or at most, each identity having a ā€œsuggestedā€ field (see above) so if you trust one publisher, likeliness is youā€™ll enjoy publishers they subscribe to. Basically, word of mouthā€¦

1 Like
49

Itā€™s easier to over engineer something than it is to get it just rightā€¦ and to be frank, i donā€™t think there is a way to get a reputation / rating system ā€œjust rightā€.

50

Holy molyā€¦ Iā€™m a bit late to the party I guessā€¦

I have not read this whole thread but only the intro postā€¦ thanks for noticing me! I am working on a prototype implementation of my ideas using Rust and GTK for the GUI. Nothing is there yet, as Iā€™m still figuring out the details of the protocol and how to effectively use IPLD rather than putting links in JSON and traversing ā€œby handā€.

51

do you have a gitlab repo or github repository about that ?

52

Not yet because I want to figure out some stuff before publishing. But I will publish as soon as I have a working minimal POC!

53

Following is a summary of a potential method to combine social media with IPFS, based on current state of the technology, that I posted on Gab at:

Iā€™m exploring creation of alt-tech, in the form of an open source, resilient, easy to install, peer-to-peer chat and file management system, targeted for use in oppressive societies, and looking for a group discussion on the merits of such a project.

The current concept being explored uses an open-source browser, modified to have a chat sidebar feature. The user would install the following apps:

  • Browser with chat feature and WebRTC built-in
  • IPFS Companion extension in the browser
  • IPFS Desktop app

IPFS would be used for the following:

  • Distributed chat/file system peer server
  • Resilient infrastructure
  • Distributed file management
  • Signaling protocol for WebRTC

When the chat feature is launched in the browser, the browser would talk to the IPFS daemon to identify available peers that also support the chat feature. The user could then try to initiate chat with one or more suitable peers, where the data streaming for the chat would occur via WebRTC

The chat UI could be designed to do some of the following to aid research/collaboration:

  • Allow linkage to IPFS file resources
  • Normal web URL hyperlinks
  • Establish chat groups
  • Identify trusted chat peers

I have a Google shared doc started for project specs development

4 Likes
54

I am at the ā€œthinking about itā€ stage for developing a WebRTC based broadcasting 1toMany application using IPFS for distribution. Although Iā€™ve played around quite a bit with Muaz Kahnā€™s RTCMulticonnection code, the IPFS portion of this idea will be a challenge for me. So Iā€™m hoping I can learn something from your development using WebRTC. Please keep me in mind.

2 Likes
55

@crockwave good job :+1:

1 Like
56

For the IPFS/WebRTC chat app, I have the chat UX requirements roughly identified.

A test of using IPFS-JS for WebRTC signaling has been previously created, but they ran into difficulties doing NAT traversal:
https://github.com/cretz/webrtc-ipfs-signaling

IPFS Issue 153 acknowledges the NAT traversal issue
https://github.com/ipfs/in-web-browsers/issues/153

For reference, here is info on WebRTC signaling protocol requirements https://www.html5rocks.com/en/tutorials/webrtc/infrastructure/

A working title for the project is IPSN (Inter-Planetary Social Network).

IPSN app would do the following:

  1. Perform normal IPFS daemon functions and data caching
  2. Identify other IPFS nodes that support IPSN (using the Identify message)
  3. Perform signaling to selected IPFS nodes in order to build an appropriate session description to be handed off to WebRTC for making and managing the datastream connection with the remote peer
  4. Perform handling of WebRTC datastream errors

An alternative to the forked browser concept would be to do the proof-of-concept in a Windows app, using IPFS-JS lib, to avoid the browser forking and to avoid install of the IPFS desktop app. The assumption is that deployment would be simplified if we can incorporate js-IPFS into the app. Issues currently include:

  1. Getting NAT traversal fully implemented into js-IPFS, so that it can pass the appropriate session description to WebRTC for P2P datastream connection
  2. Getting IPFS daemon fully operational in js-IPFS
  3. Allowing Identify message metadata such as IPSN protocol support, avatar, and user name.

In order to populate a Contacts list in the IPSN app, each IPSN app would need to include metadata with their IPFS client, passed in the Identify message, so that the other IPFS clients deployed as IPSN nodes would gracefully add that particular IPFS node into the IPSN contacts list. The UX would be much better if an IPSN app could create a user name, which appears in the remote IPSN client chat contact list.

Thatā€™s the basic concept, single app that performs both IPFS functions and P2P chat. The exact approach is yet to be determined.

On Issue 3, is the IPFS Identify message overridable, such that once an IPSN protocol capability is detected, an Identify/push variant can be used to push extra metadata to compatible IPFS client nodes, containing custom metadata, such as avatar, user name, etc.?

Iā€™m also looking for opinions/comments on best approach for a project with regard to app structure and which libraries to use.

1 Like
57

Hello crockwave and all,

i discover recently this project : Secure Scuttlebutt - Scuttlebot and i found this approach very intresting you can test that with heavy client write into javascript and electron patchwork GitHub - ssbc/patchwork: A decentralized messaging and sharing app built on top of Secure Scuttlebutt (SSB). and here you can found all docs.

If i understand correctly your approach some mechanism of that can help you on your project.

thatā€™s depend your approach :

for exemple if you choose to store your project directly in IPFS (clientless) youd need a basic application html 5 / js (with react / js-ipfs / lib to manage cryptography ) can be sufficent

but if you wish store all localy on user space maybe a heavy client rust / go / html5 + js (electron) can be a good path.

Regards

1 Like
58

4 posts were split to a new topic: Off-Topic Discussion from ā€œSocial Media Architecture with IPFSā€

59

Iā€™ve developed a fairly feature-rich platform that I want to become ā€œThe Linux of Social Mediaā€, meaning itā€™s an open-source platform anyone can use and/or build on and extend as part of the IPFS ecosystem.

I wonā€™t try to describe it because it really takes the screencasts in the links below to do it justice. Iā€™m looking for other Java + TypeScript developers who want to join me in working on it. I donā€™t have any funding. This has been a side project up until now.

https://quanta.wiki

3 Likes
60

I like the look and feel of the platform. Thatā€™s a lot of very good work.

However, I donā€™t use Mongo DBā€¦ nor most of the rest of the software stack. I noticed that your sample domain is running on Linodeā€¦ of which I am a big fan :slight_smile:

My go-to DB is postgresql. However, Iā€™ve been trying out OrbitDB which runs within IPFS as indicated throughout several other threads here.

The features I like a lot:

  • Markdown enabled
  • Intuitive editor
  • Clean layout
  • Drag and drop

The architecture aspects that prevent me from deploying the platform on my own domains:

  • Significant Non-IPFS components
  • Use of DB that Iā€™m unfamiliar with
  • Use of embedded web server which requires me to modify my iptables and forward ports.
61

Quanta is Java + TypeScript, so it would only be interesting to those kinds of developers, for sure. When I researched platforms a couple of years ago I was unable to find any Java-based platforms similar to Quanta. At least now Quanta is out there and the Java world has a platform option (if I can say that humbly)

About the editor, I have Ace Editor in the codebase but I temporarily commented that out just to get the default spell-checking from the browser.

About the data store, thereā€™s a persistence interface that could be made a pluggable API, so that other back-ends could be used, an would be a 2 to 4 week effort to complete Iā€™d say. I wish I had funding for that. :slight_smile:

Regarding web server, it actually just ā€˜requiresā€™ a Servlet Engine, so getting it out of the embedded Tomcat that Springboot uses is not really much coding, just configuration stuff.

1 Like