Using pubsub as a persistent database

Help
1

I’ve looked into pubsub and the base aspects of how it’s supposed to work. With plans to make a platform that runs entirely within IPFS, it definitely looks like what I’ll want to go with for things like chat and notifications and search! However there is one fundamental problem, which I don’t fully understand yet and depending on the answer might throw a wrench in my idea of using this system.

Can pubsub messages be persisted and ultimately used as a database? Or are they sent in a similar fashion to TCP packets, meaning they’re only sent once and lost if the receiver isn’t currently online? If it’s the later scenario, is there some alternative to have a persistent flexible database within the network?

My plans definitely involve using some sort of database, which is where IPFS makes things tricky as you can’t use stuff like mysql the way you can on a conventional web server. Pubsub looks like it would perfectly do the trick if used intelligently, but it’s useless for this if it’s not a persistent database. I’m hoping I’d be able to use the JS API to dynamically generate a channel for each user and site component that needs it, from which information (stored as messages) can be retrieved at any given moment.

2

You could make each message you pub the has of an ipfs object referencing all previous things and also the new changes – thus allowing for message history, etc

3

I can’t quite see how that would work. If a pubsub message is only received if the receiving computer is online and fails to deliver otherwise, the system wouldn’t work because the receiver has to be connected in order to register anything. The machine sending the message could of course also log it in a file, but that defeats the purpose of using pubsub altogether as the receiving machine can simply read that file later.

4

Yeah, you would either have to paid for the next pub, or update an IPNS entry on every pub and fetch that on first connect to get the history while waiting for the next pub.

5

You should look into orbit-db. It does precisely what you are after: persisting data via ipfs, sending updates via pubsub, and syncing/merging/conflict resolving by means of CRDTs.


orbitdb on freenode

3 Likes
6

Sounds interesting, thanks. Could be very good for inspiration at least. I was hoping pubsub could have a persistent mode by default, but I understand that’s not how it was intended to be used.