The TL;DR is that the PubSub API does not expose the required options to make an effective mesh (validators, algorithm tweaking, …) and that it make more sense for consumers to write the ~20 lines of Go code required to create a Libp2p node and run go-libp2p-pubsub on top of this node rather than implement a huge rube goldberg machine that expose the required callback over Kubo’s HTTP API.
We want to be sure we are not missing a usecase, so if you have any comments about our migration plan please raise them here or in the github issue. We also want to know what are the usecases you have if you are using the PubSub HTTP API.
Thx
I mean, I think the main use case is access to the pubsub API for projects that aren’t written in golang. It’s about having a programming-language-agnostic API to access pubsub functionality (via HTTP).
@SionoiS we are not gonna remove IPNS or IPNS over PubSub. We are gonna remove the low level PubSub API /api/v0/pubsub/*.
Sounds like you could move the IPFS part of your app from Kubo to your inside your app.
I belive apps become much better when you do that because you have a consistent type system and environment. This can be hard to do today, that why we are working on go-libipfs and go-libipfs’s example.
You could run a wasm or helia (/ js-libp2p) node that do webrtc & webtransport.
This used to be impossible due to mixed content issues and the lack of good browser to browser connections (webrtc-star ) but thoses issues were recently solved (/ or are close to be).
True, we are gonna provide a binary that provides the same features (you could also easily fork Kubo to revert the pubsub changes).
You will have to write a bit of Go if you want to access lower levels go-libp2p-pubsub options tho (go is really easy boring language so as long as this is small it should be fine ?).
Is this for real ? I’ve read the issue and understand the reasons, but wonder if there isn’t a compromise that could be reached ? The idea of using a websocket sounds pretty good.
I think there’s quite a lot of applications that use kubo’s pubsub HTTP API without major issues. Yes, message delivery isn’t perfect but the API is pretty straightforward and provides an easy communication system between nodes. I think that pubsub should be a core feature of IPFS, especially for the kubo implementation. But if pubsub support can be installed as a plugin it’s not so bad.
Websockets would make the reading of message pretty good which is not the problem that sparked all of this.
The main issue here is that a good pubsub mesh require to make decision on which messages are relayed, this is a thing Kubo hides from you with the SeenMessageCache which is a terrible solution (use lots of CPU, does not scale, is not resilient).
Vyzo wrote a new better default that would only accept increasing nonces and store them in a DB, but that somehow broke compat with js-ipfs (this is also not a silver bullet and only solve some particular problems), this breaks some behaviours, for example before if messages where out of order you would get both messages, now you only get the second one because the first one have a nonce that is higher blocking the first one to get through.
Ideally you users make the decision of how to validate message, because each app is slightly different.
This is implemented with callback, and doing callbacks over HTTP is not really a thing. We would need some complex machine, probably something where your app register an http handler endpoint and when the callback is called arguments are serialised and kubo do an HTTP call to your app. Other solutions are possible but they all comeback to kubo calling your app from time to time, which is not easy to write, or to use.
Turns out there this thing called the Go ABI, it already supports callback, it’s performant and somewhat intuitive to use. It’s also really easy to write (it’s just part of the language, you just pass a function pointer).
Free engineer time is at a premium, there is no good solution that makes everyone happy, so I think our time is better spent allowing users to make their own application aware choices and tradeoffs for this complex problem of message validation rather than maintaining something that always a bit broken, because something that always a bit broken is something that always generate new github issues.
If you are happy with the current solution there will be a binary the expose the same API but I hope most peoples wont use this because I don’t think this is a good implementation of pubsub.
) but thoses issues were recently solved (/ or are close to be).