IPNS - beyond the basics - no IPNS pinning service? Any docs on this?

In case that wasn’t clear, you can, you just need to relay with a local IPFS node or a pinning service.

:slight_smile: Thanks, yes I get that. As I pointed out in the OP the idea was to create a CMS editor that doesn’t need any backend other than js-ipfs. So no local IPFS node. No cron/bash script. Nothing.

I can use a pinning service for the IPFS part when the user (me) changes the website, e.g. via the pinata API. But as we’ve covered, there is no existing “pinning” service for IPNS that will do the required periodic republishing.

So now a “backend” IPFS node is needed to IPNS publish that new cid so that https://www.mysite.tld shows the new content. I can’t even as an alternative let the CMS hold an API key for my DNS provider so it can change the _dnslink TXT record because of same origin policy. (I’m guessing no DNS provider in their right mind would allowing modifying DNS using CORS.) And then I’d (again) need a backend. → Abandon project.

The whole idea was to try to make a CMS (or other dynamic website) that was entirely without a backend. If I have to have a backend process running, there is almost no point in me using IPFS. Then I might as well use a traditional rails, express, SpringBoot, or whatever backend.

Now that I think a little more about it, a pinning service, such as pinata, is also a “backend”. So however one looks at it, there needs to be a backend even if/when js-ipfs fully supports interaction with the IPFS network. To pin IPFS and to periodically republish IPNS. It sure would help if there was a service that could do the periodic IPNS republishing though.

Thanks for your contributions. I understand the problem domain much better now.

2 Likes

Also, is any of this documented anywhere other than the relatively short IPNS documentation?

Be sure you’ve seen OrbitDB, and also, unrelated to IPFS there’s WebTorrent which I’ve used successfully and is pretty cool too.

I don’t belive so, the issue is that this is implementation details, any documentation about this will get wrong quickly if anyone just make a PR fixing thoses issues.

Im trying to work on this problem. I am implementing it in Go. And I need lots of eyes on it to review if it is how people want it and is the right way to implement it. It is an Open Source project and my plan is to make it both republishing and IPNS following. Here it is: IPNSGoServer

Im actually currently reworking the README to cover more information on why it is important to have. AND the limitations and problems with IPNS. So this post really helped!

I found this to be a very informative discussion on IPNS Understanding guarantees of the improved IPNS over pubsub

1 Like

Hey, Adrian from https://kubelt.com. We’re building the CMS on top of IPFS. We’re live with a couple customers in private beta and are almost ready to launch with some open source tools as services.

Jump on our Matrix channel if you want early access :muscle:

1 Like

Looks interesting. Is the entire Kubelt tech-stack open-source, or are there some proprietary or closed-source components in it?

We’re open sourcing all the functional code (SDK, plugins into other CMS like Sanity.io, our own UI and more) in a clojurescript mono repo.

How it works in a nutshell is, we take a CQRS-like pattern to content management and content delivery (reads over IPFS, writes locally and over a gateway for non-functional services) with a user-centric/wallet model.

We’re also hiring!

1 Like

We are running a free IPNS pinning service, its mainly used in the .eth websites ecosystem
see here: https://dwebservices.xyz

If one were willing to host such a service yourself, how to do it?

We have a use case where from an immutable container we need to find a single update to an object in IPFS which will be immutable, and never be modified or updated. And we need a way to find that update in a reasonable time.

So originally we thought we could:

  1. Create directory in IPFS
  2. Store data in IPFS
  3. Create IPNS keypair
  4. Store IPFS CID and IPNS public key into immutable container
  5. Do container magic and blockchain anchoring
  6. Store blockchain address & update based on container magic into directory created in step 1
  7. Update IPNS record to point to new CID of directory
  8. Forget IPNS key as it will never be needed again…

We’re ensuring persistence of the IPFS records over a few nodes.

All we would need is to also be able to ensure IPNS those addresses over these same nodes. But we could find no documentation on this, and no commands that seem related…

So can IPNS even deliver long term persistent addressing? How?

It is being discussed for web pages and the like, but given its apparently limited/short life time it does not seem it is actually suitable for that?

I’m kind of curious how one would define an IPNS pinning service? What service is it envisioned to provide?

Permanence for IPNS names / mapping.

Right now, if there is an IPNS name/mapping defined and provided, it is lost in seconds to hours, apparently defined by its time to live. The originating IPFS node may refresh the mapping, but that makes IPNS fully dependent on that single node/cluster. Effectively, in the current approach, IPNS is somewhat more centralised and less redundant than DNS.

What confused me at first is that the use cases mentioned in the IPNS documentation include things like web pages. These usually have longer time horizons than seconds, minutes or hours. So it does not seem like IPNS is really useful for that. It’s therefore not quite clear to me what the actual use case of IPNS could be with that short a mapping period. Perhaps that is one of the reasons it does not seem to be considered a mainstream feature, and its use is apparently largely discouraged.

In essence:

For IPFS objects itself I can make sure any number of nodes keep a copy permanently or for a long period of time. For IPNS there is no such thing to preserve the mapping of /ipns/ to /ipfs/.

Since the concept of providing long-term availability for IPFS data is called “pinning”, it seems intuitive that the equivalent would be “IPNS pinning.”

There seems to be some confusion in general about IPN. You might find this recent thread helpful How do I make my IPNS records live longer? - #10 by danieln

I find the use of the term pinning for IPNS confusing since there already is a term pinning that does something different.

I believe IPNS over the DHT is discouraged but pubsub works quite well.

It seems like there are a lot of separate things being proposed all under the same heading of “better IPNS”

1 Like

Does it, though?

I’m not talking about the underlying technical details. Those are (and should be) largely irrelevant to the people using a solution. In the “IPFS space”, the concept of “pinning” is basically to make data permanent either via a third party service and/or your own node, at a level of redundancy that you get to decide by choosing the right pinning service and/or node / network of nodes.

The underlying problem people are struggling with when talking about IPNS pinning is just that: Make mapping permanent, independent of the local node that you used to upload/generate the mapping.

So one is permanence for IPFS, the other is permanence for IPNS.

I get that those would be achieved in very different ways.

But conceptually, they don’t seem so different to me.

So how does one “switch” to pubsub? And if one does so, does DHT IPNS no longer work? I’ve struggled in the past to find documentation that would explain what it would mean to have “IPNS over pubsub” vs “IPNS over DHT”, to which extent they are perhaps mutually exclusive, and how to configure them.

Thanks for that - it is indeed helpful.

But participants there also seem to struggle with understanding configuration and persistence.

While I definitely have not understood everything perfectly yet, to me DHT vs pubvsub just seems like different distribution / dissemination mechanisms - and therefore orthogonal to the question of permanence (also referred to as “pinning” by many other posters in this forum).

And as for the time frames involved: People talk about hours or days. But for IPNS to truly be useful we’d probably have to be in the range of years, perhaps decades, as mapping gets more important the longer lived information really is. And there is no guarantee that original node will be around for that long.

The main problem is that IPNS keys live with the node that create them ATM.

IPNS records are just crypto signed CIDs with extra steps. The keys are what matters. No keys no update.

As for persisting the records themselves, right now IPNS via pubsub with very long lifetime is what you want.

You might also want to save the records offline too.

1 Like

And as for the time frames involved: People talk about hours or days. But for IPNS to truly be useful we’d probably have to be in the range of years, perhaps decades, as mapping gets more important the longer lived information really is. And there is no guarantee that original node will be around for that long.

How exactly are such long-lived IPNS names (years/decades) useful for permanence? It’s mutable, there’s no guarantee of permanence, I think that’s the point, it was created to emulate the same old problems. If your goal is permanence you should be using IPFS instead.

And does “information” really change?

For example, if I measured the outside temperature to be X°C right now, tomorrow it will still be true that I measured X°C right now. Let’s say I publish this information through IPNS. The IPNS content would more or less represent the “current” outside temperature, not the temperature I measured right now. If someone from outside wanted to link to the temperature of YYYY-MM-DD HH:MM, they should use the corresponding IPFS URI, never the IPNS name.

Another example: if a friend publishes their site with an IPNS name and I want to link to it from a post of my own, I don’t see any reason to use the IPNS URI other than the (slim) hope that they might significantly update the post in the future. For permanence, I should use the IPFS URI instead (and pin it myself too).

By using IPNS you’re putting the “responsibility” of permanence on the other side. You have to cross your fingers and hope that they keep all their URIs up and running for eternity. This is exactly why IPFS was created.

EDIT: otherwise I agree with the IPNS related usability concerns previously mentioned in this thread.

Yes

Nothing is permanent. I think a better definition of pinning is simply to protect a resource from GC. The term is slightly overloaded now that there is a remote pinning service api but not overly so in my opinion since it’s exactly what it says “remote”. It’s just pinning but on someone else’s node.

Putting the technical details aside, I’m still not sure what you’re proposing with IPNS pinning other than making it permanent.

Permanent is immutable which is the opposite of what IPNS is unless you’re thinking of some other idea of permanence like availability.

There are a couple of experimental config options to enable it kubo/experimental-features.md at master · ipfs/kubo · GitHub

It doesn’t disable DHT publishing but augments it. I don’t know the exact details but I believe that it will initially use both and then updates are made via pubsub.

It does seem to be a source of a great deal of confusion. It’s unfortunate because I’ve heard IPNS over DHT to be basically unusable but IPNS pubsub works quite well but is still listed as experimental. This causes the belief that IPNS doesn’t work to persist. This probably hurts adoption since the initial impression is a system that is fundamentally broken.

As far as I can tell the problem is that DHT nodes stop reproving after 12 hrs regardless of what the ttl or lifetime is set for so it needs to be republished which can be done by a node that doesn’t have the private key but isn’t currently being done. I think what people are referring to as IPNS pinning is marking an IPNS record as republish able on a non private key containing node but I could be totally wrong.

What I’d rather see the word pinning applied to IPNS is to pin an IPNS record would be to pin the underlying IPFS record and any subsequent IPFS record after update. ie. pin this any any thing it might refer to in the future.

What would be nice is a pointer in an IPNS record to an IPFS snapshot of the previous record so you could chain back to all previous names.

EDIT: It looks like there were a couple of additional posts to that thread that clarified what I found to be confusing in regards to a possible “pinning service”. The difference between re- provide, 1st and 3ed party republishing. (still lacks a name. Having to describe it as 1st and 3ed party republishing, while accurate, is an awkward way to refer to it)

You can try dWebServices which does IPNS pinning for free

1 Like