IPNS - beyond the basics - no IPNS pinning service? Any docs on this?

Help
1

The “Hello IPFS” exercise with IPNS from the docs works great. Is there more IPNS documentation someplace?

While something like pinata can pin IPFS hashes, there apparently is no service that can “pin” IPNS records.

After a little further investigation, it seems there is a lot of IPNS understanding/documentation that I’m missing.

E.g. I discover things like --lifetime that implies that republishing is required and that there is a difference between PubSub and DHT operation of IPNS. Neither “lifetime” nor “PubSub” are mentioned in these docs, and “republish” is only mentioned in the context of changing content.

Will a node republish automatically (as implied by the Ipns.RepublishPeriod config setting) or will I need a cron job for that?

In addition, @adin posted:

[…] the fact that publishing IPNS records from more than one node is unsupported behavior and so occasionally runs into weird corner cases.

So IPNS can only publish from one node? Where is this documented? If only one node can publish IPNS records, how do I avoid that single node from being a single-point-of-failure?

I then see that in cloudflare’s IPFS gateway guide IPNS is not mentioned at all. Only dnslink. I’ve also read rumblings here and there that IPNS is best avoided, but not clearly why.

Should IPNS even be used in to 2022 for “dynamic” content? What are the ideal use cases for IPNS and where is it best avoided? Where/when should dnslink and Ethereum naming service (ENS) be used instead of IPNS and when used in addition to IPNS?

Background

I’d like to create a web-based CMS based on IPFS. I was thinking of running IPFS in the browser and creating an IPNS key and storing it in localStorage with a dialog to export/import the key so it can be stored in a password manager or browser extension.

And then use in-browser markdown to create and edit posts, and publish the edited site in IPNS directly from the browser when I’m done.

Edited to add: Using IPFS in the browser should remove the need for having any backend at all.

But for this to work (or to abandon the idea for good reasons), I’d like a deeper understanding of IPNS than what I can read from the docs.

Especially if only one node can publish using a particular key, then I can’t use IPNS publishing from the browser, since that IPFS node will be ephemeral and I’ll want some other IPNS to do the republishing.

2 Likes
2

no IPNS pinning service?

Because no one made one yet.
This is a feature we want, no one got around doing it yet.
web3.storage works on a similar feature I belive.

Will a node republish automatically (as implied by the Ipns.RepublishPeriod config setting) or will I need a cron job for that?

Your node will republish automatically as long as it runs.

In addition, @adin posted:

[…] the fact that publishing IPNS records from more than one node is unsupported behavior and so occasionally runs into weird corner cases.

The main issue is that if your have multiple nodes publishing IPNS records they will fight over themself and have nonce colisions.

IPNS records have a “Nonce: Natural-Number” field. In theory it is simple, each new record is the previous one plus something non 0.
The DHT have an issue, it is likely there can be multiple state of an IPNS in flight at any time.
Let’s assume you found two records:

Key: k111
Path: Qmfoo
Nonce: 100

Key: k111
Path: Qmbar
Nonce: 200

Both one have the same key k111 however they point to different CIDs, Qmfoo and Qmbar.
Then you just lookup the nonce and the biggest one is said to be valid, so in this case Qmbar is what IPFS will show you.

However this nonce counter is local to a node.
If you have two nodes publishing under the same key, the first one can publish using nonce 100 and the second one, unaware that the first one already used 100, will also use nonce 100.
Giving you this situation:

Key: k111
Path: Qmfoo
Nonce: 100

Key: k111
Path: Qmbar
Nonce: 100

And then you are fucked, because you can’t know which one is the newest one.
I don’t know what IPFS does in this case, there are three possibilities:

  • The second node DHT publish fails because the DHT nodes hosting the record knows they already have a record with nonce.
  • The second node DHT publish succeed because due to usual DHT churn some nodes doesn’t know about the first record and thus accepts the second record.
    • Then while searching it fails.
    • Or while searching IPFS returns the first one it finds (making it essentially random as which one you will land depend on PING and DHT randomness).

I then see that in cloudflare’s IPFS gateway guide IPNS is not mentioned at all. Only dnslink. I’ve also read rumblings here and there that IPNS is best avoided, but not clearly why.

IPNS doesn’t work well, it require a single node to stay online and republish else they go offline.
They have bad or atrocious propagation time if you don’t use pubsub.

For now, IPNS is more a beta / alpha feature, dnslink work.

While something like pinata can pin IPFS hashes, there apparently is no service that can “pin” IPNS records.

That a whole other question.
So there is an ambiguity when we talk about “IPNS pinning”.
To me and the way we use this in IPFS issues / discussion, mean “pinning and republishing the IPNS record, it is a 300Bytes ~ 3KiB piece of data, that give you the public key, the nonce, the expiry and the path you need to follow”.

It seems the feature you ask is more “IPNS following”, where pinata would download new content and pin it when they see a new IPNS key.
From a pinning point of view thoses are really different.
IPNS following would require pinata to periodically check the key, and maybe download lots of data often when you update your key.

This feature can be done with a cron job and a 5 lines bash script, it is then less priority, because republishing of IPNS record is something you can’t fix outside of IPFS (unlike follow).

I’d like to create a web-based CMS based on IPFS. I was thinking of running IPFS in the browser and creating an IPNS key and storing it in localStorage with a dialog to export/import the key so it can be stored in a password manager or browser extension.

Seems a perfectly good idea to me.
But since IPFS in browsers have very limited (essentially no) access to the IPFS network.
You would need a server somewhere (a local IPFS node, web3.storage, …) to relay your IPNS records, that something we would like to able to do, just no one coded it yet.

1 Like
3

If you’re wanting to code in Java (back end) SpringBoot project for this new CMS let me know because we can collaborate. I have a lot done toward this goal.

4

I’m always a little confused when people refer to a possible “IPFS pinning service”. What functionality are people referring to? Being able to pin the content pointed to by an IPNS record or republishing IPNS records?

5

I think people are looking for an app or service similar to cloud backup services, image sharing apps, file sharing apps, etc.

They don’t necessarily want to install their own gateway and may not even be a developer. For an end user to “use IPFS” and be assured their stuff is stored on IPFS with IPNS etc, they need some service to do it for them… Sadly there are VERY few such services. Quanta (my little hobby-horse), having it’s own imperfections (haha) is the only “open source” attempt at such a system, which would make it easy for people to run their own instance OR build commercial cloud infrastructure on.

The Quanta demo instance (quanta dot wiki) isn’t usable as a commercial site right now either, but my “idea” is definitely “correct”. I’m doing what needs to be done. I’m just not commercializing it, because I want it open source instead of proprietary.

1 Like
6

First of all my no-backend CMS apparently isn’t possible yet, because:

IPFS in browsers have very limited (essentially no) access to the IPFS network.

Yeah, that is an issue that brings an immediate halt to this project. Bummer :frowning:

:slight_smile: You’re right. There is.

In my future CMS, there will be a Submit button somewhere. In its onClick event handler call-back, I’ll need to do the equivalent of:

  1. ipfs add -r website/
  2. ipfs name publish /ipfs/$hash

Whether the phrase “IPNS pinning” includes item 1 or not doesn’t really matter to me, since I already know how to accomplish item 1. For what its worth I thought of it as only meaning item 2.

But I’d also want it to be a remote service so that the js-ipfs node can terminate shortly after 2 has been completed, so that the user can close the web browser.

But it all doesn’t matter if I have to have a backend anyway. I was hoping that js-ipfs was all the backend I was going to need.

7

In case that wasn’t clear, you can, you just need to relay with a local IPFS node or a pinning service.

8

:slight_smile: Thanks, yes I get that. As I pointed out in the OP the idea was to create a CMS editor that doesn’t need any backend other than js-ipfs. So no local IPFS node. No cron/bash script. Nothing.

I can use a pinning service for the IPFS part when the user (me) changes the website, e.g. via the pinata API. But as we’ve covered, there is no existing “pinning” service for IPNS that will do the required periodic republishing.

So now a “backend” IPFS node is needed to IPNS publish that new cid so that https://www.mysite.tld shows the new content. I can’t even as an alternative let the CMS hold an API key for my DNS provider so it can change the _dnslink TXT record because of same origin policy. (I’m guessing no DNS provider in their right mind would allowing modifying DNS using CORS.) And then I’d (again) need a backend. → Abandon project.

The whole idea was to try to make a CMS (or other dynamic website) that was entirely without a backend. If I have to have a backend process running, there is almost no point in me using IPFS. Then I might as well use a traditional rails, express, SpringBoot, or whatever backend.

Now that I think a little more about it, a pinning service, such as pinata, is also a “backend”. So however one looks at it, there needs to be a backend even if/when js-ipfs fully supports interaction with the IPFS network. To pin IPFS and to periodically republish IPNS. It sure would help if there was a service that could do the periodic IPNS republishing though.

Thanks for your contributions. I understand the problem domain much better now.

3 Likes
9

Also, is any of this documented anywhere other than the relatively short IPNS documentation?

10

Be sure you’ve seen OrbitDB, and also, unrelated to IPFS there’s WebTorrent which I’ve used successfully and is pretty cool too.

11

I don’t belive so, the issue is that this is implementation details, any documentation about this will get wrong quickly if anyone just make a PR fixing thoses issues.

12

Im trying to work on this problem. I am implementing it in Go. And I need lots of eyes on it to review if it is how people want it and is the right way to implement it. It is an Open Source project and my plan is to make it both republishing and IPNS following. Here it is: IPNSGoServer

Im actually currently reworking the README to cover more information on why it is important to have. AND the limitations and problems with IPNS. So this post really helped!

13

I found this to be a very informative discussion on IPNS Understanding guarantees of the improved IPNS over pubsub

1 Like
14

Hey, Adrian from https://kubelt.com. We’re building the CMS on top of IPFS. We’re live with a couple customers in private beta and are almost ready to launch with some open source tools as services.

Jump on our Matrix channel if you want early access :muscle:

1 Like
15

Looks interesting. Is the entire Kubelt tech-stack open-source, or are there some proprietary or closed-source components in it?

16

We’re open sourcing all the functional code (SDK, plugins into other CMS like Sanity.io, our own UI and more) in a clojurescript mono repo.

How it works in a nutshell is, we take a CQRS-like pattern to content management and content delivery (reads over IPFS, writes locally and over a gateway for non-functional services) with a user-centric/wallet model.

We’re also hiring!

1 Like
17

We are running a free IPNS pinning service, its mainly used in the .eth websites ecosystem
see here: https://dwebservices.xyz

18

If one were willing to host such a service yourself, how to do it?

We have a use case where from an immutable container we need to find a single update to an object in IPFS which will be immutable, and never be modified or updated. And we need a way to find that update in a reasonable time.

So originally we thought we could:

  1. Create directory in IPFS
  2. Store data in IPFS
  3. Create IPNS keypair
  4. Store IPFS CID and IPNS public key into immutable container
  5. Do container magic and blockchain anchoring
  6. Store blockchain address & update based on container magic into directory created in step 1
  7. Update IPNS record to point to new CID of directory
  8. Forget IPNS key as it will never be needed again…

We’re ensuring persistence of the IPFS records over a few nodes.

All we would need is to also be able to ensure IPNS those addresses over these same nodes. But we could find no documentation on this, and no commands that seem related…

So can IPNS even deliver long term persistent addressing? How?

It is being discussed for web pages and the like, but given its apparently limited/short life time it does not seem it is actually suitable for that?

1 Like
19

I’m kind of curious how one would define an IPNS pinning service? What service is it envisioned to provide?

20

Permanence for IPNS names / mapping.

Right now, if there is an IPNS name/mapping defined and provided, it is lost in seconds to hours, apparently defined by its time to live. The originating IPFS node may refresh the mapping, but that makes IPNS fully dependent on that single node/cluster. Effectively, in the current approach, IPNS is somewhat more centralised and less redundant than DNS.

What confused me at first is that the use cases mentioned in the IPNS documentation include things like web pages. These usually have longer time horizons than seconds, minutes or hours. So it does not seem like IPNS is really useful for that. It’s therefore not quite clear to me what the actual use case of IPNS could be with that short a mapping period. Perhaps that is one of the reasons it does not seem to be considered a mainstream feature, and its use is apparently largely discouraged.

In essence:

For IPFS objects itself I can make sure any number of nodes keep a copy permanently or for a long period of time. For IPNS there is no such thing to preserve the mapping of /ipns/ to /ipfs/.

Since the concept of providing long-term availability for IPFS data is called “pinning”, it seems intuitive that the equivalent would be “IPNS pinning.”