Hi there, I’m looking to ask for advice about IPNS.
As it stands, it seems IPNS can only be published from one kubo daemon, because of the internal counter on the record. I might also be misunderstanding that so please correct me if I’m wrong there.
I would like the IPNS record to stay available and reachable even if the machine hosting the daemon has to be taken down for backups, kubo updates, OS updates etc. If the machine fails entirely, then I’d also have to migrate to another machine, and in that interim I want to make sure this IPNS record stays available. Since I can’t publish from more than one machine, I can’t make sure it stays available even if an individual node goes down.
I was looking into DNSLink as another option as a mutable pointer to my CIDs, but I was hoping to stay within the IPFS ecosystem’s solutions.
Would you advise going with DNSLink if I want availability, or is it possible to reliably and easily publish the same IPNS record from multiple machines?
It’s the public key that matters. Only the node with the same public key can re-publish the IPNS key.
So, small example. When you publish IPNS it’s being send to X number of nodes. At the time of your publish it’s TTL is 22 hours or so? a day? 2 days? Don’t know the exact TTL but something fairly short. During that time any node that has your IPNS key works to resolve it because it’s time hasn’t expired yet.
To “renew” the time you re-publish (happens automatically) and for that the node who did the publish must be online to re-publish. If your node is not able to re-publish then, after that TTL time expires, your key won’t resolve anymore.
Yep, i want that too! Doesn’t work that way Your node must be online or your key will expire.
I had this issue too and have made many different solutions over the years. What i settled on is just a central key/value on some known site and query the latest CID from there.
What others have done (like @fabiobozzo with https://ipcm.dev/ is another way. You store the CID in a smart contract and query that contract. You simply update the key in your contract when you need to.
It all adds a bit of extra tooling to have “mutable” CIDs without IPNS.
Not sure for the DNSLink options. Isn’t that just moving the issue to your TXT record as you specify an IPNS CID there. If you can get away with just specifying an IPFS cid and keep that up to date in your DNSLink then i’d go for that! Or at least try if that is workable in your situation.
No. An IPNS record with a valid lifetime/validity can be published by anyone. You can also re-publish an IPNS record with the Delegated Routing API, like the public one at https://delegated-ipfs.dev/
When publishing an IPNS record to the DHT, it will typically be persisted by 20 peers (for 48 hours). So your node is not the only one hosting it.
DNSLink is well integrated into the IPFS ecosystem. We use it for most of the public IPFS sites, e.g. the IPFS Blog:
The main benefit of DNSLink is human-readable names and no special infrastructure (DNS is commodified and therefore cheap have hosted and update). The main disadvantage of DNSLink is that it lacks the self-certifying properties of IPNS.
That is confusing but you’re right.
A still valid record (aka, not expired) can be republished by anyone.
If it becomes expired (so after the 48 hours) the original node has to be online to refresh it. I’m fairly sure that even the delegated routing isn’t solving this either.
I’m guessing it isn’t strictly the original node that has to be online to refresh it but a node that has the signing private key. Are there any games you can play by sharing the ed25519 private key among a group of cooperating nodes?
The private key rules. The node with the private key that did the original publish needs to be there to refresh the key when the TTL expires.
If you move the private key around to multiple nodes then yes, multiple nodes can refresh it. I don’t know what happens when two nodes are both online that both can refresh the key (aka, both have the private key). It sounds like a recipe for disaster waiting to happen.
I figured it would be bad mojo to have multiple nodes with the same key but I’d be interested in what that disaster would look like. I asked something similar a while ago but I think it was about two nodes having the same PeerID.
The key point is that the validity (a timestamp derived from the lifetime parameter) of the IPNS record can be longer than how long DHT servers will keep a record for, i.e. record expiration (48 hours). The validity can be for a year, which allows for delegated republishing without giving away your private key (to a service or what not).
Indeed, the private key rules!
Also I build a little debugger for IPNS this week. You can use it to inspect and create records.
How do you do delegated republishing? Is there any service out there?
Say you do set the validity at a year (and the TTL at 48 hours). I’m assuming it still resolves after 48+ hours (as it’s still valid). Now what would happen if a new publish to a new CID is done after the TTL expires but before the validity expires? Same private key. Just a publish with new content.
In the case of point 2 i guess it depends entirely on what a peer finds first. If it finds a IPNS key which happens to be the old pre-ttl-refreshed one then it will happily feed you the old data. Another node that might find the refreshed one will serve you the new data. That is an accurate assumption of how it works, right?
Besides that there’s w3name and filebase but they control the key I think.
If it was stored on the DHT, i.e. on 20 peers. After 48 hours they will drop the record. So it won’t resolve, unless someone republishes it.
There’s a sequence field which functions as a nonce that is incremented and essentially invalidates older records. When you update CID, it will be automatically incremented.
This is why clients will typically try to resolve from more than once peer so they can be sure there isn’t a newer version.
TTL is like max-age or DNS TTL. It just tells clients how long to cache it for before trying to resolve.
So to circle back.
In general and the common case: It won’t resolve after the TTL is expired.
Options to make it work - without the original host being online - require custom configuration or even handing control out to an external party to do it for you as a service (and thereby re-introducing centralization, as they now control your name).
From here on it’s not specifically directed toward you, Daniel, more in general for the IPNS usecase.
I get why people would want to use IPNS (mutability). The idea in principle is great but in practice is so incredibly limiting that it’s limits make it useless. Use IPCM or use some other online key/value store and query that instead to do your mutable → immutable lookup. You have to make wrapper tools around it but you have to do that too if you persist in using IPNS anyhow. I’ve burned/wasted many weeks/months in the past getting IPNS to work just to discover that… well… the effort isn’t worth it. Don’t use IPNS, it’s not worth it! It’s a conclusion mostly everybody comes to after wasting so many hours trying to get it to work. I’d even go as far putting warnings in the docs advising people against using it unless in very specific usecases.
If you’re going to say something as strong as “Don’t use IPNS, it’s not worth it” can you at least provide a reason why? This really confuses what the current situation around IPNS really is. Did you have some use case that it didn’t fit? Is it really worthless for any use? Your alternative that you suggests using smart contracts which come with lots of problems as well.
It’s not worthless but it’s usefulness is very limited. Here are some cases where it’s useful:
If your node can be online 24/7. This implies non-home usage, aka your node in a VPS or dedicated server environment.
If you have a popular site/app. This implies a server and hence you satisfy point 1 above too.
You have a thorough understanding of key management. As an IPNS is tied to a key. A node can have multiple keys (and thus multiple IPNS records) but merely going this route is a mental challenge on it’s own. It does work and can be used to publish multiple “sites”* from one node.
In theory IPNS is super useful and awesome! In practice it was very bad, it became a little better but it’s still just not very usable. At least not for the “dev who plays with tech”. It’s use is mutability through a non-changing entrypoint (the IPNS CID) and having that all in-house (as in within the IPFS protocols) gives you - the user - a neat way to have mutability.
In the past there have been many many many many many many many many issues around IPNS. Questions here, on slack and as bug reports about why their IPNS CID doesn’t resolve. That it takes (or took, that’s fixed i think) forever to even publish an IPNS record. Not being able to resolve it even when connected to the very node that has the data right there. The problems surrounding this amazing piece if tech are - in my opinion - not worth it anymore. You want mutability, use something else that gives you that mutability. The smart contract approach is just one option. The DNS approach is another option with pkarr perhaps being the next evolution of it.
It all boils down to tooling. If your use is command line based then your tooling might as well be a dig call to a txt record somewhere. Use your DNS as key/value store. Wherever you did ipfs name publish, replace it with a tool that updates your key/value lookup. Could be your DNS, could be a smart contract, you have many options here. If your target is a website, things become much harder. For a browser i’d assume we’re talking about a site. What you’d need there is a static site (so it too can be hosted through IPFS), it would do the key/value lookup for you and load the site. This can be as simple as a static site looking at a query argument (ex: http://<cid>/?key=smart_contract_address) where it internally would fetch the getMapping value from that contract and reload itself with that new cid it got from that contract. You can also omit the query argument and just always let your static entrypoint load the smart contract value. This is all pure static code and probably** works much better and more stable then IPNS anyhow! And no ttl at all! When you update your value in the key/value store of choice, your site is updated instantly.
Please do tell me @zacharywhitley, are you actually using IPNS? And if so, how?
* I’m calling it sites just for the ease of mindset here. It can be anything, obviously.
** Well, there’s always CORS that can ruin things massively…
An IPNS name fail to resolve after all the nodes holding it drop the record. For the DHT that expiration period is 48 hours.
The TTL and Validity fields in the IPNS record are independent of the expiration.
You haven’t clearly stated what exactly is wrong. A valid signed IPNS record is self-certifying and can be validated by anyone holding the record. By delegating the publishing of the record to another entity, you are not centralising it in any way. You are just delegating the work, the same way that using a pinning service is not necessarily centralising anything.
When you use delegated routing to re-publish a record whose key only you hold, you are not giving away the key.
How can we improve IPNS
I want to make this discussion productive. I understand you had problems with it, perhaps rooted in a false, misleading, and outdated docs.
That’s why I created this inspector.
If you have any other suggestions, please let me know. My suspicion is that most of what you encountered is a DX and DHT implementation problem.
@zacharywhitley I suggest reading the following (there’s a permalink to the important section) for some of he trade-offs between IPNS, DNSLink and something like ENS (or any smart contract solution).
Is that mix of time-implied words not making your head go nuts? TTL, Validity and Expiration, great… Can it be any more complicated? Anyhow. I might be wrong on the technical “it should work” level, but i’m fairly sure i’m not wrong in “real life how it behaves”. And that equates to: TTL expired = no resolve.
To replicate:
Spin up an IPFS node.
ipfs name publish something, but remember it’s CID.
shut down this IPFS node
Resolve works during TTL time (so for 48 hours it “seems” like all is fine)
After TLL expires (aka, 48 hours + 1 nanosecond): No more resolve
I am 99% sure this used to be how IPNS worked. I would’ve tried this locally if kubo didn’t kill my modem in a couple minutes (another story for not another time) and i don’t feel like spinning up a VPS to try this.
It could be that lookup semantics changed between me last actively using it a year or two ago and now. I feel like i’ve been very up to date on developments in this regard and nothing, to my knowledge, has substantially changed. Sure, the TTL went up, there’s a better strategy for publishing and more improvements over the years so i don’t rule out me missing something.
Skipping over the cryptography part. I get the public/private/signing stuff (as an aside, i made an end-to-end encrypted own media player and storage, yes, i know how this works)
But this one i’m not so sure:
Can you confirm this for me please. Does delegated routing only re-publish (aka, just keep the record on the DHT on your behalf, no changes) or does it also update the TTL on your behalf.
Note i’m saying this still in the assumption that an expired TTL = no IPNS resolve anymore!
If delegated routing updates the TTL (thus the ipns resolve works even after the initial ipfs name publish + it’s default 48h TTL then - and only then - i get why delegated routing is useful in an IPNS context. If it doesn’t do this then i don’t see how delegated routing helps for an IPNS specific case.
You asked
This was my setup.
Node A (VPS somewhere)
Node B (my home, node running on my laptop)
Node C (someone else’s home, effectively unused)
Site with DNSLink to the published IPNS CID, going to the site was essentially on the gateway of node A.
I was developing a site which i name publish on B. A script on A and C was run run - through ssh - to force these nodes to connect to B and resolve the site. The script also made sure each node had the exact same data (essentially rsync) and it was added to IPFS. B and C had the DHT disabled (modem crash on both even with different IPSs) and were using peering to make sure A, B and C can all reach each other.
I was essentially using IPFS through the Node A gateway all the time unless i wanted to update my IPNS with new content, then i enabled my DHT on node B, did the publish and disabled the DHT again. This, while a hassle, worked fine as long as i was working on my site and updated my site within the TTL expiry time (22h back then i think?). At some point you’re done on a site and you don’t update it anymore, that’s when i got the problem of it not being able to resolve. The cause back then? Node B needed to reprovide within the TTL but wasn’t because well… crash… (the irony, B had the latest data, just unreachable through IPNS) I “thought” being smart by moving the IPNS publish to Node A thinking it’s on a VPS, always online (DNSLink changed to new CID, etc…), so all is happy, right! Right? … I still occasionally had the issue of IPNS not being able to resolve. Why? As long as IPFS on A was responsive the site did indeed work and resolved just fine. But IPFS, to this very day*, occasionally has an issue where it’s CPU usage goes up to 100% and stays there, completely non responsive. In essence the node is dead and reprovides definitely don’t happen from it anymore. That again gave me a non resolving site. That was on any place i tried (Node B, C, random public gateway… None of it, just dead). All nodes could just happily resolve my data as IPFS cid but not as IPNS. Later on i just kicked IPNS out and used the IPFS CID in DNSLink which had it’s own issue of updating the txt record…
Now i mostly noticed these non resolve issues a day, sometimes 2 days, after it last worked. By that, and a plethora of bug reports, i reasoned that the TTL value is the one determining factor here. Back then i did not have monitoring setup to see if my ipfs backed site is still online. So my experience of 1 or 2 days might well have been with a TTL of 1 hour instead of 48 (i think the defaults changed a couple times over the past years). What i do know is that all discussions back then with debugging this - mostly on discord - were pointing very firmly at the TTL. So yet again, the TTL is the key here. Either that or I’ve been wrong for years
* This 100% cpu issue occurred once a week or so in my early ipfs on VPS days. This later - till just a couple months ago - is heavily reduced to being rare. Say once every few months or so, but it does still happen!
Thanks for the link but I’m not sure why you thought I was confused about how they work or what the tradeoffs are. I’m still flabbergasted by the amount of bad information and confusion there is out there about the state of IPNS.
TTL: How long it should the record be cached before attempting resolution. This is similar to max-age in the cache control header or TTL in DNS.
Because IPNS is transport agnostic, it doesn’t dictate how you distribute IPNS it. When distributed using the DHT, it’s the DHT that imposes a 48 hour expiry. Unlike TTL, you can’t change the 48 hour record expiration period.
In this example what was the TTL set to? How do you attempt to resolve the name (if you shut down the node)? Using a gateway?
Your reproduction can be recreated with a TTL set to 1 minute, at the cost of resolution being a bit slower after the first time. The most important field is the lifetime, which defaults to 48 hours in Kubo. Meaning the record is valid for as long as nodes keep it around for.
I know that a little over a year ago Kubo started respecting the TTL field for IPNS records (https://github.com/ipfs/boxo/pull/459). Prior to that there was a bug whereby the TTL field wasn’t respected for caching purposes.
Delegated routing only re-publishes the record. It can’t update the TTL without the private key, as the TTL is signed. The idea is that you can set the lifetime of a record to however long you want to allow delegated republishing, and then republish the record (using a cron job, gh action or whatever) with an HTTP request to the delegated routing endpoint.