IPNS use cases / what is dragging down IPNS?

Hi,

From my point of view, IPFS is really good at storing static content, but basically every application has a dynamic component. Being able to translate the hash of content to content is only useful when you already got that hash. But, you cannot refer to new content from old, because old content is immutable and you cannot point to the hash of the new one.

So, to me, the problem of fully decentralized applications can be summarized in:

  • Being able to download content from a link. IPFS resolves this.
  • Being able to link new content from old content. IPNS could resolve this?

For example, to create a decentralized “twitter” you basically need to:

  • Store the tweets: text, images, video…
  • Have a link to get your latest tweet (each tweet could point to the previous tweet hash).

So, my question is, what is dragging down IPNS?

I’ve read some bad things about IPNS and I couldn’t make it work myself (published with go-ipfs, tried to resolve with a gateway and with js-ipfs).

Anything you add to IPFS needs to be stored “somewhere”. IPNS itself is also a little tiny piece of data (called a record or an IPNS record), you are storing it yourself when you do ipfs name publish. Now for someone else to be able to resolve your IPNS hash, it needs to find a node anywhere that can give it that IPNS record.

Any node that accesses your IPNS record (and resolves it) becomes a “host” for it if you will.
There is a bit more to it then this, but in general this is about the gist of it.

Now to get an IPNS that is quick you want a given node to quickly be able to find another node that has your IPNS record. So you want your IPNS record to be stored by a lot of nodes.

You want a “pinning service” (a party that hosts your data on x number of IPFS nodes) for IPNS. And that, as far as i know, doesn’t exist yet.

The easiest way i know of is to spin up 1 or 2 nodes on a VPS service (vultr, digitalocean, …) and let those resolve your IPNS hash. It’s a bit of a manual step to get it working properly. Once you have a popular website and a lot of your users running IPFS you have a self sustaining system with regards to the IPNS record. By that i mean that your users, due to them using IPFS, keep the IPNS record alive. You can then drop the servers.

Be aware that an IPNS record has a lifetime.
Note this part of the IPFS documentation for ipfs name publish:

  -t, --lifetime   string - Time duration that the record will be valid for.
                            Default: 24h.
      This accepts durations such as
                            "300s", "1.5h" or "2h45m". Valid time units are

                            "ns", "us" (or "Âľs"), "ms", "s", "m", "h".

In simple terms this means that your IPNS record expires in 24 hours. This is not applicable to the node that did the actual ipfs name publish as that node is the author. It’s always is alive on that one. But for every other node that isn’t the author the lifetime plays a vital role. If the lifetime is expired, they go looking for you to get an update on that record. If your node is offline then your IPNS still can’t be resolved. And yes, they want that update from the author! So if you, for whatever reason, lose your IPFS keys then you lose the ability to update your IPNS key (with the hash staying the same). In that case your ipns name publish will give you a new unique hash. Just something to keep in mind.

Now i was told to see IPNS as a sort of name register (not a DNS) for IPFS. And with that logic in mind it makes sense to do an ipfs name publish with a --lifetime of something very long. I think 1 year works fine. That would be --lifetime 8760h (to supply it in hours).

Next, there is also the TTL option. As if it wasn’t confusing enough yet :slight_smile: For this i’m quoting @adin who explained this really clearly to me when i was having these IPNS issues.

As described in the spec https://github.com/ipfs/specs/blob/master/IPNS.md, the TTL recommends how long the record should be considered "fresh" for (i.e. no need to requery the DHT for updates) while the lifetime is about how long the record should be considered "valid" for (e.g. if you were prevent by a malicious party from getting IPNS updates how long could a user go without being prompted that something fishy was happening).

Source: IPNS over pubsub loses resolve after 24 hours ¡ Issue #996 ¡ libp2p/go-libp2p ¡ GitHub

Good luck :slight_smile:

2 Likes

Thanks for the reponse, Mark!

I’m aware of the need of storaging it “somewhere”. However, is IPNS intrinsically worse at this than content-addressed info on IPFS?

My understanding is that if users of your app are accessing IPFS directly by becoming an IPFS node, and not just accessing through a gateway, then other IPFS nodes will contribute to the storage needs of the app, just like users of the app contribute to the storage needs of other IPFS users. Is this a wrong asumption?

Of course, I understand that there are no guarantees about the publicly stored data. Although I understand that if the node that publish some data is still online, then the data will be accessible (I assume that the network is not usually partitioned).

However, this is clearly a weak guarantee. Therefore, my next question would be, can some other user (in “twitter” for example a follower) pin the data?
Because, if the guarantee is that your data will be accessible if at least you or one of your “followers” are online, then, it’s highly likely it will.

Now to get an IPNS that is quick you want a given node to quickly be able to find another node that has your IPNS record. So you want your IPNS record to be stored by a lot of nodes.

I understand that the main problem of this, compared to IPFS, is that IPNS records are mutable, therefore, if you get the record, you may or may not have gotten the latest record, whereas in IPFS you get the data or you don’t (no stale info).

At the same time, I see an advantage of IPNS over IPFS, IPFS blocks can be “big”, but I assume IPNS records are really small, since they basically need to stored a signed object with a counter or timestamp (to know which records are the latest) and the CID of the target.

Just like I asked about application-level pinning of data, could an application manually pin IPNS record of other “users of interest” (same example, followers)?

The funny thing I see here is that pinning the data of your friends mostly requires what the app actually needs: you need to download the latest data from your followed accounts in order to show the timeline, so why not share it when another user requests it?

The problem I see is that something like this requires a big amount of IPNS resolutions (one per each followed user). And, as you pointed out, each IPNS resolution requires multiple requests to multiple hosts.

I guess that there is no perfect solution for this, as the problem is intrinsic to decentralized mutable state. But, I think that combining requests is the way to go, if you want to resolve 100 IPNS addresses of the 100 users you follow, chances are, many of these users will follow some of the other 99 users.

They both have their own usecases.
IPNS should be used if you want mutable content but keep the hash as-is. The content here is still in IPFS, you just have a mutable entrypoint which makes updates a lot easier.
IPFS is just for whatever content.

You have a correct understanding of how IPFS works :slight_smile:

In essence, you are correct. But there’s more to it. Data is available if it’s pinned by at least one node and that node can be found by the network. You should have one node of your own online that serves all the data of your “twitter alternative” so that people can access it for data that no other user has accessed yet.

And yes, users can pin your data. If you run IPFS Companion, you even have a button in there that says “PIN IPFS resource” (it then pins the site you’re looking at). But here you should realize that a pin is just the content that’s available at that time. If you update your data, the user needs to pin the updated data. I believe the IPFS team is working on a pinning service API where they did discuss “pin followers” (or something alike). It would essentially mean that if i pin something and that something gets updated that i automatically pin the update too. Don’t nail me down on this, i’m not entirely sure if it did this exactly and if it even made it.

The IPNS record itself is at most a few kilobytes. I don’t even think it comes close to 1 KiB. It’s like storing a shortcut on your desktop, only in the IPFS world :slight_smile:

Yes and no.
If you pin an IPNS record, you pin the underlying IPFS object it links to.
And by simply pinning, the user has to resolve (happens automatically) the IPNS record to get the IPFS hash which means that user now helps in keeping the IPNS record alive.

That’s true.
Developing an application/website in the “decentralized” world (quick note, IPFS is distributed, not decentralized) is a whole different beast to tame. It takes a lot of careful consideration to make something work effective in a distributed world.

Just as an example, let’s take twitter here. As you seem keen on making an alternative for that :wink:
I don’t think you realize yet how complicated that would be to be fully distributed. How would you do user login? You have no server! How would you post a new tweet? You have no server! How would you moderate, as you again have no server that contains the filters. How do you even create an admin panel? How are you going to search for a user if you have no database to search in? The distributed world, as you now probably realize, has quite some challenges for true distributed sites.

All of the challenges above are fairly simple to solve when you have a server. There are well established libraries that can help you there. In the distributed world however… Things work differently. If you truly want to endeavor into all of this then you’re going to need DID, there the Ceramic Network has you covered. And you probably need A LOT of smart contracts for the management of it all.

Good luck :slight_smile: I very much encourage you to do this! If you manage to do this without any server (minus one to host your data as “seed server”) then you’d probably have the world’s first true distributed twitter alternative. I know there are sites out there that “claim” to be distributed but they quite often end up using “cloud services” or still some small centralized part.

1 Like

Is IPNS still relevant when there are blockchain based name systems like ENS and Unstoppable?

IPNS is slow and not global
blockchain content resolving is fast and global

I don’t have to pay for IPNS.

2 Likes

I see what you mean. I had the same thoughts some while ago. As IPNS does “feel” unstable and blockchain feels stable and fast.

So it is easy to think that you can use the blockchain instead of IPNS and effectively discard IPNS completely. The thought is tempting but likely flawed, as i discovered :slight_smile:

So here’s a usecase for IPNS that you would find very difficult to do with blockchain.
Picture yourself building a website. You add a folder to IPFS and publish that to get an IPNS. To the outside world you distribute your IPNS hash to see your awesome website. It’s all good and awesome till you want to fix a bug. You make your change, add the folder again to IPFS and publish it again. You now have an updated IPNS record that points to your new published site. Your users simply press F5 and see your awesome site!

Here’s how the same usecase would work if you were using a blockchain.
In your first add to IPFS you get the ipfs hash and add it to your favorite blockchain as some custom data or whatever. Let’s assume you use ENS here as that combination can work.
Every update you do in ENS costs you money. Every update you do costs time to propagate (just due to the chain verifications).
Now if you update your site, you have to go through updating ENS as it - with an IPFS hash - needs to be updated to point to the new data. And this change again costs you money.

What you can do is both. ENS + IPNS. If in the above example you had chosen to add an IPNS hash then your update would not even have to touch ENS. But then you’re back to the “IPNS is slow” problem and are back to where you started from :slight_smile:

1 Like

Free is not a feature, it’s an issue

What does that even mean? Free is an issue? You’d like it to not be free? How is being free not a feature? You’d like to build a distributed web where everyone can participate but as long as they pay? And now you’re going to have to pay every time you change something on your web site? At least now with the regular web I can just buy my hosting and update it whenever I want at no additional cost.

2 Likes

I think we should think about why ads or a subscription model is needed (if it is), and I think the anwser is hardware costs. Hardware cost is usually linear respect the number of users, whereas you clearly don’t need to multiply your engineering costs by the number of users.

Therefore, I think that the business model will be imposed by the computation model:

  • Twitter model. Computing on company servers => Costly => Ads or subscriptions
  • Bitcoin model. Computing on third party servers => Costly => Fees to those third parties
  • New model. Computing on your device, tit for tat with other peers => Cheap => donations?

In Twitter and Bitcoin models you need to pay for the computing power, in the new model you don’t, your own hardware is enough. Of course, you may not be online 100% of the time, and you may not have the necessary performance 100% of the time, but you can make for it. Your hardware works for a peer some time, and the peer’s hardware will work for you later. Cooperation is key.

Still, I don’t think that the new model could replace both models everytime. For example, I cannot imagine a new model for money. Money requires an enforcement of guarantees that seems difficult to maintain without miners. However, I can imagine a new model for most big tech apps: whatsapp, twitter, facebook, instagram, youtube, twitch… I hope to see this.

1 Like

IPNS works fine. I just did a test the other day with two fresh nodes, published IPNS for a specific new key in each, was able to resolve the record from the other node in each respective case. Laggy … and had to retry a few times before Alice could see Bob’s published IPNS but within a few mins they got it. So idk, seems to work for me to build the future.

Tracking the amount of state and frequency of changes to said state is well beyond the design parameters of IPNS. Something like Textile.io’s ThreadDB ThreadDB | Documentation | Textile or OrbitDB https://orbitdb.org/ might be closer to what you’re after.

Is there a tutorial somewhere showing how to get this done? I have three nodes running js-ipfs, go-ipfs, and a js-ipfs embedded api client, can’t seem to get them to see any published IPNS records, though they can see IPFS records just fine, and can see their own published IPNS records. Already configured NAT, everything seems to be online.

You may just not be waiting long enough in your tests. IPNS can take several MINUTES for a single update to complete.

Going on three days.

And I am republishing every 12 hours.

To clarify, it seems like this works no problem in go-ipfs, when using an RSA key only as ‘self’. Any other configuration of js-ipfs / browser-ipfs, or using an secp256k1 or ed25519 key is a no-go.

Sounds like it may be a bug in the JS implementation, but I’m just guessing. Thanks for posting helpful info though.

Check out this.

Looks interesting, but still can’t get the browser-ipns-publish example to work at all.