I’m trying to understand how IPNS really works, but I find the documentation very laconic on the technicalties.
It is said that the IPNS address is the hash of the public key of the publisher. Is it because, when someone wants to resolve the IPNS address, they find the public key of the publisher using the DHT and then ask them directly what is the last version of their website? But in this case, what is the point of hashing the public key? Couldn’t we just use the raw public key as an IPNS address, so that we can contact the publisher directly?
And if all that IPNS was doing is ask the DHT to find a public key, and then ask the owner of this public key directly, why does it take so much time?
I have also heard someone say that IPNS lookup waits for a quorum of 16 nodes in order to resolve the address. Is it true, and what does this mean?
IPFS’ DHT find content by looking up for their hash. To figure out which peer to ask for, peers need an identity of the same format. So the peerID are also multihash of the same size.
Then, IPNS records are also published on the DHT, so it make sense to use the peerID for routing, AKA the hash of the public key.
About the peers waiting for 16 peers to resolve an adress: this is no longer true, or won’t be true soon as all IPNS records will be signed by the published, so the receiving end is able to verify and use it instantly.
So just to make sure, I was correct about asking directly the publisher of the IPNS record for the latest version of the website? It means that if the publisher is offline, there is no way to retrieve the latest version of the website?
I think I got it. I thought that the peerID in the DHT was the public key of the node, but in fact it is the hash of the public key. But why isn’t the peerID simply the public key? Is it because the RSA public key size (2048 bits) is too long compared to the Kademlia peerID size (256 bits)? But I have heard that a 256 bits ECC key is as secure as a 2048 bits RSA key, so would it be theorically possible to use an ECC key as a peerID (and thus the IPNS address is the public key)?
It begs 2 questions:
why hasn’t it been done before? I mean it should increase the lookup time tenfold.
if we use 16 nodes quorums to resolve IPNS addresses, while we’re at it, why don’t we use them to have human-readable names? Let’s say I want to buy the “wikipedia.org” domain name, I “hire” the 256 nodes who have the peerID the closest to hash(“wikipedia.org”) so when someone asks them the peerID associated with the domain name, they must return mine. Sure, some of them might not be interested to be hired, some of them might even be corrupted (return the wrong peerID) but the person doing the lookup just needs, let’s say, 16 times the same answer to consider that they found the good peerID. There may even be a disclosure mechanism, in the case that an aberrant peerID is returned, to desincentivize corruption. To prove to me that they did their job correctly and to get their salary, the hired nodes must give me all the lookup queries they received with the reply they answered, signed by them and the person doing the query. There should also be a mechanism that prevents creating lookups queries signed by public keys created on the fly to get money without actually doing the job.
If you can contact them, this is the fastest way.
Pro tip: If you have a particular set of peers you always fetch update from, you can build a “peering agreement” with them so you and them stay connected. They have to accept too. This is optional as the DHT will try to contact them or close peers having the record anyway.
Fortunately, this is incorrect. IPNS records are replicated on close nodes (DHT-distance-wise). They are replicated dynamically when nodes become unreachable, so it shouldn’t be a problem.
Yes.
Yes. Actually, this is the plan, and you can already generate ECC keys/identity. It will become the default next release or the one after that. Actually, you are probably already connected to peers which address is 12… instead of Qm… These are ECC keyed peerID.
Now that I think of it, it is not true yet. The plan is to make ECC the default identity for peers, so that signatures of IPNS records are smaller, so that it is efficient and not too much of a memory burden to broadcast and store signed IPNS records. Coming soon to a node next to you™.
I think it was for historical reasons (simpler to implement), then considerations on performance and bundle size, and it needed a refactor on how info about peers were managed (now complete), and the need to switch to ECC for more compact signatures.
This is a completely different problem. There is different solutions for human-readable names, depending on how you want to solve the problems of many people wanting the same name, domain equating, phishing, etc.
There is actually a centralized solution proposed by IPFS: the DSNLinks. Basically, it means storing in a DNS record the IPFS multihash of the resource, or the IPNS record. Resolution is as fast and reliable as a DNS lookup, but is centralized. If you want a human-readable name for your IPFS resources, you can buy a domain.
A decentralized solution is Ethereum Name Service(ENS) + IPFS.
You proposed solution is an alternative that would have to overcome many challenges (yet probably possible):
“Hiring” means money or crypto. Meaning centralisation or blockchain, so a totally different project.
Also, how do you manage churn and Sybil attacks inserting nodes closer to the target?
This is actually the case: they should return an IPNS record signed by you. If they don’t, you are more likely to drop the connection with them.
This is the case: only the publisher can sign the IPNS record. (Anyone can forward it, though).
I don’t really see how asking 16 nodes to confirm that an IPNS record does indeed belong to someone is simpler to implement than just signing the record. I still have some questions on this matter if you don’t mind:
how are the the 16 nodes chosen? Are they the 16 closest nodes to the owner of the record by the XOR distance?
is the IPNS resolve time this long because we have to contact these 16 nodes? In this case, when the IPNS records will be signed by the owner, we just have to find one person (the owner or one of his neighbours) who stores the record to resolve the IPNS address, so the resolve time should be way faster, right?
The idea is quite different from blockchain-based naming systems like ENS, Namecoin, Handshake, you name it… because the consensus does not rely on a blockchain but on the quorum. The name/address pair is not stored in a distributed ledger, but on multiple nodes that have a peerID close to the hash of the name.
This idea is reward-agnostic, I could pay you in bitcoin, in ruble or in apricot if you want.
I know that this idea has a lot of flaws, but my point was that if we chose to use quorums to achieve consensus, we can store records of anything we want (domain-names, usernames, etc.) not just peerIDs.
Besides, the 16 nodes quorum system you described (without signatures) used in IPNS right now is also vulnerable to Sybil attacks, no?
Sorry my explanation wasn’t very clear. Let’s say Wikipedia hires some node to store the “wikipedia.org” => “/ipfs/Qm…” record. Whenever it receives a request from someone to get the latest CID of Wikipedia, it takes the record, timestamp and sign it, and return the result to the requester. In return, the hired node asks the requester to send back the signature proving that they received the record. Then, the hired node sends all the signed lookup requests it received to Wikipedia to get the reward.
But what I meant in my other message is that some hired node could create fake identities to sign a lot of lookup requests to get a lot of rewards from Wikipedia. It is a bit reminiscent of a Sybil attack, but not in the sense that someone would flood the network with fake identities to maximize the odds of having a peerID close to hash(“wikipedia.org”).
I think that IPNS came along way after IPFS. But to be honest I don’t really know the details.
It’s the 16 first to answer. There can be more in the network. You can find peers having the record anywhere if they fetched it themselves, but you are more likely to find some near the publisher. Yes, according to XOR distance.
Yes, I think so.
(Worth noting, you may end up with a record when there is a newer available, so you might want to still contact the publisher or ask for quorum. But if it’s signed, at least it’s up to you.)
Yes, for now, it is vulnerable, and this is another reason why they are changing this.
Yes, it would be an easy attack against the publisher.
Also, if the publisher record is not signed, there is a Sybil attack possible by aggressively connecting to the victim to be the majority of its connections, and coordinate to serve them a different page. A censor’s dream.
If records are signed by publisher, a censor can only send outdated valid records.
If records and timestamp is signed, then there is no need for consensus, and censor can only visibly block content or serve outdated content by being all the connections of the victim.
Even if all relaying nodes behave correctly, there is another attack: another publisher paying nodes to redirect seekers of wikipedia.org toward /ipfs/QmMyEvilVersion. How do you chose which one is correct and avoid phishing?
A solution is rely on already existing name: this is DNSLink, and the rules to claim a human-readable name is the rules to have a domain name.
A solution is ENS, and the rule to claim are those of ENS.
See also Unstoppable Domains which is integrating IPFS.
I know there are calls for integration from some other alternative domain names whose names I forgot.
But honestly, I think for already existing web domains, DSNLink is better: free, easy to configure. For mutable content, put a IPNS record instead of an IPFS multi hash, and it will soon be fast and secure.
Ok so if I understand correctly, IPNS uses quorum not to make sure that a record is legit (the signature does that), but to make sure that it is up-to-date (unless we could contact the publisher directly so no need for quorum). It explains why IPNS is slow even when the record is signed, because there is still need of a quorum.
And I reckon that what does IPNS over PubSub is just create a list of nodes that had the record the first time we asked so that next time we don’t have to go to the trouble of asking random people for the latest version of the record. I also assume that when the publisher changed the record, it notifies its neighbours (XOR distance wise) of the changes, and that there is also a cache system when we fetch a record.
Sybil attacks are the reason why the idea I proposed doesn’t work as it is now. I have thought about a way to fix this, it is basically to hire more nodes than the attacker has close to the target, and we put in the IPNS record the number of nodes needed to validate it, but this idea doesn’t work anymore if the attacker corrupted more than 51% of the nodes of the network. So I wonder how feasible it is for this to happen, in this publication they say that they found a way to do Sybil attacks on the mainline DHT (27 million daily users) at a low cost, so it would be totally possible to corrupt more than half of a smaller network like IPFS, right?
Anyway, quorum systems, or “Proof of Popularity” as one may call them, cannot be a consensus mechanism, because it is far easier for an attacker to create 51% of sybils in a network than to own 51% of its hashing power or 51% of its crypto. It is a bit reminiscent of the “Argumentum ad Populum” fallacy, because one person could create a lot of fake accounts to reviewbomb an Amazon product.
But I’m really bothered by decentralized naming systems leveraging blockchain, like NameCoin and ENS, because it’s like saying “You want to decentralize DNS? Then store the entire DNS on every computer!” I would rather prefer a more partitioned solution. Maybe I have to take a look at blockchain sharding.
I think the strategy to wait for more records will be configurable. So you will probably be able to take the first record you receive if you want. In fact I think IPNS record updates will spread quite fast because a limited number of nodes are responsible for the record and they are likely well connected together (because they are close XOR-wise). So just using the first you receive when asking for it is a decent strategy, and I expect it to be the default.
Like I wrote, it’s not implemented yet. It is expected to be faster after that.
Actually it doesn’t work if it has more power than the target, not the rest of the network, so the attack is trivial on many targets.
But again, why would you want a quorum? If the message is signed with its timestamp/version number, you don’t need all this complexity and it is way more secure (because you need one honest node, not the majority).
What’s wrong with storing signed IPNS records on the DHT, as planed? It’s partitionned, hence the name of this technology.