From @fsteff on Fri Aug 26 2016 17:56:58 GMT+0000 (UTC)
From @fsteff on August 25, 2016 20:40
First, if I chose the wrong place for this topic, please move it or at least refer me to the right one.
I am wondering how fast the lookup for a piece of data is, especially for a large number of small blocks.
I hypothetically think about some sort of (decentralized) search engine, based on IPFS.
The index would be a large merkle tree of tags and topics with documents or other files as “leaves”.
Searching this tree would require at least thousands of times looking up for a block.
As far as I understood this simplified works as follows (please correct me if I misunderstood something):
- Take a hash value and ask the DHT for nodes that store a copy of it (which requires multiple hops though the DHT tree)
- Download the data from one of the nodes
- Check if the data is correct (check if the hash of it matches)
How long does this take?
(Of course it mainly depends on the nodes, but let us assume ideal conditions)
I did not do any tests, but I can imagine this is somewhere from 10 ms to 100 ms or even more (per lookup)…
Also, does IPFS avoid looking up over the DHT if a file/block is stored locally? (would be easy if not)
But despite this little obstacle, thank you for this great piece of work! I have loads of ideas that are much easier to realize with IPFS than by reinventing the wheel^^
Copied from original issue: ipfs/ipfs#197
Copied from original issue: https://github.com/ipfs/faq/issues/167