This is a question, that has occurred to me today. There are currently, to my knowledge, no trust-less, distributed ways of searching through the content published on IPFS. Would there be an interest in a search engine, that followed the IPFS principles of trust-less and distributed? I would say…

[image] CSDUMMI: A Crawler is a peer, that adds to INDEX and verifies other who add to the INDEX. The crawlers use Public Key Cryptography to sign off on each others updates and to the INDEX. Before a crawler signs off on an INDEX File Change, they verify the changes. Will they publish the…

I think you touched and descriped many important aspects. I am convinced, that as the general model, yours is quite sufficient. There are Crawlers, that Index IPFS or other crawlers how they see fit and publish their results to whoever wants to listen. Then there are Searchers, that choose the kin…

But if there is such a need as you are saying that it is and as I can imagine there to be, then I am asking: What kind of INDEX File Format should be used? Should there even be a defined standard? What kinds of search index should be supported? ( https://en.wikipedia.org/wiki/Search_engine_indexi…

Looks like a good model. I would organize the pool as following (mostly summing up your design). Running crawlers: Run a few first crawlers for bootstrapping purpose. They connect to each other and subscribe to a common topic: generalNewsCrawlers/v1. They run a common OrbitDB in CRDT mode They ad…

I have started to try to develop a crawler MVP . I am not yet going to implement pools, because they increase the complexity. But for this, I still need a way to observe the network for files being transferred. The conditions: The crawler has to be given the CID before adding them to the index. …

Great! Best of luck :muscle:! Also: this … :kissing_smiling_eyes: :notes:

There is a few already. There is cyber.page, which is a search engine \ protocol which works with IPFS. And there are a few crawlers Here are the docs for cyber. PS. I am affiliated In short the way cyber works is something like this or here . There is of course the WP @CSDUMMI would love to chat…

This is true. IPFS based Search Engine already works and everyone can take part in creating a knowledge graph. I would be very happy if IPFS community takes an active part in this.

I’m greeted with total darkness. And can it be accessed from IPFS as well? If it can’t, then it is no different then using DuckDuckGo with the site:ipfs.io in the search line.

This is more like a cyberlink knowledge graph. You must upload your files or IPFS hashes with keywords to the knowledge graph. After that you will be able to find it in search engine. The knowledge graph is now in the process of filling. It already has over 100k cyber links. https://cyber.page/

Would there be an interest, in an IPFS Search Engine?

Ecosystem and Usage

Akita May 27, 2020, 7:05am 18

Great! Best of luck !

Also: this…

Topic		Replies	Views
Any suggestion to make IPFS content searchable/addressable by a user defined tag? Help	23	5662	May 27, 2020
Questions after first learning about IPFS Help	12	1734	May 23, 2017
Decentralized Search Engines for the Decentralized web Ecosystem and Usage search	15	4669	December 19, 2020
Work-plans for kubo, helia, & other Shipyard IPFS projects in 2025 kubo , helia	12	613	December 12, 2024
Design idea to solve the scalability problem of a decentralized social media platform using IPFS Help	26	2537	June 11, 2022

Would there be an interest, in an IPFS Search Engine?

Related topics