Metadata-query-addressable filesystem vs content-addressable filesystem

1

Here’s a my random thought of the day:

It makes sense theoretically that there are two use-cases for getting a file:

  1. you know the contents of the file and want those contents - use the IPFS hash
  2. you are looking for a file based on a set of metadata - this is the use case I am curious about.

I have seen many systems built to use database queries to find filenames based on metadata but now that I am learning about semantic web concepts I am seeing just how generalizable this paradigm is. IPFS is the leading solution for (1) above but it is not until I find a good solution for (2) that I can truly lean into this new web framework.

I am imagining a filesystem be built which references files using SPARQL queries instead of directory locations. A SPARQL endpoint could be set up to return IPFS hash locations which can be used via FUSE mount. Hell, you could build a whole “evergreen” collaborative programming language/library on this.

I am new to these things so I am very interested to learn what like this is already out there before I start hacking together prototypes that recreate existing functionalities (again). Any guidance is welcomed.

1 Like
2

There is Underlay but I haven’t seen anything come of it Knowledge Engineering | Protocol Labs Research

IPFS-FAN a function addressable computation network IPFS-FAN: A Function Addressible Computation Network | Protocol Labs Research but again I haven’t seen anything come of it. Something similar but not based on IPFS would be Unison. https://www.unison-lang.org

You would probably look into IPLD and JSON-LD and possibly Linked data fragments https://linkeddatafragments.org

1 Like
3

I can tell IPLD is interesting and relevant but I am not quite grasping the concepts yet. Do you know of any successful real-world projects using IPLD?

1 Like
4

Are you interested in IPLD specifically or more broadly in IPFS. I don’t think that looking at projects using IPFS/IPLD is going to help if you’re working on picking up the general co cents. I’d start with the IPFS.io docs and tutorials. IPFS has been around long enough that many of the blog posts are sometimes old and a bit out of date but the general concepts haven’t changed.

If you have any specific questions you can ask here.

5

IPLD specifically. I am already pretty familiar with IPFS and have used it a few times on different systems. Another attempt at my specific question is:

My understanding of IPLD is very broadly and generically defined. Are there some specific use-cases or real-world examples? All the example code I have seen is illustrative but what I am looking for is an application which is using IPLD and an explanation of how it is being used.

And revisiting my initial post:

Does IPLD bridge that gap to make IPFS metadata-query-addressable?

6

Linked Data Fragments might be what I am looking for. Their list of data servers doesn’t include anything connected with IPFS, so maybe I finally have come up with a novel idea. :stuck_out_tongue_winking_eye: Definitely not a project I want to undertake on my own unfortunately.

My thoughts after reading more on the others :

  • Underlay seems entirely conceptual and stale - I reached out via email though.
  • IPFS-FAN seems focused on distributed computing
  • Unison is a programming language
  • IPLD doesn’t seem to be about the semantic web; it is about interconnecting hashed protocols.
1 Like
7

I had composed a long response but I can’t seem to do it without devolving into a rant. Just save yourself a lot of grief and years of your life and don’t get sucked into the semantic web. It’s fools gold. I actually see a lot of disturbing parallels between that and IPFS.

1 Like
8

@7yl4r KEN Labs is working on two product: Pando - a sidechain metadata store for IPLD which has a IPLD Database inside - JiaoziDB.

1 Like
9

Over a year has passed and I still daydream about this. I want to hear that rant.

1 Like