# Sets or Maps (lookup tables) on IPFS **URL:** \ **Category:** Ecosystem and Usage\ **Created:** [October 29, 2019, 7:07pm UTC](https://discuss.ipfs.tech/t/sets-or-maps-lookup-tables-on-ipfs/6556 "2019-10-29T19:07:48Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![linas](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/linas/32/3166_2.png) [@linas](https://discuss.ipfs.tech/u/linas)\ **Post date:** [October 29, 2019, 7:07pm UTC](https://discuss.ipfs.tech/t/sets-or-maps-lookup-tables-on-ipfs/6556/1 "2019-10-29T19:07:48Z")
I’m wondering how to best implement the idea of a (mutable) set or map on IPFS. Posting to “ecosystems” because this seems like a generic feature that many apps/app-developers would want to have. In my case, I am trying to port an existing graph database to IPFS, and deep down in the guts, it conceptually boils down to a big (mutable) store of (guid, cid) pairs. That is, a lookup table of guid to cid. Here, “guid” is a “globally unique ID” – same thing as CID, since all users know the content, so all users can always get the guid on their own, without any help. What I need to do is to tell the user what the latest flavor-of-the-day CID is that goes with it. So - a basic, simple lookup. The gotchas: - There may be dozens or hundreds of writers simultaneously updating the lookup table. - The typical lookup table has from 1 million to 100 million entries in it (based on “typical” current uses). The design alternatives: - Put the lookup table in one huge file. Well, clearly, that’s not “decentralized”, and has all sorts of issues with multiple writers. (for example, making that file be a CRDT append-only log is not practical, if it starts with 100 million entries, and only gets larger. And if it’s not CRDT, then it requires a lock to update.). - Create an IPNS key for each guid. Well, creating 1 million or 100 million different IPNS keys does not seem very practical or scalable, either. Is there a third way? All I can think of is an IPNS extension, so that, given a (PKI-key, GUID) pair, I can look up the CID that goes with it. I’m imagining that most database apps, or anyone maintaining a mutable index of data, is going to face this issue. It’s not hard to solve with a single, centralized master file holding the lookups, but a single centralized massive file sure doesn’t feel very “decent” to me … ideas? How to move forward? ---
**Author:** ![carsonfarmer](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/carsonfarmer/32/1630_2.png) [@carsonfarmer](https://discuss.ipfs.tech/u/carsonfarmer)\ **Post date:** [October 29, 2019, 7:22pm UTC](https://discuss.ipfs.tech/t/sets-or-maps-lookup-tables-on-ipfs/6556/2 "2019-10-29T19:22:03Z")
There are several decentralized databases (key/value tables) built on IPFS. See [https://github.com/textileio/go-textile-threads](https://github.com/textileio/go-textile-threads) for a new work in progress, also [https://github.com/orbitdb](https://github.com/orbitdb) for one another one built on IPFS and pubsub. ---
**Author:** ![carsonfarmer](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/carsonfarmer/32/1630_2.png) [@carsonfarmer](https://discuss.ipfs.tech/u/carsonfarmer)\ **Post date:** [October 29, 2019, 7:24pm UTC](https://discuss.ipfs.tech/t/sets-or-maps-lookup-tables-on-ipfs/6556/3 "2019-10-29T19:24:37Z")
Actually, if you are ok with Go, there’s also this, which is more of a Map on IPFS: [https://github.com/ipfs/go-ds-crdt](https://github.com/ipfs/go-ds-crdt) ---
**Author:** ![linas](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/linas/32/3166_2.png) [@linas](https://discuss.ipfs.tech/u/linas)\ **Post date:** [October 29, 2019, 9:52pm UTC](https://discuss.ipfs.tech/t/sets-or-maps-lookup-tables-on-ipfs/6556/4 "2019-10-29T21:52:56Z")
Hmm, OK, thanks, I was kind of hoping for something more low-level, so as to avoid dependencies and overhead. – In my case, each vertex/edge in my graph DB contains a key-value DB inside of it (so e.g. the natural language apps have a 1 to 100 million key-value DB’s in them, with one per word/phrase/sentence/paragraph, that DB holding suff like probabilities, counts and other non-traversable/non-querieable content) – layering all that on top of a flat key-value singleton kind-of defeats the purpose of having it be graph structured. But I’ll take a look, see if I can steal some ideas. ---
**Author:** ![hector](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ipfs.tech/hector/32/178_2.png) [@hector](https://discuss.ipfs.tech/u/hector)\ **Post date:** [October 31, 2019, 10:45am UTC](https://discuss.ipfs.tech/t/sets-or-maps-lookup-tables-on-ipfs/6556/5 "2019-10-31T10:45:26Z")
> [@linas](#): > > layering all that on top of a flat key-value singleton kind-of defeats the purpose of having it be graph structured. The thing that comes to mind is that you could split into multiple flat-key/value stores (one per vertex?) and then keep a flat one for the graph?