I’ve noticed that publishing to IPNS is taking a very long time on my server. I just timed it at 5 minutes, 52 seconds. This this normal? Is there any way to reduce this time? I perused the output of “ipfs name publish --help”, but nothing seems like it would help.
I am running the dameon with the following options:
/usr/bin/ipfs daemon --enable-pubsub-experiment --enable-namesys-pubsub –
enable-gc
I also just started doing IPNS today, and noticed the same thing. It was taking a crazy long time, like a full minute, just to do a name publish. My setup is using the go-ipfs docker and these env settings in the yaml (docker config). I hadn’t tried to troubleshoot by looking at log files yet.
routing: "dhtclient"
IPFS_PROFILE: "server"
IPFS_PATH: "/data/ipfs"You could manually connect to relevant peers that’ll be querying your content. Maybe someone else knows something but if those peers are unknown or it should be accessed globally, I think that’s just what it is right now. Compared to architectures like DNS, a minute is still a pretty good duration for name propagation though, wouldn’t you agree @wclayf ?
So I have this big platform (quanta.wiki) and was about to add the ability to make every node of data be an IPFS DAG entry (which would be akin to Twitter storing each tweet as an IPFS DAG node), and my plan was to make every “node” also have a stable/permanent IPNS name, but I think if IPNS needs to distribute each change to tons of peers then I may be mis-using (or misunderstanding) the kind of scalability that IPNS is good for. I may need to rethink. Thanks very much for the reply @StationedInTheField
@wclayf, you might be better off publishing a directory with each node of data as a file under it with a permanent name, so that you have something like /ipns/$MY_HASH/$NODE_1_NAME. That way you can reduce the number of publishes you need to do by batching the changes.
That’s a very good concept @teknomunk, thanks. I haven’t done anything with the “internal paths” that can be used to reference stuff inside a CID, but I should learn that first. I was going to use DAG instead of files, for my “tweet-like” things, because DAG seemed easiest way to store arbitrary small chunks of JSON which is what I’m doing, but interestingly each of my JSON chunks DOES already have a slash-delimited path property which is how the tree-structure of the data is achieved.
UPDATE: In my use case it’s actually more important to have the best performance than to even have stable (unchanging) CIDs, so I was just assuming DAG was the fastest way to read/write small chunks of text and was more appropriate than storing as actual “files” in IPFS. I might be wrong that DAG is faster.
Looks like what I want, essentially an --async option, is detailed in this go-ipfs issue, but the last change was made over a year ago in May of 2019. While I am a programmer, I both don’t know golang syntax and am not familiar with the go-ipfs codebase. I guess I’ve just got to wait for someone else to get around to it…
Actually what I need is very simple: A mutable tree.
Each tree node has a payload (JSON) and an ordered list of children. I need to be able to add/remove children from any node, and update the JSON on any node. My tree however may be 100s of levels deep, so approaches that require me to modify a parent node every time a child changes would not perform well enough. Obviously a Merkle tree is out because it does require a traversal too the root any time any node changes, to update the hashes of all ancestors/parents. My operations on add/delete/update need to operate in constant time (nearly) and therefore cannot be proportional to tree depth, so it’s a challenge.
My current implementation is a MongoDB database where each node (mongo document) has a “path” property, and that path is where the tree structure is derived from, but i wanted to get to where I can perfectly represent the DB in IPFS for various purposes.
I am aware of Orbit DB but frankly their keystore is just a key store, and can’t do mutable trees just like IPFS itself can’t do mutable trees. (other than merkle snapshots of trees). If IPNS however were high-performance (minutes won’t cut it) that actually would allow ‘trees’ to be implemented, because each tree node could have a fixed CID…and avoid the ‘chain reaction’ of updating each parent.
TL;DR: Is MFS (Mutable File System) the best way to achieve a ‘mutable tree’ even if you don’t really need ‘files’, but just JSON chunks.
Each tree node has a payload (JSON) and an ordered list of children. I need to be able to add/remove children from any node, and update the JSON on any node.
I have been trying to solve the same problem as well. Needing to “update” a DAG node efficiently seems to be a reoccurring challenge in any conventional kind of app that IPFS can help decentralise. But perhaps IPFS is not made for those use cases? I don’t know yet. Perhaps that problem should be approached differently.