pinata now asks for this cid and finds that my local node has it. pinata gets the file and stores it and also on all its nodes for safety.
What if as soon as cid is successfully brought to pinata, immediatelly my local node shut down(user turned browser or anything like this), this means that when pinata asks for the cid, it won’t find it anywhere(my node is closed). This could happen quite a lot. What’s the workaround in this case ?
Actually the purpose of having Pinata pin it, is the data itself is also sent to their nodes, where they’ll re-host it for you. So when the local node goes offline, the CID will still be accessible .
@Discordian true, haha ,but I explained a different scenario.
Let’s follow together.
we upload file to local node in browser with ipfs.add and this returns cid.
we then do await ipfs.pin.add(cid). After this is resolved, It means that cid was received by pinata nodes. So what they will do is ask for the cid to the ipfs nodes. and pinata node will find it in our local node, but the problem is that what if as soon as pinata receives cid from my local node and starts to retrieve file by cid, at that very moment, local node goes offline, meaning that it no longer serves the file and pinata will not be able to find it at all for the first time.
Okay I understand. So you’d want to ensure you’re not just doing await ipfs.pin.add but also doing await ipfs.remote.pin.add afterwards, so Pinata pins it. I believe ipfs.remote.pin.add will also block until Pinata completes the pin, so you could display a message when it completes, so the user doesn’t close the page too early.
Thank you! Sorry about that, I totally missed the reply!
That’s awesome if it’s how it’s going to work, but:
Q1: I guess, if user uploads a really big file, for sure, it’s gonna take a good amount of time. which seems acceptable.
Q2: If you remember that we agreed on using pinata, and I don’t use the back-end service at all. I realized one problem. Where should i store pinata private key ? I mean, storing in front-end can be exploitable by other people. you might say that i can store it on CI/CD , but again, this just makes it harder for malicious users to get their hands on it, but it’s still possible, because the key is still built in the front-end. any ideas ?
Q2. Oh jeeze I hadn’t thought that through I suppose . You could setup your own bootstrap node, and have other nodes communicate with it over PubSub, and have that node do the ipfs.remote.pin.add. I’m hoping to have a demo more-or-less demonstrating a way to do this up hopefully today (at least a draft) (edit: draft likely won’t be ready until tomorrow).
tl;dr for how I’m doing it, I do the add, like you’re doing right now, then I broadcast some JSON over a PubSub topic, from that a python script running on a bootstrap node (but you can use nodejs) gets the JSON and then adds the embedded CID to the node. I use JSON for some metadata, and that’s it really.
Yes, I think using pinata is still a good choice and as you said, this can be done from the back-end node that is mine…
Can’t wait to see the DEMO <3
What do you mean by bootstrap node ? is there anything my browser local node can do to connect with bootstrap node better ? but I guess, this will be shown in the demo itself.
So a bootstrap node just refers to a node that’s queried on initial connect for peer discovery. So what I do is:
await ipfs.bootstrap.add("<multiaddr of node>");
await ipfs.swarm.connect("<multiaddr of node>");
The multiaddr should be your own node that you’re hosting. The bootstrap.add is so next time, it’ll connect to the desired node on startup. The swarm.connect is so it connects to the node immediately.
The biggest pitfall to this is I find after some time, you can lose connection to the bootstrap node. To remedy this, I found if I listen on a topic (like keep-alive), and publish to it (can be anything) on both the bootstrap node and the browser node, the connection can stay active for several hours (possibly days if stable enough). I briefly describe this pitfall in a different upcoming blogpost here (keep in mind this is an older draft, but I think the info there will still be useful).
If using pubsub, how can I be sure that when browser local node publishes the json, then server node can start to check what’s on the cid that’s on the json and then start to fetch it ? The problem could be the same. When server node starts to fetch it, that’s when local node might go offline and we can’t really do the await on pub/sub system.
I think the best way is to push files to my node.js server and I do the ipfs add and pinning from there completely. This ensures that whenever my await is done, everything is finished.
Oh true, that makes total sense to me. Could have the client wait for the server to broadcast a message that effectively says “CID successfully pinned”, and the client could signify somehow it received such a message. Pushing the files directly to a server could definitely work too.
Since this discussion was going on very well, I wanted to ask if its still possible to create a local node in the browser using ipfs-http-client or ipfs-core. I didnt manage to get ipfs-core running like you did in your example. I get ipfs-http-client to run, but I am not able to create a local node in the browser. Can you or anyone else help me (for example what to put in the ipfsHttpClient.create() method to create a browser node)?
EDIT: The local node is just for adding, pinning will be done in the backend. I want to create a node in browser, because I dont want to rely on public gateways. I get the ipfs-http-client to run with Infura API, but as said I want to get it running as local “browser” node
.
