haha, just saw this when you commented.
Thanks a lot.
One problem i can think of…
The process looks like this.
- browser uploads file to local node and gets cid.
- cid is then sent to pinata.
- pinata now asks for this
cid and finds that my local node has it. pinata gets the file and stores it and also on all its nodes for safety.
What if as soon as
cid is successfully brought to
pinata, immediatelly my local node shut down(user turned browser or anything like this), this means that when pinata asks for the
cid, it won’t find it anywhere(my node is closed). This could happen quite a lot. What’s the workaround in this case ?
Actually the purpose of having Pinata pin it, is the data itself is also sent to their nodes, where they’ll re-host it for you. So when the local node goes offline, the CID will still be accessible .
@Discordian true, haha ,but I explained a different scenario.
Let’s follow together.
- we upload file to local node in browser with
ipfs.add and this returns cid.
- we then do
await ipfs.pin.add(cid). After this is resolved, It means that
cid was received by pinata nodes. So what they will do is ask for the
cid to the ipfs nodes. and pinata node will find it in our local node, but the problem is that what if as soon as pinata receives
cid from my local node and starts to retrieve file by
cid, at that very moment, local node goes offline, meaning that it no longer serves the file and pinata will not be able to find it at all for the first time.
makes sense ?
Hey , hope you had a wonderful weekend. Just as a reminder, what do you think about my last scenario ?
Okay I understand. So you’d want to ensure you’re not just doing
await ipfs.pin.add but also doing
await ipfs.remote.pin.add afterwards, so Pinata pins it. I believe
ipfs.remote.pin.add will also block until Pinata completes the pin, so you could display a message when it completes, so the user doesn’t close the page too early.
Thank you! Sorry about that, I totally missed the reply!
That’s awesome if it’s how it’s going to work, but:
Q1: I guess, if user uploads a really big file, for sure, it’s gonna take a good amount of time. which seems acceptable.
Q2: If you remember that we agreed on using pinata, and I don’t use the back-end service at all. I realized one problem. Where should i store
pinata private key ? I mean, storing in front-end can be exploitable by other people. you might say that i can store it on CI/CD , but again, this just makes it harder for malicious users to get their hands on it, but it’s still possible, because the key is still built in the front-end. any ideas ?
Thanks a lot and no worries.
Q2. Oh jeeze I hadn’t thought that through I suppose . You could setup your own bootstrap node, and have other nodes communicate with it over PubSub, and have that node do the
ipfs.remote.pin.add. I’m hoping to have a demo more-or-less demonstrating a way to do this up hopefully today (at least a draft) (edit: draft likely won’t be ready until tomorrow).
tl;dr for how I’m doing it, I do the add, like you’re doing right now, then I broadcast some JSON over a PubSub topic, from that a python script running on a bootstrap node (but you can use nodejs) gets the JSON and then adds the embedded CID to the node. I use JSON for some metadata, and that’s it really.
Yes, I think using
pinata is still a good choice and as you said, this can be done from the back-end node that is mine…
Can’t wait to see the DEMO <3
What do you mean by
bootstrap node ? is there anything my browser local node can do to connect with
bootstrap node better ? but I guess, this will be shown in the
bootstrap node just refers to a node that’s queried on initial connect for peer discovery. So what I do is:
await ipfs.bootstrap.add("<multiaddr of node>");
await ipfs.swarm.connect("<multiaddr of node>");
The multiaddr should be your own node that you’re hosting. The
bootstrap.add is so next time, it’ll connect to the desired node on startup. The
swarm.connect is so it connects to the node immediately.
The biggest pitfall to this is I find after some time, you can lose connection to the bootstrap node. To remedy this, I found if I listen on a topic (like
keep-alive), and publish to it (can be anything) on both the bootstrap node and the browser node, the connection can stay active for several hours (possibly days if stable enough). I briefly describe this pitfall in a different upcoming blogpost here (keep in mind this is an older draft, but I think the info there will still be useful).
@Discordian I am starting to realize one thing.
pubsub, how can I be sure that when browser local node publishes the json, then server node can start to check what’s on the
cid that’s on the
json and then start to fetch it ? The problem could be the same. When
server node starts to fetch it, that’s when local node might go offline and we can’t really do the
I think the best way is to push files to my node.js server and I do the ipfs add and pinning from there completely. This ensures that whenever my
await is done, everything is finished.
Makes sense ?
Oh true, that makes total sense to me. Could have the client wait for the server to broadcast a message that effectively says “CID successfully pinned”, and the client could signify somehow it received such a message. Pushing the files directly to a server could definitely work too.
Hope all is going well with you…
I had 2 favors to ask…
- when will your draft be ready ?
- is there any chance you can take a look at FileCoin and Pinata and Ipfs file sharing
Thank you, I hope all is well with you too
Draft demo is live! Haven’t begun the blog post yet: http://portal.thedisco.zone/testing/paste_bundle
Looks like people already help you out on those
hey there, I just wanted to ask , what endpoint you used here inside ipfs.create() method since you are using piñata.cloud.
Hey there as well…
I tried pinata and it didn’t work with
create() function. Not sure really. Could be that pinata doesn’t expose IPFS’s Rest endpoints.
I am sure you got the same problem where connection times out …
yeah that’s why I was asking
Did you find any solution or you ditched piñata ?
I am thinking about using fleek and having a chat right now if they can support what I want…
Will let you know if anything good comes out of it…
Let me know if you find a way with pinata
I just went to their slack channel and asked about this issue, waiting for their reply!
Hi, do you have now a solution to pin files from user’s frontend?
I plan to use Pinata but placing pinata keys on frontend seems not a good solution.
Since this discussion was going on very well, I wanted to ask if its still possible to create a local node in the browser using ipfs-http-client or ipfs-core. I didnt manage to get ipfs-core running like you did in your example. I get ipfs-http-client to run, but I am not able to create a local node in the browser. Can you or anyone else help me (for example what to put in the ipfsHttpClient.create() method to create a browser node)?
EDIT: The local node is just for adding, pinning will be done in the backend. I want to create a node in browser, because I dont want to rely on public gateways. I get the ipfs-http-client to run with Infura API, but as said I want to get it running as local “browser” node