Newbie questions

1

Good day all.

By way of background: I’m an old IT fart based in the USA with way too many servers at home connected to the internet via 100MBS up/down fibre (yes, I prefer the old spelling). A friend in Switzerland recently mentioned IPFS so I thought I’d check the project out. I have a fairly strong background in projects like Storj, ScPrime (formally SiaPrime), and cybercurriences. I currently run as public services a Tor non-exit relay, an I2P router, and some ScPrime space.

I’m not currently completely groking IPFS. I spun it down onto a fairly idle server with a few TB of spare space and the installation appeared well documented and easy to do. Noticed my Gentoo based server has a portage entry for it, so uninstalled and reinstalled via that - just so that updates will come through with the rest of the system.

So first question: Does it help the network at all if I spin up a daemon and presumably allocate those TBs of space to it, even if I’m not looking to post files to it (see question 2)? If so, what do I change in the configuration to allocate some of the ZFS based free TBs? (Maybe obvious if I could get the WebUI work - see question 3).

Second question: Seems like I could use this to function as a replacement for my $119/year Dropbox service, which I use simply to share files within the family and to know my important stuff is backed up. Is that a reasonable thought?

Third question: The server I installed on is headless. Suspect it would be handy to use the WebUI via my PC which is on the same network. I made a quick attempt at changing the API to listen on my internal network (172.16.4.0/22) instead of localhost but could not web to that. Now understand that was silly. Tried doing the same using the GATEWAY configuration line, but still couldn’t web over to its IP address (172.16.4.14:8080). Suspect I’m doing something trivially wrong - any suggestions?

Thanks in advance!

2

Yes, any storage offering will help the network. Just mount the blocks and datastore folders onto your zfs location. (or simply, install the IPFS directly onto zfs drive). Then run the node and make sure to serve it as public gateway. Publish the gateway address to public. That’s it. You are helping the network now. But do remember, it will consume most of your fibre bandwidth and it will cost you around 1 TB or more per month.

Possible. Make your node as ‘private’. Then distribute the gateway webUI address to your family members only. More info here on how to do it.

Why are you trying for port 8080? The default port of webUI for IPFS node is 5001. Have you changed it to 8080? Always use lan ip address for binding the IPFS service if you want to serve to all other lan users.

3

Hi,

I would say that running an IPFS node reliably does help the network but indirectly. Other user are NOT storing data on your node, only you can do that but, the DHT benefit from your reliability.

As for your Dropbox replacement idea, know that the IPFS network is PUBLIC. You do not have any privacy only a layer of anonymity. Private IPFS cluster can be setup for what you want.

Lastly, IPFS is backup only if the data is actually replicated somewhere (which is very easy to do).

4

Thanks.

I eventually just did what you suggested and placed the ipfs home directory on the ZFS partition. Verified with “ipfs diag sys”. Will setup NAT for the Gateway address:port in a bit. Is there anything else I need to do to “Publish the Gateway” (feels like there should be)?

Ah…stumbled over https vs. http and making some progress with WebUI (helps if you actually pay attention to the log entry). I’m using Brave and it defaults to https. Only way to get an unsecured connection is to force http (Thanks Firefox).

Current Error Screen

Case 1 wasn’t the problem, followed Case 2 using /ip4/172.16.4.14/tcp/5001 instead of the localhost version, and restarted. Problem solved! Nice!

Status page shows 40MiB of data with 50/20 KiBs incoming/outgoing and files are showing up in the datastore directory.

I presume the system is self-regulating and will throttle once available space is used?

Oh… will drop the Dropbox thoughts for now.

Thanks again.

5

Use swarn.announce function in config to explicitly announce your gateway ips to dht. This will solve much problems. Described here. Also check about basic connection manager on the same page.

Use some reverse proxy like NGINX, use LetsEncrypt certificates and force incoming connection over ssl. That will be much easier for public. You can use any third party load balancer for hassle free solution.

Are you accessing this page directly from the node terminal/server? If not, need to do some port forwarding / iptables for port 5001

Yes, definately. You must handle garbage cleaning properly otherwise your storage will be immediately fill up once you publish the gateway to public and ipfs gateway will get knocked out.

Your idea of dropbox alike is good one only when you use it as private gateway.

6

OP is trying to use node as public gateway. In this case, his storage capacity can definitely benefit the public, let it be read-only or writable.

1 Like
7

Happy to allow it to be writable. Currently showing ~200MiB of files, so guessing it already is? Just set “Announce”: ["/ip4/216.146.251.8/tcp/4001"], in config and restarted. Presuming that was the key to Publishing the node. (Obviously 216.146.251.8 is my public IP address, firewall will NAT traffic to the previously mentioned 172.16.4.14)

Other than those changes, everything appears to be up and running at this point.

Three questions:

  1. If I have a storage failure, I’m presuming all data on my node, as currently setup, is redundant. EG. Nothing precious will be lost?

  2. If I get an itch to monetize this effort, I presume the path would be to bring up a full Filecoin node? (Recognizing this is a lot more resource intensive than something like running a full Bitcoin node.) I would not consider doing so until I had more than a 100 mbit up/down connection to the internet, based on what I’m inferring is Filecoin’s 30 block time, but my ISP indicated better connectivity should be coming rather soon. The cost of a good full-node (e.g. mining) Filecoin server appears to be on the order of $10K-$15K - about the same cost (and power consumption with all that memory) as a Bitcoin S19pro miner.

  3. Speculation, but can IPFS thrive without Filecoin? I tend to see the two of them mentioned like they are joined at the hip.

8

Nah. Until you publish the gateway address to public, no one except you can use the storage you offered. And the 200mb you are seeing are the files you have accessed using your gateway address / node in past untill last gc.

Announcing will only enable other nodes globally to check whether any specific file is available at your node (‘read’ process explicitly, no ‘write’). This is what called as dht, somehow. So, your storage will not be used by public for ‘store’ purpose.
Providing the public gateway means, you have to offer the node’s port 8080 address to public to use it as IPFS endpoint (as an exit relay in tor network). Just like the other public gateways listed on this page. One example is https://ipfs.io/ipfs/`*theUniqueHash*`
Whatever you are doing is just ‘running an IPFS node’ for yourself only.

Yes. If the files you want are not cached/pinned by any other running and healthy node, yes, your data will be lost forever.

IPFS is a protocol and Filecoin is built on this IPFS. But running just an IPFS node is not usable for Filecoin. You have to run Filecoin node itself on your system to monetize your system from Filecoin.

IPFS is base protocol. Filecoin is built on top of ipfs. So, filecoin may/may not be there but ipfs will be there until web3 system gets any new and more efficient and lightweight protocol for decentralized storage mechanism.

No. There is/will not be any hip between these two. Say, IPFS is a whole space and Filecoin is in ipfs’s domain.

9

Sorry for being a pest, but want to make sure I get this write. I’m trying to making a public r/w Gateway to maximize my passive support. (“passive”: setup and forget)

I do have port 8080 exposed on my firewall. I recently created a DNS A record for ipfs.carpenter-farms.us

I just added a DNS txt record for carpenter-farms.us with a value of:

_dnslink=/ipfs/12D3KooWKbuo2r2C6zPafi9YavsRX95S6riowMeF5sxTCuEwCs4G where
12D3KooWKbuo2r2C6zPafi9YavsRX95S6riowMeF5sxTCuEwCs4G is my Peer ID per my reading of the IPFS Gateway doc (its references {cid}, which I’m presuming is the Peer ID?).

In the ipfs configuration file I have:

`"Gateway": "/ip4/172.16.4.14/tcp/8080"`

Which is the local IP. Do I need to use the public IP?

I do have:


 "Announce": [
      "/ip4/216.146.251.8/tcp/4001"
    ],

and

 "API": {
    "HTTPHeaders": {
      "Access-Control-Allow-Methods": [
        "PUT",
        "GET",
        "POST"
      ],
      "Access-Control-Allow-Origin": [
        "*"
      ]
    }
  },

along with:

"Writable": true in the “Gateway” section.

Does that look correct? Pulled most of that from the This Guide

10

I am unable to reach the given url. Plz check.

No public/private ip here. You should be using this only.
"Gateway": "/ip4/0.0.0.0/tcp/8080"

11

Thanks for checking. Weird that the _dnslink was there but the A record I added in the same session wasn’t. Should be now.

Changed the gateway, which in retrospect makes perfect sense.

12

The port 8080 on your ip is open. But ipfs is not reachable there. Seems some error in config.
Can you reach ipfs on the address?

13

The webui is working fine: http://puu.sh/J4Adw/b6d69c5df5.png

Here is the full config file:

ipfs@deskserv ~ $ ipfs config edit
{
“API”: {
“HTTPHeaders”: {
“Access-Control-Allow-Methods”: [
“PUT”,
“GET”,
“POST”
],
“Access-Control-Allow-Origin”: [
“"
]
}
},
“Addresses”: {
“API”: “/ip4/172.16.4.14/tcp/5001”,
“Announce”: [
“/ip4/216.146.251.8/tcp/4001”
],
“AppendAnnounce”: [],
“Gateway”: “/ip4/0.0.0.0/tcp/8080”,
“NoAnnounce”: [
“/ip4/10.0.0.0/ipcidr/8”,
“/ip4/100.64.0.0/ipcidr/10”,
“/ip4/169.254.0.0/ipcidr/16”,
“/ip4/172.16.0.0/ipcidr/12”,
“/ip4/192.0.0.0/ipcidr/24”,
“/ip4/192.0.2.0/ipcidr/24”,
“/ip4/192.168.0.0/ipcidr/16”,
“/ip4/198.18.0.0/ipcidr/15”,
“/ip4/198.51.100.0/ipcidr/24”,
“/ip4/203.0.113.0/ipcidr/24”,
“/ip4/240.0.0.0/ipcidr/4”,
“/ip6/100::/ipcidr/64”,
“/ip6/2001:2::/ipcidr/48”,
“/ip6/2001:db8::/ipcidr/32”,
“/ip6/fc00::/ipcidr/7”,
“/ip6/fe80::/ipcidr/10”
],
“Swarm”: [
“/ip4/0.0.0.0/tcp/4001”,
“/ip6/::/tcp/4001”,
“/ip4/0.0.0.0/udp/4001/quic”,
“/ip6/::/udp/4001/quic”
]
},
“AutoNAT”: {},
“Bootstrap”: [
“/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb”,
“/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt”,
“/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ”,
“/ip4/104.131.131.82/udp/4001/quic/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ”,
“/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN”,
“/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa”
],
“DNS”: {
“Resolvers”: {}
},
“Datastore”: {
“BloomFilterSize”: 0,
“GCPeriod”: “1h”,
“HashOnRead”: false,
“Spec”: {
“mounts”: [
{
“child”: {
“path”: “blocks”,
“shardFunc”: “/repo/flatfs/shard/v1/next-to-last/2”,
“sync”: true,
“type”: “flatfs”
},
“mountpoint”: “/blocks”,
“prefix”: “flatfs.datastore”,
“type”: “measure”
},
{
“child”: {
“compression”: “none”,
“path”: “datastore”,
“type”: “levelds”
},
“mountpoint”: “/”,
“prefix”: “leveldb.datastore”,
“type”: “measure”
}
],
“type”: “mount”
},
“StorageGCWatermark”: 90,
“StorageMax”: “8TB”
},
“Discovery”: {
“MDNS”: {
“Enabled”: false,
“Interval”: 10
}
},
“Experimental”: {
“AcceleratedDHTClient”: false,
“FilestoreEnabled”: false,
“GraphsyncEnabled”: false,
“Libp2pStreamMounting”: false,
“P2pHttpProxy”: false,
“StrategicProviding”: false,
“UrlstoreEnabled”: false
},
“Gateway”: {
“APICommands”: [],
“HTTPHeaders”: {
“Access-Control-Allow-Headers”: [
“X-Requested-With”,
“Range”,
“User-Agent”
],
“Access-Control-Allow-Methods”: [
“GET”
],
“Access-Control-Allow-Origin”: [
"”
]
},
“NoDNSLink”: false,
“NoFetch”: false,
“PathPrefixes”: ,
“PublicGateways”: null,
“RootRedirect”: “”,
“Writable”: true
},
“Identity”: {
“PeerID”: “12D3KooWKbuo2r2C6zPafi9YavsRX95S6riowMeF5sxTCuEwCs4G”,
“PrivKey”: “XXXXX” (obviously modified for this post)
},
“Internal”: {},
“Ipns”: {
“RecordLifetime”: “”,
“RepublishPeriod”: “”,
“ResolveCacheSize”: 128
},
“Migration”: {
“DownloadSources”: ,
“Keep”: “”
},
“Mounts”: {
“FuseAllowOther”: false,
“IPFS”: “/ipfs”,
“IPNS”: “/ipns”
},
“Peering”: {
“Peers”: null
},
“Pinning”: {
“RemoteServices”: {}
},
“Plugins”: {
“Plugins”: null
},
“Provider”: {
“Strategy”: “”
},
“Pubsub”: {
“DisableSigning”: false,
“Router”: “”
},
“Reprovider”: {
“Interval”: “12h”,
“Strategy”: “all”
},
“Routing”: {
“Type”: “dht”
},
“Swarm”: {
“AddrFilters”: [
“/ip4/10.0.0.0/ipcidr/8”,
“/ip4/100.64.0.0/ipcidr/10”,
“/ip4/169.254.0.0/ipcidr/16”,
“/ip4/172.16.0.0/ipcidr/12”,
“/ip4/192.0.0.0/ipcidr/24”,
“/ip4/192.0.2.0/ipcidr/24”,
“/ip4/192.168.0.0/ipcidr/16”,
“/ip4/198.18.0.0/ipcidr/15”,
“/ip4/198.51.100.0/ipcidr/24”,
“/ip4/203.0.113.0/ipcidr/24”,
“/ip4/240.0.0.0/ipcidr/4”,
“/ip6/100::/ipcidr/64”,
“/ip6/2001:2::/ipcidr/48”,
“/ip6/2001:db8::/ipcidr/32”,
“/ip6/fc00::/ipcidr/7”,
“/ip6/fe80::/ipcidr/10”
],
“ConnMgr”: {
“GracePeriod”: “20s”,
“HighWater”: 900,
“LowWater”: 600,
“Type”: “basic”
},
“DisableBandwidthMetrics”: false,
“DisableNatPortMap”: true,
“RelayClient”: {},
“RelayService”: {},
“Transports”: {
“Multiplexers”: {},
“Network”: {},
“Security”: {}
}
}
}

14

The webUI is shows the background process status, say dht.
It does not show the gateway perspective status.

The given config is ok.

15

Ahh. Just found.
Your gateway is running.

image
image1129×684 62.5 KB

I was missing the point here. I was not using the port number 8080 at the url. Sorry, my bad.
Check the attached screenshot above. This url.

Now, you have to use some reverse proxy / load balancer for SSL provision. Also, using port 80 instead 8080 will be convenient for the users. Reverse proxy will do that.

16

Alas certbot insist on using port 80 for verification, at least the last time I dived into it. I currently have 2 ssl web servers behind my firewall and have to toggle port 80 between them in order to renew.

If I support SSL on this server, it will become the third using certbot. I can do that, the question is: “Is it really necessary?”. If “yes”, its OK - just going to add to my support load a tiny bit.

Appreciate all the help. Good to see the basics are working!

17

Use reverse proxy like nginx for this. And get wildcard ssl certificate for carpenter-farms.us domain.
This will allow you to use subdomain ipfs.carpenter-farms.us to your node, apex domain for your main business and port 80 will be universal then.

Yes. It’s necessary. Anything that is unencrypted, is unsecure. Also, it builds trust for the end user.
I don’t think any user using ipfs network (which is already an advanced use case) will be choosing unsecure gateway connection for all his transactions.

18

Thanks!

Alas, I’d need that wildcard certificate on 3 different physical machines, updated every renewal period. I’ll work something out.

19

Rather, why don’t you use Cloudflare?
That will serve the ssl traffic and keep your IP address protected too.

20

Equipment & time “rich”, money “poor”