Files are rarely available through public gateways

Help
1

My setup:
ipfs/kubo:v0.22.0 docker image.
AWS ECS + EFS + Internal load balancer. Node is open to the internet via vpc.
Configuration changes:
I added the vpc ip list to swarm and the vpc ip list to Announce.

 - ipfs config --json Addresses.Announce '["/ip4/'"$IP_PUBLIC"'/tcp/4001","/ip4/'"$IP_PUBLIC"'/udp/4001/quic","/ip4/'"$IP_PUBLIC"'/udp/4001/quic-v1","/ip4/'"$IP_PUBLIC"'/udp/4001/quic-v1/webtransport"]';
 - ipfs config --json Addresses.Swarm '["/ip4/'"$IP_PUBLIC"'/udp/4001/quic","/ip4/'"$IP_PUBLIC"'/tcp/4001","/ip4/0.0.0.0/tcp/4001","/ip6/::/tcp/4001","/ip4/0.0.0.0/udp/4001/quic","/ip6/::/udp/4001/quic"]'

My problem:
The files I pin are rarely available through public gateways. Diagnostic tools, find no problems.

How can I improve file accessibility?

1 Like
2

Diagnostic tools, find no problems.

What diagnostic tools have you used?

1 Like
3

pl-diagnose. I also use curl hitting the endpoint:

https://ipfs-check-backend.ipfs.io/?cid=QmViQnX5wPd...

this is what an example response looks like:

{
  "ConnectionError": "",
  "PeerFoundInDHT": {
    "/ip4/52.91.125.189/tcp/4001": 10,
    "/ip4/52.91.125.189/udp/4001/quic": 10
  },
  "CidInDHT": true,
  "DataAvailableOverBitswap": {
    "Duration": 35529541,
    "Found": true,
    "Responded": true,
    "Error": ""
  }
}
1 Like
4

I’ve been mirroring my content to https://web3.storage to get it on the public gateways. I’m not sure how else to reliably get my content available on a gateway besides that rn. I hope @adin has some advice here :pray:.

5

Those results should indicate that you’re in good shape. Although when I tried connecting to your peer I got that the address was unreachable. Not sure if I happened to ping it when your machine was off or if you’ve got something wrong with your networking setup in AWS.

Using GitHub - ipfs-shipyard/vole: a collection of tools for digging around IPFS nodes

❯ vole libp2p identify --allow-unknown-peer /ip4/52.91.125.189/tcp/4001
panic: failed to dial: failed to dial QmadAdJ3f63JyNs65X7HHzqDwV53ynvCcKtNFvdNaz3nhk:
  * [/ip4/52.91.125.189/tcp/4001] dial tcp4 0.0.0.0:59035->52.91.125.189:4001: i/o timeout

goroutine 1 [running]:
main.main()
        C:/Users/adin/go/src/github.com/aschmahmann/vole/main.go:329 +0xed6

Note: I only used --allow-unknown-peer because I don’t have your peerID, you can just pass yours.

Given that you’re running this node in AWS I’d do a couple sanity checks on your local machine.

  1. Does vole libp2p identity <aws machine multiaddr> work?
  2. Does vole bitswap check <cid> <multiaddr> work?
  3. If your data is a (UnixFS) file, does vole bitswap get <cid> <multiaddr> work?
  4. Does check.ipfs.network or pl-diagnose say that the cid is advertised?
  5. Is the data discoverable using a local node (e.g. running kubo locally – make sure the data isn’t cached locally)?
2 Likes
6

I paste the results below. The IP address I provided earlier was an old address.

  1. vole libp2p identify --allow-unknown-peer /ip4/100.26.177.243/tcp/4001
    result:
PeerID: "12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
Protocol version: ""
Agent version: "kubo/0.22.0/3f884d3"
Listen addresses:
	- "/ip4/100.26.177.243/tcp/4001/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
	- "/ip4/100.26.177.243/udp/4001/quic-v1/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
	- "/ip4/100.26.177.243/udp/4001/quic-v1/webtransport/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
	- "/ip4/100.26.177.243/udp/4001/quic/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
Protocols:
	- "/ipfs/bitswap"
	- "/ipfs/bitswap/1.0.0"
	- "/ipfs/bitswap/1.1.0"
	- "/ipfs/bitswap/1.2.0"
	- "/ipfs/id/1.0.0"
	- "/ipfs/id/push/1.0.0"
	- "/ipfs/kad/1.0.0"
	- "/ipfs/lan/kad/1.0.0"
	- "/ipfs/ping/1.0.0"
	- "/libp2p/autonat/1.0.0"
	- "/libp2p/circuit/relay/0.2.0/hop"
	- "/libp2p/circuit/relay/0.2.0/stop"
	- "/libp2p/dcutr"
	- "/x/"
  1. vole bitswap check QmViQn... /ip4/100.26.177.243/tcp/4001/p2p/12D...
    result:
{"Found":true,"Responded":true,"Error":null}

  1. cid is advertised:

    image
    image837Ă—402 26.4 KB

  2. file is discoverable using a local node

/ # ipfs cat QmWUrkDEx4fGheThWi32g4L6Gqszi14ZtrgZCF7WXGCc8o
{
 "aaa": "asdjashdjaksdhajdh8u3893482342834",
 "ccc": "dfnsidfyu8wefhsdjfhksjdfh89ewfyddihfusiyfh8wehdfhskf",
 "bbb": "dasojdkjasdjakdj",
 "fff": "ahsacmniaoncaslkcm"
}
1 Like
7

That’s interesting, looks like there’s something fishy going on.

If I try to reach you using your UDP/QUIC-v1 port I end up not being able to run libp2p’s identify protocol against it to discover what protocols it supports but if I try reaching you with TCP this isn’t a problem.

❯ vole libp2p identify /ip4/100.26.177.243/udp/4001/quic-v1/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
PeerID: “12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t”
Protocol version: “”
Agent version: “”
Listen addresses:
- “/ip4/100.26.177.243/udp/4001/quic-v1/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t”
Protocols:

❯ vole libp2p identify /ip4/100.26.177.243/tcp/4001/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
PeerID: “12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t”
Protocol version: “”
Agent version: “kubo/0.22.0/3f884d3”
Listen addresses:
- “/ip4/100.26.177.243/tcp/4001/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t”
- “/ip4/100.26.177.243/udp/4001/quic-v1/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t”
- “/ip4/100.26.177.243/udp/4001/quic-v1/webtransport/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t”
- “/ip4/100.26.177.243/udp/4001/quic/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t”
Protocols:
- “/ipfs/bitswap”
- “/ipfs/bitswap/1.0.0”
- “/ipfs/bitswap/1.1.0”
- “/ipfs/bitswap/1.2.0”
- “/ipfs/id/1.0.0”
- “/ipfs/id/push/1.0.0”
- “/ipfs/kad/1.0.0”
- “/ipfs/lan/kad/1.0.0”
- “/ipfs/ping/1.0.0”
- “/libp2p/autonat/1.0.0”
- “/libp2p/circuit/relay/0.2.0/hop”
- “/libp2p/circuit/relay/0.2.0/stop”
- “/libp2p/dcutr”
- “/x/”

Do you have any special kubo configuration (e.g. messing around with the libp2p resource manager, manually added addresses, etc.)? Also, could you try running with a later version of kubo (e.g. v0.23.0) and see if running vole libp2p identify against the quic-v1 port works.

Note: For now you might be able to get things working by disabling quic in your config, but that really shouldn’t need to happen so any debugging/fixing we could do here would be great :pray:

1 Like
8

@michalgrzegorczyk
Can you add /quic-v1 to your listen addresses? You can do that in your ipfs config file.

9

I made a few changes:

  1. update node to v0.23.0
  2. I noticed that in the configuration I have different addresses in the Bootstrap parameter than those in the local node. I changed them.
    old:
"Bootstrap":[
      "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
      "/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
      "/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
      "/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
      "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
      "/ip4/104.131.131.82/udp/4001/quic/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
   ],

new:

"Bootstrap": [
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
    "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN"
  ],
  1. I replaced quic with quic-v1

After these changes, vole libp2p identify returns the following responses:

vole libp2p identify /ip4/3.227.234.217/udp/4001/quic-v1/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t

PeerID: "12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
Protocol version: ""
Agent version: "kubo/0.23.0/3a1a041"
Listen addresses:
	- "/ip4/3.227.234.217/tcp/4001/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
	- "/ip4/3.227.234.217/udp/4001/quic-v1/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
	- "/ip4/3.227.234.217/udp/4001/quic-v1/webtransport/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
Protocols:
	- "/ipfs/bitswap"
	- "/ipfs/bitswap/1.0.0"
	- "/ipfs/bitswap/1.1.0"
	- "/ipfs/bitswap/1.2.0"
	- "/ipfs/id/1.0.0"
	- "/ipfs/id/push/1.0.0"
	- "/ipfs/kad/1.0.0"
	- "/ipfs/lan/kad/1.0.0"
	- "/ipfs/ping/1.0.0"
	- "/libp2p/autonat/1.0.0"
	- "/libp2p/circuit/relay/0.2.0/hop"
	- "/libp2p/circuit/relay/0.2.0/stop"
	- "/libp2p/dcutr"
	- "/x/"


vole libp2p identify /ip4/3.227.234.217/tcp/4001/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t

PeerID: "12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
Protocol version: ""
Agent version: "kubo/0.23.0/3a1a041"
Listen addresses:
	- "/ip4/3.227.234.217/tcp/4001/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
	- "/ip4/3.227.234.217/udp/4001/quic-v1/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
	- "/ip4/3.227.234.217/udp/4001/quic-v1/webtransport/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
Protocols:
	- "/ipfs/bitswap"
	- "/ipfs/bitswap/1.0.0"
	- "/ipfs/bitswap/1.1.0"
	- "/ipfs/bitswap/1.2.0"
	- "/ipfs/id/1.0.0"
	- "/ipfs/id/push/1.0.0"
	- "/ipfs/kad/1.0.0"
	- "/ipfs/lan/kad/1.0.0"
	- "/ipfs/ping/1.0.0"
	- "/libp2p/autonat/1.0.0"
	- "/libp2p/circuit/relay/0.2.0/hop"
	- "/libp2p/circuit/relay/0.2.0/stop"
	- "/libp2p/dcutr"
	- "/x/"
1 Like
10

@adin what do you mean by changing p2p settings?. Which configuration parameters apply to p2p?

I have one more question. When can I expect the file to be available on public gateways? Are the files available after a few seconds, minutes or even longer?

11

Could you drop the quote from me you’re asking about? Probably just me missing things, but I’m not sure what you’re referring too.

This is from the kubo config (e.g. ipfs config show should output it).

Heads up I (and some other folks I see on this thread) will probably take longer to get back to you and debug things this + early next week due to travel and being involved in a number of IPFS, libp2p, etc. events happening around the Labweek23 event.

As soon as the data is advertised in the DHT it should be available to all nodes, including the ones backing ipfs.io, dweb.link, or any of the gateways operated by people listed at Public Gateway Checker | IPFS. There shouldn’t be much special going on with public gateways vs local gateways here. Part of why this is weird and it seems like there’s some bug/weird behavior going on here.

12

I can attach the configuration file. I just don’t know if I can publish all the data.

I hope you are having a great time in Istanbul. :wink:

13

Your private key is in the config file, make sure that’s redacted and you should be good (retrieving it via ipfs config show strips it by default).

14 
{
  "API":{
     "HTTPHeaders":{
        "Access-Control-Allow-Origin":[
           "*"
        ]
     }
  },
  "Addresses":{
     "API":"/ip4/0.0.0.0/tcp/5001",
     "Announce":[
        "/ip4/3.227.234.217/tcp/4001",
        "/ip4/3.227.234.217/udp/4001/quic-v1",
        "/ip4/3.227.234.217/udp/4001/quic-v1",
        "/ip4/3.227.234.217/udp/4001/quic-v1/webtransport"
     ],
     "AppendAnnounce":[
        
     ],
     "Gateway":"/ip4/0.0.0.0/tcp/8080",
     "NoAnnounce":null,
     "Swarm":[
        "/ip4/3.227.234.217/udp/4001/quic-v1",
        "/ip4/3.227.234.217/tcp/4001",
        "/ip4/0.0.0.0/tcp/4001",
        "/ip6/::/tcp/4001",
        "/ip4/0.0.0.0/udp/4001/quic-v1",
        "/ip6/::/udp/4001/quic-v1"
     ]
  },
  "AutoNAT":{
     
  },
  "Bootstrap":[
     "/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
     "/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
     "/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
     "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
     "/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
     "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN"
  ],
  "DNS":{
     "Resolvers":{
        
     }
  },
  "Datastore":{
     "BloomFilterSize":0,
     "GCPeriod":"1h",
     "HashOnRead":false,
     "Spec":{
        "mounts":[
           {
              "child":{
                 "path":"blocks",
                 "shardFunc":"/repo/flatfs/shard/v1/next-to-last/2",
                 "sync":true,
                 "type":"flatfs"
              },
              "mountpoint":"/blocks",
              "prefix":"flatfs.datastore",
              "type":"measure"
           },
           {
              "child":{
                 "compression":"none",
                 "path":"datastore",
                 "type":"levelds"
              },
              "mountpoint":"/",
              "prefix":"leveldb.datastore",
              "type":"measure"
           }
        ],
        "type":"mount"
     },
     "StorageGCWatermark":90,
     "StorageMax":"10GB"
  },
  "Discovery":{
     "MDNS":{
        "Enabled":true
     }
  },
  "Experimental":{
     "FilestoreEnabled":true,
     "GraphsyncEnabled":false,
     "Libp2pStreamMounting":true,
     "OptimisticProvide":false,
     "OptimisticProvideJobsPoolSize":0,
     "P2pHttpProxy":true,
     "StrategicProviding":false,
     "UrlstoreEnabled":true
  },
  "Gateway":{
     "APICommands":[
        
     ],
     "HTTPHeaders":{
        
     },
     "NoDNSLink":false,
     "NoFetch":false,
     "PathPrefixes":[
        
     ],
     "PublicGateways":null,
     "RootRedirect":""
  },
  "Identity":{
     "PeerID":"12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
  },
  "Internal":{
     
  },
  "Ipns":{
     "RecordLifetime":"",
     "RepublishPeriod":"",
     "ResolveCacheSize":128
  },
  "Migration":{
     "DownloadSources":[
        
     ],
     "Keep":""
  },
  "Mounts":{
     "FuseAllowOther":false,
     "IPFS":"/ipfs",
     "IPNS":"/ipns"
  },
  "Peering":{
     "Peers":null
  },
  "Pinning":{
     "RemoteServices":{
        
     }
  },
  "Plugins":{
     "Plugins":null
  },
  "Provider":{
     "Strategy":""
  },
  "Pubsub":{
     "DisableSigning":false,
     "Router":""
  },
  "Reprovider":{
     
  },
  "Routing":{
     "AcceleratedDHTClient":false,
     "Methods":null,
     "Routers":null
  },
  "Swarm":{
     "AddrFilters":null,
     "ConnMgr":{
        
     },
     "DisableBandwidthMetrics":false,
     "DisableNatPortMap":false,
     "RelayClient":{
        
     },
     "RelayService":{
        
     },
     "ResourceMgr":{
        "Limits":{
           
        }
     },
     "Transports":{
        "Multiplexers":{
           
        },
        "Network":{
           
        },
        "Security":{
           
        }
     }
  }
}
15

Hey!
Could you please take a look and help me with this.
@adin @Discordian

1 Like
16

Thanks for sharing the config. It seems that your node has a different IP from the posts before.

Either way, I was able to run identify and ping your node:

âžś  ~ ipfs ping 12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
PING 12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t.

Pong received: time=407.69 ms
Pong received: time=237.33 ms
Pong received: time=825.08 ms
Pong received: time=123.57 ms
Pong received: time=3538.90 ms
Pong received: time=287.15 ms
Pong received: time=398.86 ms
Pong received: time=116.72 ms
Pong received: time=272.68 ms
Pong received: time=6129.95 ms
Average latency: 1233.79ms
âžś  ~
âžś  ~ ipfs id 12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
{
	"ID": "12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t",
	"PublicKey": "CAESIJvw7rRtdExbJGRWwEhzLLS9OhOC0Sy3+b6V9P5Te7/n",
	"Addresses": [
		"/ip4/100.27.12.56/tcp/4001/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t",
		"/ip4/100.27.12.56/udp/4001/quic-v1/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t",
		"/ip4/100.27.12.56/udp/4001/quic-v1/webtransport/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
	],
	"AgentVersion": "kubo/0.23.0/3a1a041",
	"Protocols": [
		"/ipfs/bitswap",
		"/ipfs/bitswap/1.0.0",
		"/ipfs/bitswap/1.1.0",
		"/ipfs/bitswap/1.2.0",
		"/ipfs/id/1.0.0",
		"/ipfs/id/push/1.0.0",
		"/ipfs/kad/1.0.0",
		"/ipfs/lan/kad/1.0.0",
		"/ipfs/ping/1.0.0",
		"/libp2p/autonat/1.0.0",
		"/libp2p/circuit/relay/0.2.0/hop",
		"/libp2p/circuit/relay/0.2.0/stop",
		"/libp2p/dcutr",
		"/x/"
	]
}

I was also able to connect to your node:

âžś  ~ ipfs swarm connect /ip4/100.27.12.56/udp/4001/quic-v1/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
connect 12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t success

The CID you mentioned before seems to also be retrievable over Bitswap:

vole bitswap check QmWUrkDEx4fGheThWi32g4L6Gqszi14ZtrgZCF7WXGCc8o /ip4/100.27.12.56/udp/4001/quic-v1/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
{"Found":true,"Responded":true,"Error":null}

So where’s the problem?

Where things get weird is that I’m not able to find your PeerID (12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t) when looking for DHT provider records for the CID QmWUrkDEx4fGheThWi32g4L6Gqszi14ZtrgZCF7WXGCc8o

This is probably why you are not able to retrieve the CID from the ipfs.io gateway.

Debug providing (publishing provider records to the DHT)

Can you please share the output of ipfs stats provide?

This should help determine why the CIDs on your IPFS are not being published.

If you could also share a CID that you are pinning that would help debugging ( besides QmWUrkDEx4fGheThWi32g4L6Gqszi14ZtrgZCF7WXGCc8o which I saw your shared before)

17

fwiw it seems to connect fine on my end, and works over /quic-v1

$ ipfs swarm peers | grep 12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
# no prior connection
$ ipfs swarm connect /p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
connect 12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t success
$ ipfs swarm peers | grep 12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
/ip4/100.27.12.56/udp/4001/quic-v1/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
$ ipfs block get QmWUrkDEx4fGheThWi32g4L6Gqszi14ZtrgZCF7WXGCc8o
{
 "aaa": "asdjashdjaksdhajdh8u3893482342834",
 "ccc": "dfnsidfyu8wefhsdjfhksjdfh89ewfyddihfusiyfh8wehdfhskf",
 "bbb": "dasojdkjasdjakdj",
 "fff": "ahsacmniaoncaslkcm"
}

that being said, i confirm being unable to fetch this via ipfs.io and cloudflare-ipfs.com (on a cold try)

1 Like
18

Thank you very much for your answer. This is the result of stats provide

{
  TotalProvides: 324,
  LastReprovideBatchSize: 0,
  AvgProvideDuration: 9834743759,
  LastReprovideDuration: 0,
}

These parameters look weird. Especially the last two. LastReprovideDuration: 0 Does this mean that reprovide doesn’t work at all?

I pinned some randomly generated files.

QmYC9ne5881CudwaTmAH1BZpf1bgPM5dg3yQ54aKQhdmp8
QmRYstz3YwbGEoNDdFERgujhsy9yKshg3uhCEYdVyEuR3n
QmQGChpvTwf2c8wjaMXFbSq2gt4J5dnCN5XVTgpoShKLwe
QmQXfuEuaUrhsKqLWjS8Qjg3NDAEmG3XDB14XjYK1hs8N1

EDIT:
I added two changes. I have enabled AcceleratedDHTClient. I also increased the CPU and memory, because it was already a bit tight, and I guess AcceleratedDHTClient needs a little more resources.

After restarting the infrastructure, this is the new IP address:

100.25.119.12
19

That’s very strange indeed. It seems that your node is having problems providing to the DHT.

I tried to find provider records for all the CIDs you shared and couldn’t find any.

âžś  ~ ipfs dht findprovs  QmQXfuEuaUrhsKqLWjS8Qjg3NDAEmG3XDB14XjYK1hs8N1
âžś  ~ ipfs dht findprovs  QmYC9ne5881CudwaTmAH1BZpf1bgPM5dg3yQ54aKQhdmp8
âžś  ~ ipfs dht findprovs  QmRYstz3YwbGEoNDdFERgujhsy9yKshg3uhCEYdVyEuR3n
âžś  ~ ipfs dht findprovs  QmQGChpvTwf2c8wjaMXFbSq2gt4J5dnCN5XVTgpoShKLwe

Depending on the number of provides in needs to do this can be quite lengthy, and beyond a certain number we recommend AcceleratedDHTClient — it obviously consumes more memory and networking bandwidth as you ptobably noticed.

The results you shared seems really high. Can you try running another provide run with ipfs bitswap reprovide?

20

Works for me!

>  ~ ipfs id 12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
{
	"ID": "12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t",
	"PublicKey": "CAESIJvw7rRtdExbJGRWwEhzLLS9OhOC0Sy3+b6V9P5Te7/n",
	"Addresses": [
		"/ip4/100.26.165.184/tcp/4001/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t",
		"/ip4/100.26.165.184/udp/4001/quic-v1/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t",
		"/ip4/100.26.165.184/udp/4001/quic-v1/webtransport/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t",
		"/ip4/100.26.165.184/udp/4001/quic/p2p/12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t"
	],
	"AgentVersion": "kubo/0.24.0/e70db65",
	"Protocols": [
		"/ipfs/bitswap",
		"/ipfs/bitswap/1.0.0",
		"/ipfs/bitswap/1.1.0",
		"/ipfs/bitswap/1.2.0",
		"/ipfs/id/1.0.0",
		"/ipfs/id/push/1.0.0",
		"/ipfs/kad/1.0.0",
		"/ipfs/lan/kad/1.0.0",
		"/ipfs/ping/1.0.0",
		"/libp2p/autonat/1.0.0",
		"/libp2p/circuit/relay/0.2.0/hop",
		"/libp2p/circuit/relay/0.2.0/stop",
		"/libp2p/dcutr",
		"/x/"
	]
}

>  ~ ipfs routing findprovs QmYC9ne5881CudwaTmAH1BZpf1bgPM5dg3yQ54aKQhdmp8     
12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
>  ~ ipfs routing findprovs QmRYstz3YwbGEoNDdFERgujhsy9yKshg3uhCEYdVyEuR3n
12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
>  ~ ipfs routing findprovs QmQGChpvTwf2c8wjaMXFbSq2gt4J5dnCN5XVTgpoShKLwe
12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
>  ~ ipfs routing findprovs QmQXfuEuaUrhsKqLWjS8Qjg3NDAEmG3XDB14XjYK1hs8N1
12D3KooWLK6ShKEFb9D1eQinsFNb12i5VHrzKnwDY4aHNr5LGc1t
>  ~ ipfs cat QmYC9ne5881CudwaTmAH1BZpf1bgPM5dg3yQ54aKQhdmp8
[
  {
    "_id": "656585a75ccc91347236e4fb",
    "index": 0,
    "guid": "cd38a07a-e326-4990-af50-f32c7f8f24dd",
    "isActive": true,
    "balance": "$2,210.12",
    "picture": "http://placehold.it/32x32",
    "age": 37,
    "eyeColor": "brown",
    "name": "Phyllis Tran",
    "gender": "female",
...
      {
        "id": 2,
        "name": "Vance Robbins"
      }
    ],
    "greeting": "Hello, Sweeney Boone! You have 6 unread messages.",
    "favoriteFruit": "banana"
  }
]
>  ~
1 Like