Config for websocket star using rendezvous

simonk · August 27, 2018, 3:09pm

Hi,

I’m trying to use js-ipfs programmatically, from Node.js, to share files between two nodes (processes) on the same machine only via Websockets using a rendezvous server running on localhost. In my case I also don’t want the nodes to listen on any ports and only make the connections via the rendezvous server to simulate the final set up where the nodes will be inaccessible from each other. I’ve read through the readme files but am confused as to the necessary config for the config.Bootstrap array setting as well as the Addresses.Swarm setting.

In the rendezvous tests it refers to an address of the form “/ip4/127.0.0.1/tcp/9090/ws/p2p-websocket-star/ipfs/”. Is the id here the peer id of the connecting node? So each process will supply a different ID generated using create() from the peer-id package?

Is this address what I put for the bootstrap node or Addresses.Swarm? If I run the rendezvous server on localhost what do I put for both settings?

Thanks for any help!
Simon

VictorBjelkholm · August 27, 2018, 3:29pm

If you’re not planning to connect to js-ipfs nodes running in the browser, you don’t need the rendezvous server, you can simply directly connect the nodes over TCP with either setting the config.Bootstrap array to have the Multiaddr of the other node or do jsipfs swarm connect <multiaddr> (node.swarm.connect(<multiaddr>, cb) in code) where Multiaddr is the other node.

The addresses in Addresses.Swarm is which addresses you listen on locally, whereas bootstrap are addresses you connect to when you start the node.

simonk · August 28, 2018, 1:12am

Hi Victor,

Thanks for the input.

Ultimately I am looking to be able share files between PCs that are sitting on different firewalled networks (such as on different corporate networks) where the only knowledge of each other is the hashes of the files they wish to share. In these situations you can only really guarantee that each node can make outgoing connections to the Internet and only via HTTP(S) on standard ports (80/443). If I’m understanding how IPFS works correctly, then an architecture that I believe would work is that each node would connect to a specific websocker start rendezvous server running in AWS, and exposed on port 443…
At the moment I’m just trying to do this in a local network as it’s simpler for initial testing purposes. The docs for the rendezvous server don’t really talk about the necessary config and I’m struggling to figure it out. What you’re saying in terms of the addresses makes sense to me in terms of direct connections but I’m not sure how this translates to the websocket star rendezvous server approach.
Thanks,
Simon

simonk · August 29, 2018, 12:26pm

I figured it out so am putting the solution here in case it’s useful for anyone else. Assuming you are running js-libp2p-websocket-star-rendezvous with the following command line on localhost:

rendezvous --port=9090 --host=127.0.0.1

You can then connect to it using the following code with JS IPFS:

const ipfsConfig = {
  repo: 'ipfs-' + Math.random(),
  config: {
    Addresses: {
      Swarm: ['/ip4/127.0.0.1/tcp/9090/ws/p2p-websocket-star']
    },
    "Bootstrap": [
    ],     
  }
};
const ipfs = new IPFS(ipfsConfig);

That is the address you specify for Addresses.swarm in the address of the rendezvous server. i.e. it not what you listen on but what others can connect to in order to communicate with you. No bootstrap addresses are required.

If you have two running instances of a NodeJS app with this code they will form a swarm and have each other as peers.

tobowers · April 7, 2020, 4:42pm

in later versions you get this: Unhandled Rejection (Error): no valid addresses were provided for transports [WebSockets,WebRTCStar,Circuit] with this config

haydenyoung · April 27, 2020, 2:20pm

I am experiencing the same issue, “(node:16090) UnhandledPromiseRejectionWarning: Error: no valid addresses were provided for transports [TCP,WebSockets,Circuit]”

IPFS v0.43.0

I do not experience the same problems with earlier versions such as 0.40.

My IPFS config:

{
          EXPERIMENTAL: {
            pubsub: true
          },
          "config": { 
            "Addresses": { 
              "Swarm": ['/ip4/127.0.0.1/tcp/9090/wss/p2p-webrtc-star'], 
              "Bootstrap": [] 
            }
         }
       }

I’m running my own webrtc server using webrtc-star.

achingbrain · May 1, 2020, 3:13pm

There are a few things going on here:

The error message is correct, there is no valid address there - there is a WebSocket-Star address but that transport has not been configured, the reason being it was not ported from callbacks to async/await to be made compatible with libp2p@0.27.0. The intention is to replace it with libp2p-stardust, please see that repo for more or for how to help out with that effort.

You are trying to configure IPFS to listen on a webrtc-star address under node.js. The transport is not configured for node by default, only the browser.

The webrtc-star transport exists to solve a specific problem - that browser nodes cannot open ports so are not diallable. Instead for one browser node to contact another, the connection negotiation is done via a webrtc-star signalling server that mediates between the two nodes.

node.js on the other hand can open ports so you should use the tcp or websocket transports, eg:

const ipfs = await IPFS.create({
  config: {
    Addresses: {
      Swarm: [
        '/ip4/127.0.0.1/tcp/9090/wss', // <-- use the websocket transport
        '/ip4/127.0.0.1/tcp/9091'      // <-- use the tcp transport
      ]
    }
  }
})

If you want to configure IPFS under node.js to dial a browser over WebRTC, take a look at the custom-libp2p example for how to configure extra transport/peer discovery mechanisms like WebRTC and also how js-ipfs does it when you start it as a daemon. Then you should be able to use ipfs.swarm.connect to dial IPFS running in a browser from node.

phillmac · May 24, 2020, 8:16am

It doesn’t seem possible to use libp2p-stardust as per the custom-libp2p example, when compared to the example in the libp2p-stardust which requires a listen address that includes the node’s peer id.
How can you specify the id in the address when the id hasn’t yet been determined by ifps?
e.g.

const ipfs = await Ipfs.create({
  libp2p: libp2pBundle,
  config: {
    Addresses: {
      Swarm: ['/dns4/stardust.mkg20001.io/tcp/443/wss/p2p-stardust/ipfs/<my id here>'],
      API: '',
      Gateway: ''
    }
  }
})

Ipfs needs to generate or load an id from the local browser storage, it’s impossible to specify it in ipfs config that’s required to create the ipfs node

What am I missing here?

vasco-santos · May 25, 2020, 9:31am

Hello @phillmac

In the IPFS configuration, you do not need to specify you own peer id, since IPFS puts it before the PeerInfo is sent to libp2p: https://github.com/ipfs/js-ipfs/blob/master/packages/ipfs/src/core/components/start.js#L39-L53

One important thing you should consider is that the stardust signalling server on the dns multiaddr you have is probably not up to date. You should run your own signalling server instead. It is detailed in stardust readme how to do that, and you can also use the image available in https://hub.docker.com/r/libp2p/js-libp2p-stardust

FYI, I am working on libp2p rendezvous and I intend to create next month a full guide on how to run IPFS/libp2p in the browser using all the available modules we have.

phillmac · May 25, 2020, 10:27am

If I could get an error about a bad server version that’d be great! However if I leave the /ipfs/ bit out, and after a bit of debugging in transport-manager.js I get a error like this

[
  {
    isFulfilled: false,
    isRejected: true,
    reason: TypeError: Cannot read property '1' of undefined
        at module.exports.getStardustMultiaddr (/workspace/ipfs-bundle-t/node_modules/libp2p-stardust/src/utils.js:16:19)
        at Listener.connectServer (/workspace/ipfs-bundle-t/node_modules/libp2p-stardust/src/listener.js:120:77)
        at Listener.listen (/workspace/ipfs-bundle-t/node_modules/libp2p-stardust/src/listener.js:70:18)
        at TransportManager.listen (/workspace/ipfs-bundle-t/node_modules/libp2p/src/transport-manager.js:152:29)
        at Libp2p._onStarting (/workspace/ipfs-bundle-t/node_modules/libp2p/src/index.js:382:33)
        at Libp2p.start (/workspace/ipfs-bundle-t/node_modules/libp2p/src/index.js:202:18)
        at start (/workspace/ipfs-bundle-t/node_modules/ipfs/src/core/components/start.js:63:18)
        at async main (/workspace/ipfs-bundle-t/src/test.js:108:16)
  }
]
(node:4221) UnhandledPromiseRejectionWarning: Error: Transport (stardust) could not listen on any available address
    at TransportManager.listen (/workspace/ipfs-bundle-t/node_modules/libp2p/src/transport-manager.js:169:23)
    at async Libp2p._onStarting (/workspace/ipfs-bundle-t/node_modules/libp2p/src/index.js:382:5)
    at async Libp2p.start (/workspace/ipfs-bundle-t/node_modules/libp2p/src/index.js:202:7)
    at async start (/workspace/ipfs-bundle-t/node_modules/ipfs/src/core/components/start.js:63:5)
    at async main (/workspace/ipfs-bundle-t/src/test.js:108:16)

And if I debug ma.toString() var at

github.com

libp2p/js-libp2p-stardust/blob/6215daf4aae866642c46325fd1dfa9ff95fc6ae5/src/utils.js#L15


const { CODE_P2P } = require('./constants')

module.exports.getStardustMultiaddr = (ma) => {
  const tuples = ma.stringTuples().filter((tuple) => {
    if (tuple[0] === CODE_P2P) {
      return true
    }
  })

  // Get first id
  return tuples[0][1]
}

module.exports.sha5 = (data) => crypto.createHash('sha512').update(data).digest()

I get "/dns4/stardust.mkg20001.io/tcp/443/wss/p2p-stardust"
So hence why I guessed I needed to include my own /p2p/ or /ipfs/ section because it wasn’t being handled for me. I even tried leaving Addresses.Swarm empty and then doing this:

const listenAddr = `/dns4/stardust.mkg20001.io/tcp/443/wss/p2p-stardust/p2p/${(await node.id()).id}`
await node.libp2p.transportManager.listen(listenAddr)

which revealed a few other spots where the passed multiaddr objects didn’t conform to what was expected, (missing encapsulate & decapsulate functions)
Ultimately it’d barf an error:

[
  {
    isFulfilled: false,
    isRejected: true,
    reason: AggregateError: 
        Error: No transport available for address /dns4/stardust.mkg20001.io/tcp/443/wss/p2p/Qme4QVLei8c4zd8P4dvax3BATz1iwXydTAPRWfvnTJYwWt
            at TransportManager.dial (/workspace/ipfs-bundle-t/node_modules/libp2p/src/transport-manager.js:83:21)
            at DialRequest.dialAction (/workspace/ipfs-bundle-t/node_modules/libp2p/src/dialer/index.js:140:36)
            at /workspace/ipfs-bundle-t/node_modules/libp2p/src/dialer/dial-request.js:58:29
            at async /workspace/ipfs-bundle-t/node_modules/p-some/index.js:53:19
        at maybeSettle (/workspace/ipfs-bundle-t/node_modules/p-some/index.js:31:11)
        at /workspace/ipfs-bundle-t/node_modules/p-some/index.js:69:23
  }
]

So I gave up in disgust.

I’m greatly looking forward to a better solution to this whole quagmire of transport issues and a coherent guide on how to properly configure a browser based ipfs node with the recent api changes would be greatly appreciated.

P.S. package.json :

"dependencies": {
    "ipfs": "^0.44.0",
    "libp2p": "^0.27.8",
    "libp2p-stardust": "github:libp2p/js-libp2p-stardust#v0.2.0-rc.0",
  }

vasco-santos · May 25, 2020, 12:06pm

If I could get an error about a bad server version that’d be great

Totally! AFAIK the deployed version is a really older version.

So, the error with the tuples[0][1] happens because js-libp2p does not support resolving dns multiaddrs yet (Context: Support .nodeAddress() for /dnsaddr · Issue #94 · multiformats/js-multiaddr · GitHub). As a consequence, stardust will not be able to get the signalling server peer ID.

For both reasons mentioned, I recommend that you go with your own signalling server for now. I have this old WIP PR integrating stardust in js-ipfs that you can have a look. It is outdated now, but the needed ideas are there.

vasco-santos · May 25, 2020, 12:07pm

Our next goal is to remove the need for this signalling servers (https://github.com/libp2p/js-libp2p/issues/385). With libp2p-rendezvous in place and using circuit relay nodes will be enough to run browsers nodes. But for now, I high recommend that you simply use webrtc-star if possible. Otherwise, if you need websockets, you can go with stardust, but using your own signal server.

Topic		Replies	Views
Trying orbitdb with jsipfs and webrtc js-ipfs js-ipfs , webrtc	5	310	November 22, 2022
How i can use rendezvous server in IPFS go implementation? Kubo	6	1159	December 31, 2019
Ws-star configuration options	6	1802	July 8, 2020
Unable to connect 2 js-ipfs nodes using webrtc-star in Swarm (but browser does connect) js-ipfs	2	462	December 23, 2021
Swarm peers not discovering electron, orbit -> node env Help js-ipfs	6	682	July 15, 2020

Config for websocket star using rendezvous

Related Topics