Change User Agent When Adding urlSource

I have a function to add files in js-ipfs via URL.

async function addFile(url) {
  try {
    for await (const file of node.add(urlSource(url))) {
      console.log('Hash for ' + url.substring(url.lastIndexOf('/') + 1));
    }
  }
  catch(e) {
    console.error('Problem downloading: ' + e.toString());
  }
}

Some URLs that I pass in work great, while others will serve a file only to certain user agents. Is there a way to specify a custom user agent of “Wget/” on the HTTP GET requests done by IPFS urlSource?

Edit: After poking around the source of js-ipfs a bit, it looks like it’s using node-fetch to make the request. I tried changing the line to:

for await (const file of node.add(urlSource(url, {"headers": ["User-Agent", "Wget/"]}))) {

and

for await (const file of node.add(urlSource(url, {"headers": {"User-Agent": "Wget/"}}))) {

based off of the node-fetch docs, but no dice. Webhook.site still shows my requests having a user-agent of node-fetch/1.0 (+https://github.com/bitinn/node-fetch).

The source for urlSource is here - it doesn’t look like it’s passing the options to the HTTP constructor, that would allow you to specify headers in the request.

Would you like to make a quick PR with a test to the ipfs/js-ipfs-utils repo to enable this functionality?

Thanks for helping me track that down! I had missed that somehow. Pull request opened!