use proxy defined in HTTP_PROXY environment variable

[floriannari]

This PR deletes axios ( prolonging PR #29 ) and uses https://github.com/nodejs/undici/blob/main/docs/docs/api/EnvHttpProxyAgent.md to use a proxy according to what is defined in the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables.

PS: I haven't updated https://github.com/sapics/ip-location-api/tree/main/cjs

Possible improvement to avoid a potential breaking change: add a setting like ILA_ENV_HTTP_PROXY to specify whether to use the EnvHttpProxyAgent dispatcher, or the default one.

[sapics]

Thanks for PR!

I agree with the direction of removing axios and unifying to fetch, especially since consistent behavior under proxy environments seems important.

However, it looks like the current PR causes an error on older versions of Node.js.
If it's possible to address this, a fix would be appreciated.
I'll also take a look when I have time.

[sapics]

Could you try setting the http_proxy or https_proxy environment variable with the current version?

It might work, since axios also supports proxy configuration via environment variables, as described in their documentation:
https://github.com/axios/axios?tab=readme-ov-file#request-config

  // `proxy` defines the hostname, port, and protocol of the proxy server.
  // You can also define your proxy using the conventional `http_proxy` and
  // `https_proxy` environment variables. If you are using environment variables
  // for your proxy configuration, you can also define a `no_proxy` environment
  // variable as a comma-separated list of domains that should not be proxied.
  // Use `false` to disable proxies, ignoring environment variables.
  // `auth` indicates that HTTP Basic auth should be used to connect to the proxy, and
  // supplies credentials.
  // This will set an `Proxy-Authorization` header, overwriting any existing
  // `Proxy-Authorization` custom headers you have set using `headers`.
  // If the proxy server uses HTTPS, then you must set the protocol to `https`.
  proxy: {
    protocol: 'https',
    host: '127.0.0.1',
    // hostname: '127.0.0.1' // Takes precedence over 'host' if both are defined
    port: 9000,
    auth: {
      username: 'mikeymike',
      password: 'rapunz3l'
    }
  },

[floriannari]

Hello,
unfortunately this does not work for us because Axios does not support connecting to an HTTPS service through an HTTP proxy.
axios/axios#4531
(whereas it works without a hitch with fetch)

[floriannari]

About stream.Readable.fromWeb which requires Node.js 17, note that global fetch (which is already used by ip-location-api) already seems to require relatively recent Nodejs according to the documentation


(I haven't tested it, so maybe I'm missing something)

[sapics]

Thanks to your explanation, I now understand that http_proxy may not work correctly in some cases.

I had forgotten this myself, but the reason I chose axios over fetch was to maintain compatibility with node.js v14.
The use of fetch was intentionally limited to cases where the system generates the database or when accessed from the browser.
I'm now considering several options, such as raising the minimum required node.js version or switching to node-fetch, among other possibilities.

[floriannari]

or switching to node-fetch

for information, the fetch included in recent nodejs is also available as a library https://github.com/nodejs/undici (undici v5.x is still maintained and is compatible with nodejs >= 14).

Regarding stream.Readable.fromWeb, Undici's request function returns a body which has pipe and on("end", listener) functions. I think this could replace the use of stream.Readable.fromWeb?

Unfortunately I don't have the time to code/test this,
but I think that replacing axios and fetch with Undici v5's request function would make it compatible with NodeJs 14, while satisfying our needs for proxification via environment variables.

(In addition, undici request is 2 to 3 times faster than fetch or axios)

[floriannari]

I've noticed that stream/promises isn't compatible with NodeJs 14 either.
As Nodejs 14 is completely obsolete, I've dropped compatibility with this version of node

But if you prefer, I've also made a branch that supports Nodejs 14 https://github.com/floriannari/ip-location-api/tree/adapt_to_nodejs_14 (using stream instead of stream/promises)

(in any case, I've reverted floriannari@41c9839 because EnvHttpProxyAgent requires Undici 6.x, which requires nodejs 18.
(But if you decide to drop nodejs 16 support (knowing that ip-location-api already didn't fully work on nodejs 16 because it used global fetch), we'd be happy to see a similar commit))

[sapics]

Have you tried using global-agent, as pointed out in axios/axios#4531?

I also tried creating a fetch-version branch of the project to experiment with fetch.
However, I'm not satisfied with this approach because it introduces global changes, which I'd prefer to avoid.

[floriannari]

Hello,
indeed global-agent works for ip-location-api.
Unfortunately, it also causes side effects and causes an essential component of our application to dysfunction...

Concerning the fetch-version branch, it's a good solution. But as it is, it doesn't work because the res.body.pipe(ws) function exists in undici request, but not in (node-)fetch.

(Note that I've run your unit tests ( https://github.com/sapics/ip-location-api/blob/main/.github/workflows/test.yml#L9 ) on my https://github.com/floriannari/ip-location-api/commits/adapt_to_nodejs_14/ and https://github.com/floriannari/ip-location-api/commits/drop_nodejs14_support/ branches, and that I've also tested these 2 branches on our servers. )
(I don't know if you've noticed, but I've completely changed the first commit compared to the initial PR.)

However, I'm not satisfied with this approach because it introduces global changes, which I'd prefer to avoid.

I understand your apprehension about modifying a central function of your library, and it's totally legitimate.
There's no urgency, in the meantime our package.json points to my fork (I just hope I don't have to maintain this fork in the long term).