Pm2: enhancement request, change gkt depedency to not use an url

[datpham23]

We have a build server thats behind a firewall w/out any internet connection. We're installing packages via a private npm registry that proxy to the internet. Since the gkt dependency is a http url npm isn't able to point to our private registry to get that package. Can we change this to a version instead of an url?

"optionalDependencies": {
"gkt": "https://tgz.pm2.io/gkt-1.0.0.tgz"
},

[vmarchaud]

Should not be a problem since its an optional dependency no ? I don't things we can make it as another form sadly.

[datpham23]

i'm declaring pm2 as a dependency in my package.json however even running with the --no-optional flag i'm still getting this error.

• node -v
  v6.9.4
• npm -v
  3.10.10
• npm install --no-optional
  npm ERR! fetch failed http://tgz.pm2.io/gkt-1.0.0.tgz
  npm WARN retry will retry, error on last attempt: Error: connect ETIMEDOUT 163.172.101.189:80

[vmarchaud]

Which pm2 version are you trying to install ?

PS : some people already have some problems with the optional dependency stopping the installation, you should checkout for closed issue about it.

[vmarchaud]

Can you try using npm install --optional false ?

[datpham23]

It seems the --no-optional flag is not recursive.
https://github.com/npm/npm/issues/14185

Is there a reason why we couldn't just use the package and version instead of the url?

"optionalDependencies": {
 "gkt": "1.0.0"
},

[vmarchaud]

It isn't a npm dependency, the url is important cause we need it to ping to our servers.
I'll see what we can do here (apart from asking npm to make a dependency manager that works)

[datpham23]

Are you sure thats not a npm dependency? I can install gkt with npm just fine.
npm i [email protected]

[vmarchaud]

Yeah but the aim isn't to install the dependency from npm, the aim is to the client to retrieve the dependency from our servers.

[datpham23]

That doesn't seem like the right approach to me for a couple reasons.

1. NPM can't proxy that request to a private registry (the problem i'm facing).
2. All your end users builds and deployments are reliant on the https://tgz.pm2.io server being up.
3. Private npm registries can't cache that package for auditing and security purposes.

[vmarchaud]

It's a optional dependency for exactly these reason, if it can't be retrieved npm shoudn't stop the download.

[cbayram]

+1

We too get our dependency via a private artifactory repo, with external HTTP(s) traffic blocked.

Little discovery:
If you have pm2 as a dependency, npm install --no-optional will fail trying to obtain the optional pm2 HTTPS resource.

However, if you initially install with --no-shrinkwrap, then generate shrinkwrap, the above command will work:
npm install --no-optional --no-shrinkwrap
npm shrinkwrap
npm install --no-optional

[orphaner]

Hello,
I am getting the same issue here. In an environment with proxy and internal registry and it's just like a pain in the * to get things working properly.
I don't understand why this dependency needs to be on your server, is there any technical reason why it can't be a real package on the npm registry and simply get rid of this ?

Anyway, I manage to install pm2 with the tip given by cbayram, but since I don't have all dependencies, will pm2 be fully featured ?

Thanks for you answer,
Nicolas

[vmarchaud]

Because the official download count is aggregated by npm and totally opaque for us. We simply have our own analytics system with this optional package.
Anyway you should have no problem using the cbayram technique.

[orphaner]

Thanks for your fast reply.
IMHO it is painfull for big companies users (usually with proxy and internal registry) and you really should document that properly.

What do you mean by "should have no problem" ? What kind of problem may I face ? Install problems ? Runtime problems ?
For the needs of my project I must be sure that everything will work properly.

Thanks, Nico

[vmarchaud]

@orphaner We know that but for us the problem is npm that doesn't work properly (because optional dependency should not fail the installation).
I mean by should that i think it will work great (because the trick is only used to not install optional dependency, and pm2 only have ikst as optional dependency) but i can't predict the future you know.
If you want i can test out if its working with this technique ?

[orphaner]

yes, npm is acting weird ; I had other issues with it in the past, IMHO yarn is doing a better work.

My first tests with pm2 looks good; I just don't know yet if it is the right tool for my needs but it is an another debate.
For the cbayram technique, my doubt is about the shrinkwrap thing not the optionnal part. I do understand well that an optionnal package used only for a download count can't be an issue. My concern is about the shrinkwrap (a concept I tried to understand few months ago without success :/ ),

[grantbrown]

The GKT package is also currently causing me to experience this issue on a docker build, fyi.

[wejendorp]

I'm hitting this issue on our CI / docker setup as well, what's the reason to depend on a private mirror instead of a public package?
(edit: sorry, re-read your reason above, but it's making it hard to ship a package that depends on pm2 (a server runtime).)

[wejendorp]

For full context, even if they've fixed the npm handling of optionalDeps, the new npm ci command will fail during build. If the developer who made package changes was able to install gkt, but the CI server is not, the package-lock cannot be satisfied.

[jeffrson]

Here's another +1:
yarn will fail to install pm2 as well (behind a firewall, using private registry). See https://github.com/yarnpkg/yarn/issues/5878#issuecomment-407903756

Yes, it might be another wrong decision for npm and yarn to forcibly scan optional dependencies for metadata. So between two wrong decisions - is there a third way to go?

IMO, as pm2 uses npm/yarn it should do whatever possible or necessary to prevent any bugs related.

Really - it took me hours to find the a deployment bug inside a corporate environment, as well as these long standing issues. Sad and frustrated.

[sethkrasnianski]

It isn't a npm dependency, the url is important cause we need it to ping to our servers.

@vmarchaud Why is that so? Couldn't this be done more transparently than a self-hosted node package which could have its source changed at an arbitrary time?

[jsmylnycky]

@vmarchaud Adding this here because it's directly related to the original request, and the dependency is now causing major issues.

tgz.pm2.io appears to be down and is currently breaking our CI environment from new builds. We include pm2 as a dependency in our project (reasons) and let start scripts handle bootstrapping the environment. Because our CI server actually uses the npm ci command, with the dependency on tgz.pm2.io due to this optionalDependency, we're no longer able to get a build to complete due:

npm ERR! code ECONNREFUSED

npm ERR! errno ECONNREFUSED

npm ERR! FetchError: request to https://tgz.pm2.io/gkt-1.0.0.tgz failed, reason: connect ECONNREFUSED 37.187.139.23:443

[jreidgreer]

Firmly agree gkt should either be removed or converted to a regular package on the npm registry. While I understand the desire for analytics, it's not worth the cost of a normal install command. I shouldn't have to remember to do some weird install steps to get around this package just because I work behind a proxy.

[framp]

+1
Your server is throwing a 503 on https://tgz.pm2.io/gkt-1.0.0.tgz and blocking all our installations

[eilinora]

+1 This is blocking deployments on our systems as well

[Unitech]

Please make sure you install latest pm2

npm install pm2@latest -g

[Unitech]

made it work again but please now use latest PM2 as this external dependency has been removed in latest PM2

[Unitech]

If you cannot upgrade to PM2 3.5.1 that does not have this module anymore, I inform you that now this package/url has been moved to cloudfront+s3