Distributions: el7 nodejs 6.4.0-1 package will not update via yum: package is not signed

Created on 17 Aug 2016  路  11Comments  路  Source: nodesource/distributions

Hi there,

Repo configured using curl -sL https://rpm.nodesource.com/setup_6.x | bash - on Amazon Linux, which set up the repo nodesource/x86_64 Node.js Packages for Enterprise Linux 7 - x86_64. Servers currently on nodejs 6.3.1 installed from there.

This morning I'm trying some upgrades, but get:

Resolving Dependencies
--> Running transaction check
---> Package nodejs.x86_64 1:6.3.1-1nodesource.el7.centos will be updated
---> Package nodejs.x86_64 1:6.4.0-1nodesource.el7.centos will be an update
--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================================
 Package             Arch                Version                                      Repository               Size
====================================================================================================================
Updating:
 nodejs              x86_64              1:6.4.0-1nodesource.el7.centos               nodesource              9.4 M

Transaction Summary
====================================================================================================================
Upgrade  1 Package

Total download size: 9.4 M
Is this ok [y/d/N]: y
Downloading packages:
Package nodejs-6.4.0-1nodesource.el7.centos.x86_64.rpm is not signed============  ] 1.6 MB/s | 8.9 MB     00:00 ETA 
nodejs-6.4.0-1nodesource.el7.centos.x86_64.rpm                                               | 9.4 MB     00:03     


Package nodejs-6.4.0-1nodesource.el7.centos.x86_64.rpm is not signed

Running a yum clean all hasn't fixed anything.

An actual repo issue, or have I done something wrong?

picture rowanbeentje
👍5

Most helpful comment

Unsigned RPM Package Builds 2016-08-16

Report Status

Everything is back to normal, things are now working as expected.

Executive Summary

A manual misconfiguration caused the rpm packages for the 4.5.0 and 6.4.0 builds of Node.js to be pushed into our repositories. This caused errors with the normal operation of yum for anybody who is using our repositories and tried to install / upgrade to these versions.

Outage Description

On Thursday Aug 11th, it was discovered that there was a vulnerability with the rpm.nodesource.com server we had been using as a download server for rpm packages. The vulnerability was with openssl itself, meaning it affected most cryptographic things on the server, such as the SSL certificate. This was discovered by an end user who noted that we were getting a failing score for our SSL certificate via SSLLabs. We decided the situation was severe enough to warrant immediate action, so @chrislea migrated everything to a new server that does not have the issue with openssl (SSL Labs report).

On this new server, it was decided to use the default fedora user as the sudo user which is what we have been doing recently, instead of creating a ns user as we have historically. Unfortunately, the location of the signing key was hardcoded to the home directory of the ns user in a configuration file which @chrislea forgot to update appropriately. This caused the signing process to fail when the builds for 4.5.0 and 6.4.0 were pushed out.

Related GH Issue: https://github.com/nodesource/distributions/issues/344

Affected users

Users in the community who used rpm based disributions and tried to install or update to the 4.5.0 or 6.4.0 releases.

Start Date/ Time:

2016-08-16 at approximately 3p PDT.

End Date/ Time:

2016-08-17 at approximately 9:10a PDT.

Duration:

18 hours.

Timeline

The new server for rpm.nodesource.com went online at approximately 5p PDT on 2016-08-11.

The 4.5.0 packages were pushed to the repositories at approximately 2p PDT on 2016-08-16. The 6.4.0 packages were pushed to the repositories at approximately 5p PDT on 2016-08-16.

@chrislea was notified that the issue existed at about 8:45a on 2016-08-17 and it was fixed at about 8:55a on 2016-08-17.

Contributing Conditions Analysis

We set up a server manually, which is a process we'd generally want to have automated. This was because of the urgency of the openssl vulnerability which was deemed pressing enough at the time to warrant the fastest possible route to resolution.

The issue was fixed almost immediately once it was reported to infrastructure.

Recommendations

We have a plan in place for an end-to-end pipeline for automated testing of the Node.js packages. Once this is in place, it is far less likely that this type of issue would reach the publishing stage and affect users. Even though we are currently resource-constrained, we plan on raising the priority of this effort to ensure that this issue does not occur again.

picture mweagle on 18 Aug 2016
❤3

All 11 comments

Repo-id      : nodesource/x86_64
Repo-name    : Node.js Packages for Enterprise Linux 7 - x86_64
Repo-revision: 1471387017
Repo-updated : Tue Aug 16 22:37:11 2016
Repo-pkgs    : 33
Repo-size    : 1.3 G
Repo-baseurl : https://rpm.nodesource.com/pub_6.x/el/7/x86_64
Repo-expire  : 21,600 second(s) (last: Wed Aug 17 09:52:53 2016)
Repo-filename: /etc/yum.repos.d/nodesource-el.repo
rowanbeentje picture rowanbeentje on 17 Aug 2016

Adding --nogpgcheck to the yum install command works as a workaround but not sure what the underlying issue is :(

igoratron picture igoratron on 17 Aug 2016

The whole point of having GPG signatures is to make sure the requested package is the one you're looking for instead of a potentially malicious one. nogpgcheck is basically asking for trouble.

SaltwaterC picture SaltwaterC on 17 Aug 2016

I think the best workaround without compromising GPG signatures is to stick with 6.3.1 until this issue is resolved. The repository provides multiple 6.x versions:

yum list --showduplicates nodejs
[...]
Installed Packages
nodejs.x86_64  1:6.3.1-1nodesource.el7.centos   @nodesource
Available Packages
nodejs.x86_64  6.0.0-1nodesource.el7.centos     nodesource
nodejs.x86_64  6.1.0-1nodesource.el7.centos     nodesource
nodejs.x86_64  6.2.0-1nodesource.el7.centos     nodesource
nodejs.x86_64  6.2.1-1nodesource.el7.centos     nodesource
nodejs.x86_64  6.2.2-1nodesource.el7.centos     nodesource
nodejs.x86_64  1:6.3.0-1nodesource.el7.centos   nodesource
nodejs.x86_64  1:6.3.1-1nodesource.el7.centos   nodesource
nodejs.x86_64  1:6.4.0-1nodesource.el7.centos   nodesource
SaltwaterC picture SaltwaterC on 17 Aug 2016
👍1

Same issue with the packages in the fc24 repo.

rkcpi picture rkcpi on 17 Aug 2016

It looks like all of the 6.4.0 packages pushed yesterday have no signature, I've checked:
https://rpm.nodesource.com/pub_6.x/fc/24/i386/nodejs-6.4.0-1nodesource.fc24.i686.rpm
https://rpm.nodesource.com/pub_6.x/fc/24/x86_64/nodejs-6.4.0-1nodesource.fc24.x86_64.rpm
https://rpm.nodesource.com/pub_6.x/fc/23/i386/nodejs-6.4.0-1nodesource.fc23.i686.rpm
https://rpm.nodesource.com/pub_6.x/fc/23/x86_64/nodejs-6.4.0-1nodesource.fc23.x86_64.rpm
https://rpm.nodesource.com/pub_6.x/el/6/x86_64/nodejs-6.4.0-1nodesource.el6.x86_64.rpm
https://rpm.nodesource.com/pub_6.x/el/7/x86_64/nodejs-6.4.0-1nodesource.el7.centos.x86_64.rpm

And rpm reports no signature:

rpm -qip https://rpm.nodesource.com/pub_6.x/el/7/x86_64/nodejs-6.4.0-1nodesource.el7.centos.x86_64.rpm
Name        : nodejs
Epoch       : 1
Version     : 6.4.0
Release     : 1nodesource.el7.centos
Architecture: x86_64
Install Date: (not installed)
Group       : Development/Languages
Size        : 34157711
License     : MIT and ASL 2.0 and ISC and BSD
Signature   : (none)
Source RPM  : nodejs-6.4.0-1nodesource.el7.centos.src.rpm
Build Date  : Tue 16 Aug 2016 10:08:31 PM UTC
Build Host  : ip-172-30-90-73.us-west-2.compute.internal
Relocations : (not relocatable)
URL         : http://nodejs.org
Summary     : JavaScript runtime
Description :
Node.js is a platform built on Chrome\'s JavaScript runtime
for easily building fast, scalable network applications.
Node.js uses an event-driven, non-blocking I/O model that
makes it lightweight and efficient, perfect for data-intensive
real-time applications that run across distributed devices.
skaymakca picture skaymakca on 17 Aug 2016
👍1

Hi everyone. Our operations team is aware of the issue and is investigating. We'll fix the problem as soon as possible and apologize for the inconvenience.

kstewart picture kstewart on 17 Aug 2016
👍1

Thanks for reporting this @rowanbeentje - we've updated the repository with signing information and you should be able to securely install now.

I will write up a more detailed report in the next couple of days with more information on the outage.

Apologies for the inconvenience @all. Thanks.

mweagle picture mweagle on 17 Aug 2016

Thanks for taking care of this. Cheers!

SaltwaterC picture SaltwaterC on 17 Aug 2016

All working now; thanks very much @mweagle and @kstewart!

rowanbeentje picture rowanbeentje on 17 Aug 2016

Unsigned RPM Package Builds 2016-08-16

Report Status

Everything is back to normal, things are now working as expected.

Executive Summary

A manual misconfiguration caused the rpm packages for the 4.5.0 and 6.4.0 builds of Node.js to be pushed into our repositories. This caused errors with the normal operation of yum for anybody who is using our repositories and tried to install / upgrade to these versions.

Outage Description

On Thursday Aug 11th, it was discovered that there was a vulnerability with the rpm.nodesource.com server we had been using as a download server for rpm packages. The vulnerability was with openssl itself, meaning it affected most cryptographic things on the server, such as the SSL certificate. This was discovered by an end user who noted that we were getting a failing score for our SSL certificate via SSLLabs. We decided the situation was severe enough to warrant immediate action, so @chrislea migrated everything to a new server that does not have the issue with openssl (SSL Labs report).

On this new server, it was decided to use the default fedora user as the sudo user which is what we have been doing recently, instead of creating a ns user as we have historically. Unfortunately, the location of the signing key was hardcoded to the home directory of the ns user in a configuration file which @chrislea forgot to update appropriately. This caused the signing process to fail when the builds for 4.5.0 and 6.4.0 were pushed out.

Related GH Issue: https://github.com/nodesource/distributions/issues/344

Affected users

Users in the community who used rpm based disributions and tried to install or update to the 4.5.0 or 6.4.0 releases.

Start Date/ Time:

2016-08-16 at approximately 3p PDT.

End Date/ Time:

2016-08-17 at approximately 9:10a PDT.

Duration:

18 hours.

Timeline

The new server for rpm.nodesource.com went online at approximately 5p PDT on 2016-08-11.

The 4.5.0 packages were pushed to the repositories at approximately 2p PDT on 2016-08-16. The 6.4.0 packages were pushed to the repositories at approximately 5p PDT on 2016-08-16.

@chrislea was notified that the issue existed at about 8:45a on 2016-08-17 and it was fixed at about 8:55a on 2016-08-17.

Contributing Conditions Analysis

We set up a server manually, which is a process we'd generally want to have automated. This was because of the urgency of the openssl vulnerability which was deemed pressing enough at the time to warrant the fastest possible route to resolution.

The issue was fixed almost immediately once it was reported to infrastructure.

Recommendations

We have a plan in place for an end-to-end pipeline for automated testing of the Node.js packages. Once this is in place, it is far less likely that this type of issue would reach the publishing stage and affect users. Even though we are currently resource-constrained, we plan on raising the priority of this effort to ensure that this issue does not occur again.

mweagle picture mweagle on 18 Aug 2016
❤3
Was this page helpful?
0 / 5 - 0 ratings

Related issues

xNarkon picture xNarkon  路  3Comments
alexcleu picture alexcleu  路  3Comments
muhhizbe picture muhhizbe  路  6Comments
bewam picture bewam  路  4Comments
MichaelBitard picture MichaelBitard  路  4Comments