Scylla: Node_exporter is not part of Scylla packaging, failing to install in a connection-less environment

[eyalgutkind]

Scylla version (or git commit hash): Scylla 1.6.1
OS (RHEL/CentOS/Ubuntu/AWS AMI): Centos 7.3

Installing Scylla in a connection-less environment (no connection to the outside world), Scylla_setup invokes the script: node_exporter_install
which try to pull from github the relevant prometheus/node_exporter package:
curl -L https://github.com/prometheus/node_exporter/releases/download/$version/node_exporter-$version.linux-amd64.tar.gz -o $dir/node_exporter-$version.linux-amd64.tar.gz

and fails.
It will be useful to package the node_exporter with the other scylla packages.

[tzach]

related to #1688

[tzach]

@slivne can we package node_exporter as part of Scylla package?
The current process is fragile

[glommer]

in the past @avikivity expressed his desired of compiling node_exporter as part of our build.
Is this a hard requirement, Avi? node_exporter is a go application and pretty foreign to seastar/scylla in that.

I think it will complicate our lives for little gain (not to mention another submodule).
I propose we just download the binary and include in the rpm as part of the rpm build process.

[tarzanek]

we build node exporter and package it in to /opt/scylladb/bin now : https://copr.fedorainfracloud.org/coprs/scylladb/scylla-3rdparty/package/scylla-node_exporter/

[amoskong]

we build node exporter and package it in to /opt/scylladb/bin now : https://copr.fedorainfracloud.org/coprs/scylladb/scylla-3rdparty/package/scylla-node_exporter/

But scylla_setup => current node_exporter_install script still downloads package from github.

• https://github.com/scylladb/scylla/blob/master/dist/common/scripts/node_exporter_install

We expect the scylla-node_exporter package is included into the unified-package, and install.sh of unified-package will install node_exporter_install package.

/CC @penberg @tzach @roydahan

[tzach]

This is true for any other packages we pull from scylla_setup (NTP?)
we can either:

• package it in Scylla
• give a clear procedure on how to install it off-line (airtight)
  Best is clearly better UX

[slivne]

@penberg / @amoskong / @syuu1228 did we solve this in the offline installer ?

[amoskong]

@penberg / @amoskong / @syuu1228 did we solve this in the offline installer ?

In offline installation, node_exporter install will be disabled by default.

[tarzanek]

to get full node exporter packaged
we need the debian package to be added (I have it ready, just waiting for permission to access our third party ubuntu repo from @syuu1228 so I can push it)
and then we need the integration of those packages into scylla of course (which I don't have and this would need guidance or a fix from Takuya or someone who owns this part of installer/reloc packaging)

[syuu1228]

Ah sorry I didn't follow this thread, I think we can easily packaging node_exporter:

1. run node_exporter_install script in source code compiling time
2. archive node_exporter binary in relocatable package (can be isolated sub-package like scylla-python3)
3. install it from install.sh
4. include it in .rpm / .deb
   In this way it will be support all relocatable package/.rpm/.deb, in minimum changes, use same method to install node_exporter on all distributions.

I'll send PR for it.

[tzach]

@syuu1228 with this, NTP is the only package scylla_setup download. Is it?

[syuu1228]

@tzach "NTP is the only package" is almost correct, but we still have few other packages:

• kernel-tools for scylla_cpuscaling_setup
• chrony or ntp, ntpdate for scylla_ntp_setup
• xfsprogs, mdadm for scylla_raid_setup

Note that we don't run yum when it's already installed, so likely the script never try to download xfsprogs and mdadm, since it supposed to install by default.