Nomad: Nomad inproperly unpack tar.gz artifact archives with symlinks

[tantra35]

Nomad version

Nomad v0.5.4

We've made artifact with many symlinks in it. Is our case this links goes to so(shared objects), like in this sample:

lrwxrwxrwx 1 root root      18 Feb  8 15:40 libcrypto.so -> libcrypto.so.1.0.0
-r-xr-xr-x 1 root root 2308736 Feb  8 14:42 libcrypto.so.1.0.0
lrwxrwxrwx 1 root root      17 Feb  8 15:40 libcurses.so -> libncurses.so.5.9
-rwxrwxr-x 1 root root  142920 Feb  8 15:41 libncurses.so.5.9
lrwxrwxrwx 1 root root      21 Feb  8 15:40 libecpg_compat.so -> libecpg_compat.so.3.5
lrwxrwxrwx 1 root root      21 Feb  8 15:40 libecpg_compat.so.3 -> libecpg_compat.so.3.5
-rwxr-xr-x 1 root root   19024 Feb  8 14:42 libecpg_compat.so.3.5

Then in job file we've specifed follow atrifact declaration:

            {
                source = "http://docker.local/td-agent.3.tar.gz"
            }

Artifact was successfully downloaded, and was extracted, but the contents of extracted dir, differs from
that which was in tar.gz archive:

-rwxrwxrwx 1 root root       0 Feb  8 15:41 libcrypto.so
-r-xr-xr-x 1 root root 2308736 Feb  8 15:41 libcrypto.so.1.0.0
-rwxrwxrwx 1 root root       0 Feb  8 15:41 libcurses.so
-rwxrwxr-x 1 root root  142920 Feb  8 15:41 libncurses.so.5.9
-rwxrwxrwx 1 root root       0 Feb  8 15:41 libecpg_compat.so
-rwxrwxrwx 1 root root       0 Feb  8 15:41 libecpg_compat.so.3
-rwxr-xr-x 1 root root   19024 Feb  8 15:41 libecpg_compat.so.3.5

As you can see, all symlinks after extract turn into simple files with zero length

[tantra35]

In go-getter lib we doesn't find any flags that responds for such behavior, so as quick workaround we pack our archives with dereference (-h) option

[dadgar]

Need to investigate whether this allows the chroot to be escaped

[imcitius]

Hello

Is there any news on this matter?
0.8.4 still has the same bug.

[shantanugadgil]

This issue is visible using Nomad 0.9.0-beta3 as well, with the raw_exec driver.

The symlink itself is gone, and the symlinks themselves become zero bytes files! :disappointed:

The use case is to download a largish app (which uses NVIDIA CUDA), extract it and run it.

The app also has OS packages, but I want to _avoid_ that to be able to use Nomad to run multiple different versions on the same machine.

[cgbaker]

thanks @shantanugadgil , I've verified this on 0.9.0-beta3 and I'm looking into it right now.

[cgbaker]

Okay, this is apparently a well-known issue in go-getter that has not been fixed yet:
https://github.com/hashicorp/go-getter/issues/60
There are some workarounds in that ticket, just FYI, but they're not universally applicable.

[cgbaker]

The associated issue has been fixed in go-getter, however, before we incorporate this fix into Nomad, we have to make sure that symlinks (malicious or otherwise) don't allow user workload to "escape" the filesystem. I've brought this up with the team and we've file an issue to address this.

[stale[bot]]

Hey there

Since this issue hasn't had any activity in a while - we're going to automatically close it in 30 days. If you're still seeing this issue with the latest version of Nomad, please respond here and we'll keep this open and take another look at this.

Thanks!

[schmichael]

Still open, see #5446 for details

[shantanugadgil]

@cgbaker any update/followup on the go-getter and thus this issue? This would be super useful addition and seems something _obvious_ to have. :smile:

As per the workarounds its not a complete blocker, though would be great to be functional directly via Nomad. :+1:

[cgbaker]

This is definitely something that we want to address, but it is not a priority. Feel free to keep pinging this issue.

[stale[bot]]

Hey there

Since this issue hasn't had any activity in a while - we're going to automatically close it in 30 days. If you're still seeing this issue with the latest version of Nomad, please respond here and we'll keep this open and take another look at this.

Thanks!

[shantanugadgil]

to keep the stale bot from closing this.

[stale[bot]]

Hey there

Since this issue hasn't had any activity in a while - we're going to automatically close it in 30 days. If you're still seeing this issue with the latest version of Nomad, please respond here and we'll keep this open and take another look at this.

Thanks!