Podman: `podman run --publish 80:80 nginx` errors
/kind bug
Description
Unexpected errors while running the command:
adel@adel-pc:~/src/libpod
> podman run --publish 80:80 nginx
Trying to pull docker.io/library/nginx...Getting image source signatures
Copying blob 688a776db95f done
Copying blob 743f2d6c1f65 done
Copying blob 6bfc4ec4420a done
Copying config 53f3fd8007 done
Writing manifest to image destination
Storing signatures
ERRO[0026] Error while applying layer: ApplyLayer exit status 1 stdout: stderr: there might not be enough IDs available in the namespace (requested 0:42 for /etc/gshadow): lchown /etc/gshado
w: invalid argument
ERRO[0026] Error pulling image ref //nginx:latest: Error committing the finished image: error adding layer with blob "sha256:743f2d6c1f65c793009f30acb07845ba2ef968192732afdab2ecf9a475515393":
ApplyLayer exit status 1 stdout: stderr: there might not be enough IDs available in the namespace (requested 0:42 for /etc/gshadow): lchown /etc/gshadow: invalid argument
Failed
Trying to pull registry.fedoraproject.org/nginx...ERRO[0028] Error pulling image ref //registry.fedoraproject.org/nginx:latest: Error initializing source docker://registry.fedoraproject.org/n
ginx:latest: Error reading manifest latest in registry.fedoraproject.org/nginx: manifest unknown: manifest unknown
Failed
Trying to pull quay.io/nginx...ERRO[0031] Error pulling image ref //quay.io/nginx:latest: Error initializing source docker://quay.io/nginx:latest: Error reading manifest latest in quay.io/ngi
nx: error parsing HTTP 404 response body: invalid character '<' looking for beginning of value: "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n<title>404 Not Found</title>\n<h1>N
ot Found</h1>\n<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>\n"
Failed
Trying to pull registry.access.redhat.com/nginx...ERRO[0032] Error pulling image ref //registry.access.redhat.com/nginx:latest: Error initializing source docker://registry.access.redhat.com/n
ginx:latest: Error reading manifest latest in registry.access.redhat.com/nginx: name unknown: Repo not found
Failed
Trying to pull registry.centos.org/nginx...ERRO[0033] Error pulling image ref //registry.centos.org/nginx:latest: Error initializing source docker://registry.centos.org/nginx:latest: Error re
ading manifest latest in registry.centos.org/nginx: manifest unknown: manifest unknown
Failed
Error: unable to pull nginx: 5 errors occurred:
* Error committing the finished image: error adding layer with blob "sha256:743f2d6c1f65c793009f30acb07845ba2ef968192732afdab2ecf9a475515393": ApplyLayer exit status 1 stdout: stderr
: there might not be enough IDs available in the namespace (requested 0:42 for /etc/gshadow): lchown /etc/gshadow: invalid argument
* Error initializing source docker://registry.fedoraproject.org/nginx:latest: Error reading manifest latest in registry.fedoraproject.org/nginx: manifest unknown: manifest unknown
* Error initializing source docker://quay.io/nginx:latest: Error reading manifest latest in quay.io/nginx: error parsing HTTP 404 response body: invalid character '<' looking for begi
nning of value: "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n<title>404 Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was not found on the server. If you entered
the URL manually please check your spelling and try again.</p>\n"
* Error initializing source docker://registry.access.redhat.com/nginx:latest: Error reading manifest latest in registry.access.redhat.com/nginx: name unknown: Repo not found
* Error initializing source docker://registry.centos.org/nginx:latest: Error reading manifest latest in registry.centos.org/nginx: manifest unknown: manifest unknown
Additional information you deem important (e.g. issue happens only occasionally):
100% reproducible
Output of podman version:
Version: 1.3.2-dev
RemoteAPI Version: 1
Go Version: go1.12.4
Git Commit: bc7afd6d71da4173e4894ff352667a25987fa2ea
Built: Tue May 28 20:50:47 2019
OS/Arch: linux/amd64
Output of podman info --debug:
debug: [10/641]
compiler: gc
git commit: bc7afd6d71da4173e4894ff352667a25987fa2ea
go version: go1.12.4
podman version: 1.3.2-dev
host:
BuildahVersion: 1.9.0-dev
Conmon:
package: Unknown
path: /usr/bin/conmon
version: 'conmon version , commit: 8fba206232c249a8fc4e2fac1469fb2fddbf5cf7'
Distribution:
distribution: manjaro
version: unknown
MemFree: 2967359488
MemTotal: 7769694208
OCIRuntime:
package: Unknown
path: /usr/bin/runc
version: |-
runc version 1.0.0-rc8
commit: 425e105d5a03fabd737a126ad93d62a9eeede87f
spec: 1.0.1-dev
SwapFree: 9448923136
SwapTotal: 9448923136
arch: amd64
cpus: 4
hostname: adel-pc
kernel: 4.19.45-1-MANJARO
os: linux
rootless: true
uptime: 3h 48m 10.33s (Approximately 0.12 days)
registries:
blocked: null
insecure: null
search:
- docker.io
- registry.fedoraproject.org
- quay.io
- registry.access.redhat.com
- registry.centos.org
store:
ConfigFile: /home/adel/.config/containers/storage.conf
ContainerStore:
number: 0
GraphDriverName: vfs
GraphOptions: null
GraphRoot: /home/adel/.local/share/containers/storage
GraphStatus: {}
ImageStore:
number: 0
RunRoot: /tmp/1000
VolumePath: /home/adel/.local/share/containers/storage/volumes
adel-mamin
All 17 comments
Additional info:
> ls /proc/self/ns; cat /proc/self/uid_map
cgroup ipc mnt net pid pid_for_children user uts
0 0 4294967295
Non priv users are not allowed to bind to ports < 1024, so this requires root.
We document it here. But we should document it in the man page, and it would be nice to print a nice error to the user.
https://github.com/containers/libpod/blob/master/rootless.md
rhatdan
on 29 May 2019
This also seems to be having trouble pulling down images... Could be a
missing entry in uidmap/gidmap
On Wed, May 29, 2019, 08:33 Daniel J Walsh notifications@github.com wrote:
Non priv users are not allowed to bind to ports < 1024, so this requires
root.We document it here. But we should document it in the man page, and it
would be nice to print a nice error to the user.https://github.com/containers/libpod/blob/master/rootless.md
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/containers/libpod/issues/3222?email_source=notifications&email_token=AB3AOCG6ZTCANPOS55XS2WTPXZZ25A5CNFSM4HQL22K2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWPFUDA#issuecomment-496917004,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB3AOCBWD2C2ANXEI7V6BVLPXZZ25ANCNFSM4HQL22KQ
.
mheon
on 29 May 2019
This is what I get when I run it.
$ podman run --publish 80:80 nginx
Trying to pull docker.io/library/nginx...Getting image source signatures
Copying blob 743f2d6c1f65 done
Copying blob 6bfc4ec4420a done
Copying blob 688a776db95f done
Copying config 53f3fd8007 done
Writing manifest to image destination
Storing signatures
Error: error from slirp4netns while setting up port redirection: map[desc:bad request: add_hostfwd: slirp_add_hostfwd failed]
if we want to improve the error message we should do it in slirp4netns, and print a nicer one when it gets EPERM trying to using a port
giuseppe
on 29 May 2019
From the original error: : there might not be enough IDs available in the namespace (requested 0:42 for /etc/gshadow): lchown /etc/gshadow: invalid argument
That doesn't look like slirp to me.
@adel-mamin Can you cat /etc/subuid and /etc/subgid and check if they're empty?
mheon
on 29 May 2019
(We should 100% update the slirp error message though)
mheon
on 29 May 2019
@adel-mamin Can you cat /etc/subuid and /etc/subgid and check if they're empty?
adel@adel-pc:~
> cat /etc/subuid
cat: /etc/subuid: No such file or directory
adel@adel-pc:~
> cat /etc/subgid
cat: /etc/subgid: No such file or directory
To verify - do you have newuidmap and newgidmap executables available on your system?
mheon
on 29 May 2019
To verify - do you have newuidmap and newgidmap executables available on your system?
adel@adel-pc:~/src/corehw/rabbit
> which newuidmap
/usr/sbin/newuidmap
adel@adel-pc:~/src/corehw/rabbit
> which newgidmap
/usr/sbin/newgidmap
Alright, so you have the executables we need, but not the configuration files they read to allocate you a range of UIDs.
Try creating /etc/subuid and /etc/subgid (they're root:root, 644 perms on my system) and adding a line like this to each:
<username>:1000000:65536
Where <username> is your username.
Podman uses newuidmap and newgidmap to get a range of UIDs and GIDs to use with rootless Podman (your own user will be mapped in as root, but we need 65k UIDs in a typical container to ensure things work properly most of the time). These are configured via /etc/subuid and /etc/subgid - adding those files and then adding a UID delegation for your user in them will allow Podman to use a full 65536 UIDs/GIDs in rootless containers and resolve this issue.
mheon
on 29 May 2019
adel@adel-pc:~
> cat /etc/subgid
adel:1000000:65536
adel@adel-pc:~
> cat /etc/subuid
adel:1000000:65536
adel@adel-pc:~
> laa /etc/subgid
4 096 -rw-r--r-- 1 root root 18 2019-05-29 16:41:23.295987915 +0300 /etc/subgid
adel@adel-pc:~
> laa /etc/subuid
4 096 -rw-r--r-- 1 root root 18 2019-05-29 16:41:37.092726609 +0300 /etc/subuid
Here is what I get for the original command:
adel@adel-pc:~
> podman run --publish 80:80 nginx
Trying to pull docker.io/library/nginx...Getting image source signatures
Copying blob 743f2d6c1f65 done
Copying blob 6bfc4ec4420a done
Copying blob 688a776db95f done
Copying config 53f3fd8007 done
Writing manifest to image destination
Storing signatures
ERRO[0015] Error while applying layer: ApplyLayer exit status 1 stdout: stderr: there might not be enough IDs available in the namespace (requested 0:42 for /etc/gshadow): lchown /etc/gshadow: invalid argument
ERRO[0015] Error pulling image ref //nginx:latest: Error committing the finished image: error adding layer with blob "sha256:743f2d6c1f65c793009f30acb07845ba2ef968192732afdab2ecf9a475515393": ApplyLayer exit status 1 stdout: stderr: there might not be enough IDs available in the namespace (requested 0:42 for /etc/gshadow): lchown /etc/gshadow: invalid argument
Failed
Trying to pull registry.fedoraproject.org/nginx...ERRO[0017] Error pulling image ref //registry.fedoraproject.org/nginx:latest: Error initializing source docker://registry.fedoraproject.org/nginx:latest: Error reading manifest latest in registry.fedoraproject.org/nginx: manifest unknown: manifest unknown
Failed
Trying to pull quay.io/nginx...ERRO[0018] Error pulling image ref //quay.io/nginx:latest: Error initializing source docker://quay.io/nginx:latest: Error reading manifest latest in quay.io/nginx: error parsing HTTP 404 response body: invalid character '<' looking for beginning of value: "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n<title>404 Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>\n"
Failed
Trying to pull registry.access.redhat.com/nginx...ERRO[0019] Error pulling image ref //registry.access.redhat.com/nginx:latest: Error initializing source docker://registry.access.redhat.com/nginx:latest: Error reading manifest latest in registry.access.redhat.com/nginx: name unknown: Repo not found
Failed
Trying to pull registry.centos.org/nginx...ERRO[0020] Error pulling image ref //registry.centos.org/nginx:latest: Error initializing source docker://registry.centos.org/nginx:latest: Error reading manifest latest in registry.centos.org/nginx: manifest unknown: manifest unknown
Failed
Error: unable to pull nginx: 5 errors occurred:
* Error committing the finished image: error adding layer with blob "sha256:743f2d6c1f65c793009f30acb07845ba2ef968192732afdab2ecf9a475515393": ApplyLayer exit status 1 stdout: stderr: there might not be enough IDs available in the namespace (requested 0:42 for /etc/gshadow): lchown /etc/gshadow: invalid argument
* Error initializing source docker://registry.fedoraproject.org/nginx:latest: Error reading manifest latest in registry.fedoraproject.org/nginx: manifest unknown: manifest unknown
* Error initializing source docker://quay.io/nginx:latest: Error reading manifest latest in quay.io/nginx: error parsing HTTP 404 response body: invalid character '<' looking for beginning of value: "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n<title>404 Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>\n"
* Error initializing source docker://registry.access.redhat.com/nginx:latest: Error reading manifest latest in registry.access.redhat.com/nginx: name unknown: Repo not found
* Error initializing source docker://registry.centos.org/nginx:latest: Error reading manifest latest in registry.centos.org/nginx: manifest unknown: manifest unknown
@giuseppe Mind taking a look here? Seems like the two files are formatted properly
mheon
on 29 May 2019
Just for clarification, is this just to change the error message so that it is more clear? I'm also only getting the error that Dan had, not the original.
ashley-cui
on 29 May 2019
I think we have two separate issues here.
- The slirp4netns error message when binding ports below 1024 is really bad
- Missing /etc/subuid and /etc/subgid leading to an inability to run containers with >1 UID/GID
The second is an environment issue, so if there are code changes to be made here, they would be the first
mheon
on 29 May 2019
@adel-mamin, can you try podman stop -a; kill -9 $(cat $XDG_RUNTIME_DIR/libpod/pause.pid); rm $XDG_RUNTIME_DIR/libpod/pause.pid ?
I am afraid the pause process keeps alive the old namespace that was configured with a single mapping and you modified the configuration for /etc/subuid and /etc/subgid.
giuseppe
on 30 May 2019
@giuseppe I followed your instructions and it works as expected now:
adel@adel-pc:~
> podman run --publish 80:80 nginx
Trying to pull docker.io/library/nginx...Getting image source signatures
Copying blob 688a776db95f done
Copying blob 6bfc4ec4420a done
Copying blob 743f2d6c1f65 done
Copying config 53f3fd8007 done
Writing manifest to image destination
Storing signatures
Error: error from slirp4netns while setting up port redirection: map[desc:bad request: add_hostfwd: slirp_add_hostfwd failed]
Thank you!
I believe the issue can be closed now.
adel-mamin
on 31 May 2019
Related issues
cruwe
·
3Comments
junjieyuan
·
4Comments
jtligon
·
4Comments
floating-cat
·
5Comments
evelineraine
·
3Comments
Most helpful comment
@adel-mamin, can you try
podman stop -a; kill -9 $(cat $XDG_RUNTIME_DIR/libpod/pause.pid); rm $XDG_RUNTIME_DIR/libpod/pause.pid?I am afraid the pause process keeps alive the old namespace that was configured with a single mapping and you modified the configuration for
/etc/subuidand/etc/subgid.