Rasa: [Docker] - Rasa init error

[tbx0912]

Rasa version:
Latest (1.4.1) - See on Docker Hub

Docker & docker-compose versions:
Docker version 19.03.4, build 9013bf583a
docker-compose version 1.24.1, build 4667896b

Operating system (windows, osx, ...):
Ubuntu 18.04.3 LTS

Issue:
With a brand new installation of Docker, dating from the day, it is impossible to launch rasa commands (_init_, _train_nlu_, ...) via Docker as described in the docs.

I tried with an empty current directory and a non-empty current directory (where volume is mounted) : the result is the same, leading to the error below.

I am connected as root, so the problem is not due to rights or anything else (about ten other diverse containers are currently working without any problem on the same machine)

Note: I have never encountered this problem with previous versions of the rasa:latest and rasa_latest-full images.

Error (including full traceback):

root@xyz:/path/to/my/project# docker run -v $(pwd):/app rasa/rasa:latest init --no-prompt
Unable to find image 'rasa/rasa:latest' locally
latest: Pulling from rasa/rasa
[...]
Status: Downloaded newer image for rasa/rasa:latest
Warning: Output is not to a terminal (fd=1).
Warning: Input is not to a terminal (fd=0).
Welcome to Rasa! 🤖

To get started quickly, an initial project will be created.
If you need some help, check out the documentation at https://rasa.com/docs/rasa.

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/distutils/dir_util.py", line 70, in mkpath
    os.mkdir(head, mode)
PermissionError: [Errno 13] Permission denied: '__pycache__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/build/bin/rasa", line 11, in <module>
    load_entry_point('rasa==1.5.0a1', 'console_scripts', 'rasa')()
  File "/build/lib/python3.6/site-packages/rasa/__main__.py", line 76, in main
    cmdline_arguments.func(cmdline_arguments)
  File "/build/lib/python3.6/site-packages/rasa/cli/scaffold.py", line 195, in run
    init_project(args, path)
  File "/build/lib/python3.6/site-packages/rasa/cli/scaffold.py", line 111, in init_project
    create_initial_project(path)
  File "/build/lib/python3.6/site-packages/rasa/cli/scaffold.py", line 119, in create_initial_project
    copy_tree(scaffold_path(), path)
  File "/usr/local/lib/python3.6/distutils/dir_util.py", line 159, in copy_tree
    verbose=verbose, dry_run=dry_run))
  File "/usr/local/lib/python3.6/distutils/dir_util.py", line 135, in copy_tree
    mkpath(dst, verbose=verbose)
  File "/usr/local/lib/python3.6/distutils/dir_util.py", line 74, in mkpath
    "could not create '%s': %s" % (head, exc.args[-1]))
distutils.errors.DistutilsFileError: could not create '__pycache__': Permission denied

Command or request that led to error:

docker run -v $(pwd):/app rasa/rasa:latest init --no-prompt

or

docker run -v $(pwd):/app rasa/rasa:latest-full init --no-prompt

Thanks in advance and best regards
TBX

[sara-tagger]

Thanks for raising this issue, @dakshvar22 will get back to you about it soon✨

Please also check out the docs and the forum in case your issue was raised there too 🤗

[dakshvar22]

Hi @tbx0912, I tried pulling the latest docker image on macOS and running the commands work as expected. Did you just switch to Ubuntu 18.04, or were you always testing on this platform for previous versions of Rasa too?

[tbx0912]

Hi @dakshvar22.
No, I've only been working on Ubuntu 18.04 since its release in stable version, so my previous (successful) tests with earlier Rasa versions were also running on Ubuntu 18.04.

I executed the command causing the abovementionned problem on other of my machines running Ubuntu 18.04: the problem and error traceback are exactly the same. And this with slightly different versions of Docker depending on the machines !

[OliverWalter]

I have the same issue with Ubuntu 18.04. What I found suspicious is that when running
docker run -it --entrypoint /bin/bash -v $(pwd):/app rasa/rasa
The user id shows up as 1001 and even a simple touch leads to a permission denied.
$ docker run -it --entrypoint /bin/bash -v $(pwd):/app rasa/rasa I have no name!@8d124f071a6f:/app$ id uid=1001 gid=0(root) groups=0(root) I have no name!@8d124f071a6f:/app$ touch blah touch: cannot touch 'blah': Permission denied
Yet interestingly an empty folder models is created with root:root permissions when running the docker with init (but crashing with the same error as in the original post)

[OliverWalter]

Changing permissions of $(pwd) used for the volume to 1001:root fixes the issue since my own user has a different id to 1001. Presumably this bug can only be reproduced if you are non running as the "first created user" on the system.

[ortsaCniveK]

@tbx0912 I'd try sudo rm -rf __pycache__ wherever it is and running the command again, if you haven't already

I think this is also related to a much larger issue where the rasa docker containers are still running commands as root, if you check file permissions in output (like models or folders created from rasa init) via ll or ls -alF in the command line if you're unix based.

See this PR in the rasa-sdk repo for what I think should be the proper way to handle file permissions and command executions. Below I don't believe allows for commands to be run as unprivileged nor handles folder permission properly.

Dockerfile_full

# ...
# Make sure the default group has the same permissions as the owner
RUN chgrp -R 0 . && chmod -R g=u .

# Don't run as root
USER 1001
# ...

versus a well-placed:

# ...
RUN groupadd -g 1000 nonroot && \
  useradd -r -u 1000 -g nonroot nonroot
USER nonroot
# ...

typically this is done after any installations that require root privileges

@wochinge I'd love to hear your thoughts, because what I've seen as a pattern for nonroot command execution is something like what's found here

[wochinge]

Below I don't believe allows for commands to be run as unprivileged nor handles folder permission properly.

Could you please explain why that's the case?

We use group 0 (aka root) in our docker images cause that's the default group used on OpenShift clusters (whereas OpenShift assigns random user ids). The only difference I see between the current instructions and the ones you shared is, that you 1) create a new group and 2) Assign a name to the user.

[AndiLeni]

I get the same error on my system.
Additionally I created a post in the forum, maybe someone has already solved this issue there.
https://forum.rasa.com/t/docker-install-fails-with-pycache-folder-error/21648

[eprabhakar]

Thanks to @AndiLeni for posting the link of the rasa forum where the issue is posted, and @ArjaanBuijk for giving a simple solution in the forum, that worked excellent for me. I am just posting it here for the convenience,

> Solution by @ArjaanBuijk 

This is how it looks for me, on Ubuntu 18.04, when I am inside the folder
 where I will install the mood-bot:

$ stat .
  File: .
  Size: 4096        Blocks: 8          IO Block: 4096   directory
Device: 801h/2049d  Inode: 5119770     Links: 2
Access: (0755/drwxr-xr-x)  Uid: ( 1000/  arjaan)   Gid: ( 1000/  arjaan)
Access: 2019-12-24 10:10:38.261990864 -0500
Modify: 2019-12-24 10:03:55.915369626 -0500
Change: 2019-12-24 10:03:55.915369626 -0500
 Birth: -

As you can see, access rights for the folder are for uid=1000.

So, running docker container as the user with uid=1000 will work:

$ sudo docker run --user 1000 -v $(pwd):/app rasa/rasa init --no-prompt

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed due to inactivity. Please create a new issue if you need more help.

[CaptXiong]

cd your_dir
stat .
checke the uid permission of the dir.
if you dont have permission,then mkdir, do as the tutorial