Dvc: Ubuntu snap installation PermissionError

Created on 1 Sep 2020 · 14Comments · Source: iterative/dvc

Bug Report

Trying to run dvc run I got forbidden access on snap/dvc folders

Please provide information about your setup

I use Ubuntu 20.04 and installed dvc using snap using command: sudo snap install dvc --classic note: --classic is required as this happens without it:

$ sudo snap install dvc
[sudo] password for ricardo: 
error: This revision of snap "dvc" was published using classic confinement and thus may perform
       arbitrary system changes outside of the security sandbox that snaps are usually confined
       to, which may put your system at risk.

       If you understand and want to proceed repeat the command including --classic.

Output of dvc version:

$ dvc version
WARNING: Unable to detect supported link types, as cache directory '.dvc/cache' doesn't exist. It is usually auto-created by commands such as `dvc add/fetch/pull/run/import`, but you could create it manually to enable this check.
DVC version: 1.6.6 (snap)
---------------------------------
Platform: Python 3.6.9 on Linux-5.4.0-42-generic-x86_64-with-Ubuntu-20.04-focal
Supports: All remotes
Repo: dvc, git

Additional Information (if any):

If applicable, please also provide a --verbose output of the command, eg: dvc add --verbose.

This is the full error return:

$ dvc run -n get_bucket_ids --verbose -d scripts/get_bucket_ids.sh -o ml/outputs/exploration/bucket_ids.txt --no-exec ./scripts/get_bucket_ids.sh > ml/outputs/exploration/bucket_ids.txt
--- Logging error ---                                                 
Traceback (most recent call last):
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 136, in emit
    msg, file=self.stream, end=getattr(self, "terminator", "\n")
  File "/snap/dvc/774/lib/python3.6/site-packages/tqdm/std.py", line 585, in write
    fp.write(s)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 41, in write
    self.__convertor.write(text)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 162, in write
    self.write_and_convert(text)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 187, in write_and_convert
    self.write_plain_text(text, cursor, start)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 196, in write_plain_text
    self.wrapped.flush()
PermissionError: [Errno 13] Permission denied
Call stack:
  File "/snap/dvc/774/bin/dvc", line 12, in <module>
    sys.exit(main())
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/main.py", line 75, in main
    ret = cmd.run()
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/command/run.py", line 56, in run
    external=self.args.external,
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/repo/__init__.py", line 31, in wrapper
    with repo.lock, repo.state:
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 126, in __enter__
    self.load()
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 225, in load
    self._prepare_db(empty=empty)
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 166, in _prepare_db
    ret = self._fetchall()
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 137, in _fetchall
    logger.debug("fetched: %s", ret)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1296, in debug
    self._log(DEBUG, msg, args, **kwargs)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1444, in _log
    self.handle(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1454, in handle
    self.callHandlers(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1516, in callHandlers
    hdlr.handle(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 865, in handle
    self.emit(record)
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 142, in emit
    self.handleError(record)
Message: 'fetched: %s'
Arguments: ([(3,)],)
2020-09-01 17:39:13,530 ERROR: unexpected error - failed to log <LogRecord: dvc.state, 10, /snap/dvc/774/lib/python3.6/site-packages/dvc/state.py, 137, "fetched: %s">
------------------------------------------------------------
Traceback (most recent call last):
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 136, in emit
    msg, file=self.stream, end=getattr(self, "terminator", "\n")
  File "/snap/dvc/774/lib/python3.6/site-packages/tqdm/std.py", line 585, in write
    fp.write(s)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 41, in write
    self.__convertor.write(text)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 162, in write
    self.write_and_convert(text)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 187, in write_and_convert
    self.write_plain_text(text, cursor, start)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 196, in write_plain_text
    self.wrapped.flush()
PermissionError: [Errno 13] Permission denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/main.py", line 75, in main
    ret = cmd.run()
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/command/run.py", line 56, in run
    external=self.args.external,
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/repo/__init__.py", line 31, in wrapper
    with repo.lock, repo.state:
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 126, in __enter__
    self.load()
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 225, in load
    self._prepare_db(empty=empty)
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 166, in _prepare_db
    ret = self._fetchall()
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 137, in _fetchall
    logger.debug("fetched: %s", ret)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1296, in debug
    self._log(DEBUG, msg, args, **kwargs)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1444, in _log
    self.handle(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1454, in handle
    self.callHandlers(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1516, in callHandlers
    hdlr.handle(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 865, in handle
    self.emit(record)
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 142, in emit
    self.handleError(record)
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 129, in handleError
    raise LoggingException(record)
dvc.logger.LoggingException: failed to log <LogRecord: dvc.state, 10, /snap/dvc/774/lib/python3.6/site-packages/dvc/state.py, 137, "fetched: %s">
------------------------------------------------------------
--- Logging error ---
Traceback (most recent call last):
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 136, in emit
    msg, file=self.stream, end=getattr(self, "terminator", "\n")
  File "/snap/dvc/774/lib/python3.6/site-packages/tqdm/std.py", line 585, in write
    fp.write(s)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 41, in write
    self.__convertor.write(text)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 162, in write
    self.write_and_convert(text)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 187, in write_and_convert
    self.write_plain_text(text, cursor, start)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 196, in write_plain_text
    self.wrapped.flush()
PermissionError: [Errno 13] Permission denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/main.py", line 75, in main
    ret = cmd.run()
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/command/run.py", line 56, in run
    external=self.args.external,
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/repo/__init__.py", line 31, in wrapper
    with repo.lock, repo.state:
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 126, in __enter__
    self.load()
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 225, in load
    self._prepare_db(empty=empty)
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 166, in _prepare_db
    ret = self._fetchall()
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 137, in _fetchall
    logger.debug("fetched: %s", ret)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1296, in debug
    self._log(DEBUG, msg, args, **kwargs)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1444, in _log
    self.handle(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1454, in handle
    self.callHandlers(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1516, in callHandlers
    hdlr.handle(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 865, in handle
    self.emit(record)
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 142, in emit
    self.handleError(record)
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 129, in handleError
    raise LoggingException(record)
dvc.logger.LoggingException: failed to log <LogRecord: dvc.state, 10, /snap/dvc/774/lib/python3.6/site-packages/dvc/state.py, 137, "fetched: %s">

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 136, in emit
    msg, file=self.stream, end=getattr(self, "terminator", "\n")
  File "/snap/dvc/774/lib/python3.6/site-packages/tqdm/std.py", line 585, in write
    fp.write(s)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 41, in write
    self.__convertor.write(text)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 162, in write
    self.write_and_convert(text)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 187, in write_and_convert
    self.write_plain_text(text, cursor, start)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 196, in write_plain_text
    self.wrapped.flush()
PermissionError: [Errno 13] Permission denied
Call stack:
  File "/snap/dvc/774/bin/dvc", line 12, in <module>
    sys.exit(main())
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/main.py", line 101, in main
    logger.info(FOOTER)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1308, in info
    self._log(INFO, msg, args, **kwargs)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1444, in _log
    self.handle(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1454, in handle
    self.callHandlers(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1516, in callHandlers
    hdlr.handle(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 865, in handle
    self.emit(record)
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 142, in emit
    self.handleError(record)
Message: '\n\x1b[33mHaving any troubles?\x1b[39m Hit us up at \x1b[34mhttps://dvc.org/support\x1b[39m, we are always happy to help!'
Arguments: ()
Traceback (most recent call last):
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 136, in emit
    msg, file=self.stream, end=getattr(self, "terminator", "\n")
  File "/snap/dvc/774/lib/python3.6/site-packages/tqdm/std.py", line 585, in write
    fp.write(s)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 41, in write
    self.__convertor.write(text)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 162, in write
    self.write_and_convert(text)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 187, in write_and_convert
    self.write_plain_text(text, cursor, start)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 196, in write_plain_text
    self.wrapped.flush()
PermissionError: [Errno 13] Permission denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/main.py", line 75, in main
    ret = cmd.run()
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/command/run.py", line 56, in run
    external=self.args.external,
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/repo/__init__.py", line 31, in wrapper
    with repo.lock, repo.state:
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 126, in __enter__
    self.load()
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 225, in load
    self._prepare_db(empty=empty)
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 166, in _prepare_db
    ret = self._fetchall()
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/state.py", line 137, in _fetchall
    logger.debug("fetched: %s", ret)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1296, in debug
    self._log(DEBUG, msg, args, **kwargs)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1444, in _log
    self.handle(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1454, in handle
    self.callHandlers(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1516, in callHandlers
    hdlr.handle(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 865, in handle
    self.emit(record)
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 142, in emit
    self.handleError(record)
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 129, in handleError
    raise LoggingException(record)
dvc.logger.LoggingException: failed to log <LogRecord: dvc.state, 10, /snap/dvc/774/lib/python3.6/site-packages/dvc/state.py, 137, "fetched: %s">

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 136, in emit
    msg, file=self.stream, end=getattr(self, "terminator", "\n")
  File "/snap/dvc/774/lib/python3.6/site-packages/tqdm/std.py", line 585, in write
    fp.write(s)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 41, in write
    self.__convertor.write(text)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 162, in write
    self.write_and_convert(text)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 187, in write_and_convert
    self.write_plain_text(text, cursor, start)
  File "/snap/dvc/774/lib/python3.6/site-packages/colorama/ansitowin32.py", line 196, in write_plain_text
    self.wrapped.flush()
PermissionError: [Errno 13] Permission denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/dvc/774/bin/dvc", line 12, in <module>
    sys.exit(main())
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/main.py", line 101, in main
    logger.info(FOOTER)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1308, in info
    self._log(INFO, msg, args, **kwargs)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1444, in _log
    self.handle(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1454, in handle
    self.callHandlers(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 1516, in callHandlers
    hdlr.handle(record)
  File "/snap/dvc/774/usr/lib/python3.6/logging/__init__.py", line 865, in handle
    self.emit(record)
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 142, in emit
    self.handleError(record)
  File "/snap/dvc/774/lib/python3.6/site-packages/dvc/logger.py", line 129, in handleError
    raise LoggingException(record)
dvc.logger.LoggingException: failed to log <LogRecord: dvc, 20, /snap/dvc/774/lib/python3.6/site-packages/dvc/main.py, 101, "
Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!">
Exception ignored in: <colorama.ansitowin32.StreamWrapper object at 0x7f8314ab5a58>
PermissionError: [Errno 13] Permission denied

awaiting response bug p1-important research ui

Source

ricoms

All 14 comments

Hi @ricoms So this error doesn't happen if you use --classic, right?

efiop on 1 Sep 2020

I noticed that I needed to change my command, removed the > ml/outputs/exploration/bucket_ids.txt at the end, it stayed like this:
$ dvc run -n get_bucket_ids --verbose -d scripts/get_bucket_ids.sh -o ml/outputs/exploration/bucket_ids.txt --no-exec ./scripts/get_bucket_ids.sh

Then I uninstalled dvc from snap and installed it via pip. Now it works.

ricoms on 1 Sep 2020

Hi @ricoms So this error doesn't happen if you use --classic, right?

No, sorry I must have explained badly. I am not able to snap install dvc without --classic argument.
So the only way to install it, for me, is or using snap install dvc --classic or pip install dvc.

Did I explain myself?

ricoms on 1 Sep 2020

I am not able to snap install dvc without --classic argument.

That's expected, we have it in all our installation instructions. E.g. https://github.com/iterative/dvc#snap-snapcraftlinux

I noticed that I needed to change my command, removed the > ml/outputs/exploration/bucket_ids.txt at the end, it stayed like this:
$ dvc run -n get_bucket_ids --verbose -d scripts/get_bucket_ids.sh -o ml/outputs/exploration/bucket_ids.txt --no-exec ./scripts/get_bucket_ids.sh

Then I uninstalled dvc from snap and installed it via pip. Now it works.

So did it start working after removing the > ... or after switching to pip?

efiop on 1 Sep 2020

I am not able to snap install dvc without --classic argument.

That's expected, we have it in all our installation instructions. E.g. https://github.com/iterative/dvc#snap-snapcraftlinux

I noticed that I needed to change my command, removed the > ml/outputs/exploration/bucket_ids.txt at the end, it stayed like this:
$ dvc run -n get_bucket_ids --verbose -d scripts/get_bucket_ids.sh -o ml/outputs/exploration/bucket_ids.txt --no-exec ./scripts/get_bucket_ids.sh

Then I uninstalled dvc from snap and installed it via pip. Now it works.

So did it start working after removing the > ... or after switching to pip?

I tested it here and it worked after removing the > ....
Basically it works now for both installations types, snap, and pip. 0o`

Sorry about that, I believe we can close this issue. Unless it would be interesting to maybe enhance that error message?

ricoms on 1 Sep 2020

@ricoms The error is very strange, I haven't seen anything like that before. The > in your command wasn't within the single quotes, so your shell was redirecting all dvc stdout (not only for your command) to ml/outputs/exploration/bucket_ids.txt. Is there anything special about your shell or that path?

Let me try to reproduce it locally...

efiop on 1 Sep 2020

@ricoms The error is very strange, I haven't seen anything like that before. The > in your command wasn't within the single quotes, so your shell was redirecting all dvc stdout (not only for your command) to ml/outputs/exploration/bucket_ids.txt. Is there anything special about your shell or that path?

Let me try to reproduce it locally...

I agree, I thought that too and I looked everywhere before submitting an issue here, but couldn't find anything similar anywhere.

Is there anything special about your shell or that path? Not that I'm aware of, I'm running it on the Visual Studio code terminal but that shouldn't be a problem I believe...

ricoms on 1 Sep 2020

👍1

Not able to reproduce :thinking: Hm, this is very strange. As I understand, you are not able to reproduce it as well now with snap nor with pip package, right?

Is there anything special about your shell or that path? Not that I'm aware of, I'm running it on the Visual Studio code terminal but that shouldn't be a problem I believe...

Ah, that might actually be the cause, though I'm not sure why (maybe some vscode terminal implementation caveat/bug). @skshetry , IIRC you are using vscode too, could you please give it a try?

efiop on 1 Sep 2020

Not able to reproduce Hm, this is very strange. As I understand, you are not able to reproduce it as well now with snap nor with pip package, right?

Is there anything special about your shell or that path? Not that I'm aware of, I'm running it on the Visual Studio code terminal but that shouldn't be a problem I believe...

Ah, that might actually be the cause, though I'm not sure why (maybe some vscode terminal implementation caveat/bug). @skshetry , IIRC you are using vscode too, could you please give it a try?

I removed everything and run it on a proper Ubuntu terminal, it worked as expected without a problem

$ dvc run -n get_bucket_ids --verbose -d scripts/get_bucket_ids.sh -o ml/outputs/exploration/bucket_ids.txt --no-exec ./scripts/get_bucket_ids.sh > ml/outputs/exploration/bucket_ids.txt

When I try to run it on VScode terminal I get the same error as before. So my VScode configuration is messing things.

ricoms on 1 Sep 2020

Indeed, can reproduce with snap package in vscode terminal. Pip package works fine with it though. Looking into it...

efiop on 2 Sep 2020

👍1

Ok, got distracted from researching this. So far colorama's ansi2win32 wrapper seems suspicious, need to take a closer look.

efiop on 3 Sep 2020

👀1

@ricoms Are you still running into this issue? Unfortunately didn't have time to research it further and I don't think we've got any reports from other users. So might be some issue in your particular setup.

efiop on 20 Oct 2020

Hi @efiop , I'm thanks for looking into it, I'm not running into this problem because I'm basically avoiding to use the VSCode terminal when using DVC or any other projects I'm working on.

we can close this issue, although someday in the future I would like to understand what is happening xD

ricoms on 20 Oct 2020

@ricoms Ok, closing for now then. If anyone else is running into this issue please let us know and we'll reopen. Thank you! :pray:

efiop on 20 Oct 2020

Was this page helpful?

0 / 5 - 0 ratings