Tox: [parallel] .tox/log already exists

[JulienPalard]

Related to #641, there is even this actual issue described in it:

Process1 mkdir(logdir)
Process2 mkdir(logdir) which raises py.error.EEXIST exception

just happened to me with tox 3.7.0:

Failed black under process 24272, stdout:

stderr:
Traceback (most recent call last):
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/py/_error.py", line 66, in checked_call
return func(*args, **kwargs)
FileExistsError: [Errno 17] File exists: '/home/mdk/clones/myproj/.tox/log'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/mdk/.pyenv/versions/3.6.5/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/mdk/.pyenv/versions/3.6.5/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/tox/__main__.py", line 4, in <module>
tox.cmdline()
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/tox/session.py", line 47, in cmdline
main(args)
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/tox/session.py", line 54, in main
retcode = build_session(config).runcommand()
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/tox/session.py", line 67, in build_session
return Session(config)
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/tox/session.py", line 394, in __init__
config.logdir.ensure(dir=1)
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/py/_path/local.py", line 530, in ensure
return p._ensuredirs()
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/py/_path/local.py", line 515, in _ensuredirs
self.mkdir()
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/py/_path/local.py", line 465, in mkdir
py.error.checked_call(os.mkdir, fspath(p))
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/py/_error.py", line 86, in checked_call
raise cls("%s%r" % (func.__name__, args))
py.error.EEXIST: [File exists]: mkdir('/home/mdk/clones/myproj/.tox/log',)

[gaborbernat]

if you switch to isolated build (which is highly recommend) you might avoid this, in general case we need to hold a lock when writing to these logs but that's tbd

[JulienPalard]

By "isolated build" you mean "not using --parallel at all"?

[obestwalter]

@JulienPalard no that's something different, it refers to the build stage, not the execution stage. See: https://tox.readthedocs.io/en/latest/example/package.html?highlight=isolated#setuptools

packaging — tox 3.7.1.dev3 documentation

[JulienPalard]

At first glance I don't really get what it's doing. It looks like by default tox is using setuptools, but if I add a pyproject.toml telling tox to use setuptools it works?

[obestwalter]

I mean this is a bug anyway (so marked it accordinlgy now), but activating isolated build together with parallel run should work around it. Can you try like it is described in the docs and let us know if it works? If the docs are unclear there and you are confused, that is also a chance for us to improve them. Thanks!

[JulienPalard]

OK for the workaround thing :)

For the moment this is probably fixing the ".tox/log already exists", at least, it didn't happened again after a few tests, but can't say for sure as it was not happening often.

[JulienPalard]

What's not clear in the doc for me it's why "isolated_build = True" forces me to use a pyproject.toml. I expected isolated_build = True to isolate the builds but continue to use the default (setuptools) in the absence of pyproject.toml.

[gaborbernat]

I actually tracked down (and fixed) this yesterday as part of #998. Will open a PR within the next week or so.

@JulienPalard isolated_build = True enforces a new best practice outlined in PEP-517 and PEP-518. All projects should have it going ahead. As tox is all about ensuring your projects build and run always I can see no reason why not to use pyproject.toml, so hence the enforcement. While true the spec outlines that in case of missing pyproject.toml fallback to setuptools, the Python zen does say that explicit is better than implicit.

[JulienPalard]

Maybe I'm just a big hater of having so many configuration files but that's a whole other story :] thanks for the great clarification and fast response \o/ \o/ and fixing the issue \o/ and finding a workaround \o/ \o/

Have a good day! :)

[obestwalter]

@gaborbernat

While true the spec outlines that in case of missing pyproject.toml fallback to setuptools, the Python zen does say that explicit is better than implicit.

I tend to agreee, but I think we might need to explain a bit better, what this PEP517/518 shift means and why we enforce it in tox like we do it - openend #1141 for that.

[obestwalter]

Maybe I'm just a big hater of having so many configuration files but that's a whole other story

yes ... but ... this is the standard that will end all other standards!1!!

[adamantike]

@gaborbernat do you know if #1145 had an impact on this issue?

[gaborbernat]

Yeah, this should be fixed now. We'll have to do a release, however due to extensive overhaul of the session package some of our plugins may broke, so I want to set up some integration test first 👍to avoid this

[gaborbernat]

It turned out that the fix had a bug, #1186 fixes it entirely now.