Mingw-packages: python3-setuptools/python3-pip: error: failed to commit transaction (conflicting files)

[sdbbs]

Just tried to update today, and I got this:

$ pacman -Syu
:: Synchronizing package databases...
 mingw32                  541.8 KiB  1278K/s 00:00 [#####################] 100%
 mingw32.sig              119.0   B  0.00B/s 00:00 [#####################] 100%
 mingw64                  542.9 KiB  3.71M/s 00:00 [#####################] 100%
 mingw64.sig              119.0   B  0.00B/s 00:00 [#####################] 100%
 msys                     178.7 KiB  17.5M/s 00:00 [#####################] 100%
 msys.sig                 119.0   B  0.00B/s 00:00 [#####################] 100%
:: Starting core system upgrade...
 there is nothing to do
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Packages (6) libpcre2_8-10.33-1  perl-libwww-6.38-1  python2-pip-19.1-1
             python2-setuptools-41.0.1-1  python3-pip-19.1-1
             python3-setuptools-41.0.1-1

Total Download Size:    4.16 MiB
Total Installed Size:  22.15 MiB
Net Upgrade Size:       0.51 MiB

:: Proceed with installation? [Y/n] y
:: Retrieving packages...
 libpcre2_8-10.33-1-...   124.0 KiB  24.2M/s 00:00 [#####################] 100%
 perl-libwww-6.38-1-any   144.8 KiB  15.7M/s 00:00 [#####################] 100%
 python2-setuptools-...   428.1 KiB  17.4M/s 00:00 [#####################] 100%
 python2-pip-19.1-1-any  1563.8 KiB  7.60M/s 00:00 [#####################] 100%
 python3-setuptools-...   452.6 KiB  18.4M/s 00:00 [#####################] 100%
 python3-pip-19.1-1-any  1546.6 KiB  1358K/s 00:01 [#####################] 100%
(6/6) checking keys in keyring                     [#####################] 100%
(6/6) checking package integrity                   [#####################] 100%
(6/6) loading package files                        [#####################] 100%
(6/6) checking for file conflicts                  [#####################] 100%
error: failed to commit transaction (conflicting files)
python3-setuptools: /c/msys64/usr/lib/python3.7/site-packages/setuptools/__pycache__/_deprecation_warning.cpython-37.pyc exists in filesystem
python3-setuptools: /c/msys64/usr/lib/python3.7/site-packages/setuptools/_deprecation_warning.py exists in filesystem
python3-pip: /c/msys64/usr/lib/python3.7/site-packages/pip/_vendor/pep517/__pycache__/build.cpython-37.pyc exists in filesystem
python3-pip: /c/msys64/usr/lib/python3.7/site-packages/pip/_vendor/pep517/build.py exists in filesystem
python3-pip: /c/msys64/usr/lib/python3.7/site-packages/pip/_vendor/pytoml/__pycache__/test.cpython-37.pyc exists in filesystem
python3-pip: /c/msys64/usr/lib/python3.7/site-packages/pip/_vendor/pytoml/__pycache__/utils.cpython-37.pyc exists in filesystem
python3-pip: /c/msys64/usr/lib/python3.7/site-packages/pip/_vendor/pytoml/test.py exists in filesystem
python3-pip: /c/msys64/usr/lib/python3.7/site-packages/pip/_vendor/pytoml/utils.py exists in filesystem
python3-pip: /c/msys64/usr/lib/python3.7/site-packages/pip/_vendor/urllib3/contrib/__pycache__/_appengine_environ.cpython-37.pyc exists in filesystem
python3-pip: /c/msys64/usr/lib/python3.7/site-packages/pip/_vendor/urllib3/contrib/_appengine_environ.py exists in filesystem
Errors occurred, no packages were upgraded.

Looked around a bit, found this: https://wiki.archlinux.org/index.php/Pacman#.22Failed_to_commit_transaction_.28conflicting_files.29.22_error

This is happening because pacman has detected a file conflict, and by design, will not overwrite files for you. This is by design, not a flaw.

The problem is usually trivial to solve. A safe way is to first check if another package owns the file (pacman -Qo /path/to/file). If the file is owned by another package, file a bug report. If the file is not owned by another package, rename the file which 'exists in filesystem' and re-issue the update command. If all goes well, the file may then be removed.

If you had installed a program manually without using pacman, for example through make install, you have to remove/uninstall this program with all of its files. See also Pacman tips#Identify files not owned by any package.

Ok, so I tried this:

$ pacman -Qo /c/msys64/usr/lib/python3.7/site-packages/setuptools/__pycache__/_deprecation_warning.cpython-37.pyc
error: No package owns /c/msys64/usr/lib/python3.7/site-packages/setuptools/__pycache__/_deprecation_warning.cpython-37.pyc

So, this file is not owned by a package, but I really don't understand how it ended up there then, or what would be the recommended way to handle this - is it ok I just use --overwrite, or should I delete the file; or should I rename, then rename back?

---

EDIT: What I did was, saved all these paths in a file; renamed them like this:

$ cat ~/filelist.txt | while read f; do echo "$f"; mv -v "$f" "$f.old"; done
/c/msys64/usr/lib/python3.7/site-packages/setuptools/__pycache__/_deprecation_warning.cpython-37.pyc
renamed '/c/msys64/usr/lib/python3.7/site-packages/setuptools/__pycache__/_deprecation_warning.cpython-37.pyc' -> '/c/msys64/usr/lib/python3.7/site-packages/setuptools/__pycache__/_deprecation_warning.cpython-37.pyc.old'
....

... then ran the upgrade again:

$ pacman -Syu
:: Synchronizing package databases...
 mingw32 is up to date
 mingw64 is up to date
 msys is up to date
:: Starting core system upgrade...
 there is nothing to do
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Packages (6) libpcre2_8-10.33-1  perl-libwww-6.38-1  python2-pip-19.1-1
             python2-setuptools-41.0.1-1  python3-pip-19.1-1
             python3-setuptools-41.0.1-1

Total Installed Size:  22.15 MiB
Net Upgrade Size:       0.51 MiB

:: Proceed with installation? [Y/n] y
(6/6) checking keys in keyring                     [#####################] 100%
(6/6) checking package integrity                   [#####################] 100%
(6/6) loading package files                        [#####################] 100%
(6/6) checking for file conflicts                  [#####################] 100%
(6/6) checking available disk space                [#####################] 100%
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/__pycache__/__init__.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/__pycache__/py31compat.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/_vendor/__pycache__/__init__.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/_vendor/__pycache__/appdirs.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/_vendor/__pycache__/pyparsing.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/_vendor/__pycache__/six.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/__pycache__/__about__.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/__pycache__/__init__.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/__pycache__/_compat.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/__pycache__/_structures.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/__pycache__/markers.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/__pycache__/requirements.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/__pycache__/specifiers.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/__pycache__/utils.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/__pycache__/version.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pkg_resources/extern/__pycache__/__init__.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools-40.5.0-py3.7.egg-info/
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools-40.5.0-py3.7.egg-info/dependency_links.txt
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools-40.5.0-py3.7.egg-info/entry_points.txt
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools-40.5.0-py3.7.egg-info/PKG-INFO
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools-40.5.0-py3.7.egg-info/requires.txt
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools-40.5.0-py3.7.egg-info/SOURCES.txt
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools-40.5.0-py3.7.egg-info/top_level.txt
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools-40.5.0-py3.7.egg-info/zip-safe
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/__init__.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/archive_util.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/build_meta.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/config.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/dep_util.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/depends.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/dist.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/extension.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/glibc.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/glob.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/launch.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/lib2to3_ex.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/monkey.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/msvc.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/namespaces.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/package_index.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/pep425tags.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/py27compat.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/py31compat.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/py33compat.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/py36compat.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/py36compat.cpython-37.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/sandbox.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/site-patch.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/ssl_support.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/unicode_utils.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/version.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/wheel.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/__pycache__/windows_support.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/_vendor/__pycache__/__init__.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/_vendor/__pycache__/pyparsing.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/_vendor/__pycache__/six.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/_vendor/packaging/__pycache__/__about__.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/_vendor/packaging/__pycache__/__init__.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/_vendor/packaging/__pycache__/_compat.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/_vendor/packaging/__pycache__/_structures.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/_vendor/packaging/__pycache__/markers.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/_vendor/packaging/__pycache__/requirements.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/_vendor/packaging/__pycache__/specifiers.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/_vendor/packaging/__pycache__/utils.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/_vendor/packaging/__pycache__/version.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/__init__.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/alias.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/bdist_egg.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/bdist_rpm.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/bdist_wininst.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/build_clib.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/build_ext.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/build_py.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/develop.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/dist_info.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/easy_install.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/egg_info.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/install.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/install_egg_info.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/install_lib.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/install_scripts.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/py36compat.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/register.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/rotate.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/saveopts.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/sdist.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/setopt.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/test.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/upload.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/command/__pycache__/upload_docs.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/extern/__pycache__/__init__.cpython-37.opt-1.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/setuptools/py36compat.py
warning: could not get file information for usr/lib/python3.7/site-packages/pip-18.1-py3.7.egg-info/
warning: could not get file information for usr/lib/python3.7/site-packages/pip-18.1-py3.7.egg-info/dependency_links.txt
warning: could not get file information for usr/lib/python3.7/site-packages/pip-18.1-py3.7.egg-info/entry_points.txt
warning: could not get file information for usr/lib/python3.7/site-packages/pip-18.1-py3.7.egg-info/not-zip-safe
warning: could not get file information for usr/lib/python3.7/site-packages/pip-18.1-py3.7.egg-info/PKG-INFO
warning: could not get file information for usr/lib/python3.7/site-packages/pip-18.1-py3.7.egg-info/SOURCES.txt
warning: could not get file information for usr/lib/python3.7/site-packages/pip-18.1-py3.7.egg-info/top_level.txt
warning: could not get file information for usr/lib/python3.7/site-packages/pip/_vendor/urllib3/packages/__pycache__/ordered_dict.cpython-37.pyc
warning: could not get file information for usr/lib/python3.7/site-packages/pip/_vendor/urllib3/packages/ordered_dict.py
:: Processing package changes...
(1/6) upgrading libpcre2_8                         [#####################] 100%
(2/6) upgrading perl-libwww                        [#####################] 100%
(3/6) upgrading python2-setuptools                 [#####################] 100%
(4/6) upgrading python2-pip                        [#####################] 100%
(5/6) upgrading python3-setuptools                 [#####################] 100%
(6/6) upgrading python3-pip                        [#####################] 100%

So, apparently the update passed, but all those warnings don't look good at all to me - in any case, would be nice to know what to do in case this problem pops up again ...

[sickc]

Have you manually installed setuptools via pip?
If so, remove these pip modules and re-run pacman.

[lazka]

Yeah, probably you used pip at one point to update a package, and pacman can't know that.
Worst case, in case something breaks, you can re-install the package with pacman.

If you use pip try using pip3 install --user to not write things into the system packages.