Curator: elasticsearch-curator-5.8.1 failing - unable to find cx_Freeze

Created on 20 Oct 2020  ·  24Comments  ·  Source: elastic/curator

pip install elasticsearch-curator-5.8.1 error as there is no module cx_freeze found:

Downloading elasticsearch-curator-5.8.1.tar.gz (225 kB) ERROR: Command errored out with exit status 1: command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-v334ie2d/elasticsearch-curator/setup.py'"'"'; __file__='"'"'/tmp/pip-install-v334ie2d/elasticsearch-curator/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-7iqcokxn cwd: /tmp/pip-install-v334ie2d/elasticsearch-curator/ Complete output (20 lines): Traceback (most recent call last): File "/tmp/pip-install-v334ie2d/elasticsearch-curator/setup.py", line 39, in <module> from cx_Freeze import setup, Executable ModuleNotFoundError: No module named 'cx_Freeze'

picture CruzanCaramele
👍32

Most helpful comment

Workaround

Tested on a python 3.6.8 (without any forced update of pip, wheel, setuptools)

# inside a virtualenv 3.6.8
# force installation of urllib and boto3 without dependencies resolution
pip install urllib3==1.24.3 boto3==1.9.142
# now we can install curator !
pip install elasticsearch-curator

cx_Freeze module is due to setup.py line 144

ImportError exception is not caught/detected, causing windows compilation process to go on... instead of linux one !

picture MadDoxXimba on 20 Oct 2020
👍14 🚀1

All 24 comments

+1

issue also exists for older versions

Python 3.6.8
Pip 20.2.4 / same behavior for older Pip versions

pip install elasticsearch-curator==5.7.6 --no-cache
Collecting elasticsearch-curator==5.7.6
  Downloading elasticsearch-curator-5.7.6.tar.gz (218 kB)
     |████████████████████████████████| 218 kB 4.3 MB/s 
    ERROR: Command errored out with exit status 1:
     command: /home/jonathan/.pyenv/versions/3.6.8/envs/cucu/bin/python3.6 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-4f2q6lq8/elasticsearch-curator/setup.py'"'"'; __file__='"'"'/tmp/pip-install-4f2q6lq8/elasticsearch-curator/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-qh209xa6
         cwd: /tmp/pip-install-4f2q6lq8/elasticsearch-curator/
    Complete output (140 lines):
    build/temp.linux-x86_64-3.6/check_libyaml.c:2:10: fatal error: yaml.h: No such file or directory
     #include <yaml.h>
              ^~~~~~~~
    compilation terminated.

    libyaml is not found or a compiler error: forcing --without-libyaml
    (if libyaml is installed correctly, you may need to
     specify the option --include-dirs or uncomment and
     modify the parameter include_dirs in setup.cfg)
    zip_safe flag not set; analyzing archive contents...

    Installed /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/PyYAML-3.12-py3.6-linux-x86_64.egg
    Searching for click<7.0,>=6.7
    Reading https://pypi.org/simple/click/
    Downloading https://files.pythonhosted.org/packages/34/c1/8806f99713ddb993c5366c362b2f908f18269f8d792aff1abfd700775a77/click-6.7-py2.py3-none-any.whl#sha256=29f99fc6125fbc931b758dc053b3114e55c77a6e4c6c3a2674a2dc986016381d
    Best match: click 6.7
    Processing click-6.7-py2.py3-none-any.whl
    Installing click-6.7-py2.py3-none-any.whl to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs

    Installed /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/click-6.7-py3.6.egg
    Searching for requests_aws4auth>=0.9
    Reading https://pypi.org/simple/requests_aws4auth/
    Downloading https://files.pythonhosted.org/packages/4e/44/dec6791ef89cef395918f6e4268d0f30096ee798523fcf26edd53117210a/requests_aws4auth-1.0.1-py2.py3-none-any.whl#sha256=6ddc92476c0ebab6c90709966e0ff3b2f16ba1f296b7ea8573645bbbe5ef9444
    Best match: requests-aws4auth 1.0.1
    Processing requests_aws4auth-1.0.1-py2.py3-none-any.whl
    Installing requests_aws4auth-1.0.1-py2.py3-none-any.whl to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs
    writing requirements to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/requests_aws4auth-1.0.1-py3.6.egg/EGG-INFO/requires.txt

    Installed /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/requests_aws4auth-1.0.1-py3.6.egg
    Searching for boto3>=1.9.142
    Reading https://pypi.org/simple/boto3/
    Downloading https://files.pythonhosted.org/packages/71/79/a2149872c6c3567bf15f548f695c373373d771fea962aff3ae64a87e4876/boto3-1.16.0.tar.gz#sha256=ab5353a968a4e664b9da2dd950169b755066525fcbfdfc90e7e49c8333d95c19
    Best match: boto3 1.16.0
    Processing boto3-1.16.0.tar.gz
    Writing /tmp/easy_install-nosfkjfw/boto3-1.16.0/setup.cfg
    Running boto3-1.16.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-nosfkjfw/boto3-1.16.0/egg-dist-tmp-qungk0yy
    zip_safe flag not set; analyzing archive contents...
    boto3.__pycache__.session.cpython-36: module references __file__
    boto3.docs.__pycache__.service.cpython-36: module references __file__
    creating /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/boto3-1.16.0-py3.6.egg
    Extracting boto3-1.16.0-py3.6.egg to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs

    Installed /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/boto3-1.16.0-py3.6.egg
    Searching for requests>=2.20.0
    Reading https://pypi.org/simple/requests/
    Downloading https://files.pythonhosted.org/packages/45/1e/0c169c6a5381e241ba7404532c16a21d86ab872c9bed8bdcd4c423954103/requests-2.24.0-py2.py3-none-any.whl#sha256=fe75cc94a9443b9246fc7049224f75604b113c36acb93f87b80ed42c44cbb898
    Best match: requests 2.24.0
    Processing requests-2.24.0-py2.py3-none-any.whl
    Installing requests-2.24.0-py2.py3-none-any.whl to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs
    writing requirements to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/requests-2.24.0-py3.6.egg/EGG-INFO/requires.txt

    Installed /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/requests-2.24.0-py3.6.egg
    Searching for urllib3<1.25,>=1.24.2
    Reading https://pypi.org/simple/urllib3/
    Downloading https://files.pythonhosted.org/packages/01/11/525b02e4acc0c747de8b6ccdab376331597c569c42ea66ab0a1dbd36eca2/urllib3-1.24.3-py2.py3-none-any.whl#sha256=a637e5fae88995b256e3409dc4d52c2e2e0ba32c42a6365fee8bbd2238de3cfb
    Best match: urllib3 1.24.3
    Processing urllib3-1.24.3-py2.py3-none-any.whl
    Installing urllib3-1.24.3-py2.py3-none-any.whl to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs
    writing requirements to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/urllib3-1.24.3-py3.6.egg/EGG-INFO/requires.txt

    Installed /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/urllib3-1.24.3-py3.6.egg
    Searching for elasticsearch<8.0.0,>=7.0.0
    Reading https://pypi.org/simple/elasticsearch/
    Downloading https://files.pythonhosted.org/packages/e6/42/ceca2fe2c49ec68abeb96da43c8024ea3f1190e4b1cebb00f4b4b950fea2/elasticsearch-7.10.0a1-py2.py3-none-any.whl#sha256=0d3d9aa105131853e3fcd618853af954a0dcd1a51fb9949d70c111bd7b67cdb8
    Best match: elasticsearch 7.10.0a1
    Processing elasticsearch-7.10.0a1-py2.py3-none-any.whl
    Installing elasticsearch-7.10.0a1-py2.py3-none-any.whl to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs
    writing requirements to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/elasticsearch-7.10.0a1-py3.6.egg/EGG-INFO/requires.txt

    Installed /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/elasticsearch-7.10.0a1-py3.6.egg
    Searching for voluptuous>=0.9.3
    Reading https://pypi.org/simple/voluptuous/
    Downloading https://files.pythonhosted.org/packages/d3/e9/861cec3f4da0728d9d085db9c338b598351cbacb10521c25f0857f55c6d4/voluptuous-0.12.0-py3-none-any.whl#sha256=0fff348a097c9a74f9f4a991d2cf01a6185780e997ad953bde49cb3efbb411be
    Best match: voluptuous 0.12.0
    Processing voluptuous-0.12.0-py3-none-any.whl
    Installing voluptuous-0.12.0-py3-none-any.whl to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs

    Installed /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/voluptuous-0.12.0-py3.6.egg
    Searching for s3transfer<0.4.0,>=0.3.0
    Reading https://pypi.org/simple/s3transfer/
    Downloading https://files.pythonhosted.org/packages/69/79/e6afb3d8b0b4e96cefbdc690f741d7dd24547ff1f94240c997a26fa908d3/s3transfer-0.3.3-py2.py3-none-any.whl#sha256=2482b4259524933a022d59da830f51bd746db62f047d6eb213f2f8855dcb8a13
    Best match: s3transfer 0.3.3
    Processing s3transfer-0.3.3-py2.py3-none-any.whl
    Installing s3transfer-0.3.3-py2.py3-none-any.whl to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs
    writing requirements to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/s3transfer-0.3.3-py3.6.egg/EGG-INFO/requires.txt

    Installed /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/s3transfer-0.3.3-py3.6.egg
    Searching for jmespath<1.0.0,>=0.7.1
    Reading https://pypi.org/simple/jmespath/
    Downloading https://files.pythonhosted.org/packages/07/cb/5f001272b6faeb23c1c9e0acc04d48eaaf5c862c17709d20e3469c6e0139/jmespath-0.10.0-py2.py3-none-any.whl#sha256=cdf6525904cc597730141d61b36f2e4b8ecc257c420fa2f4549bac2c2d0cb72f
    Best match: jmespath 0.10.0
    Processing jmespath-0.10.0-py2.py3-none-any.whl
    Installing jmespath-0.10.0-py2.py3-none-any.whl to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs

    Installed /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/jmespath-0.10.0-py3.6.egg
    Searching for botocore<1.20.0,>=1.19.0
    Reading https://pypi.org/simple/botocore/
    Downloading https://files.pythonhosted.org/packages/7a/38/317bec1175e74b817a51ca806f3e5822f340bb3b344e8693425122499f4f/botocore-1.19.0-py2.py3-none-any.whl#sha256=9bf8586b69f20cf0a8ed1e27338cd10ce847751d1a2fd98b92662565c8a2df24
    Best match: botocore 1.19.0
    Processing botocore-1.19.0-py2.py3-none-any.whl
    Installing botocore-1.19.0-py2.py3-none-any.whl to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs
    writing requirements to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/botocore-1.19.0-py3.6.egg/EGG-INFO/requires.txt

    Installed /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/botocore-1.19.0-py3.6.egg
    Searching for idna<3,>=2.5
    Reading https://pypi.org/simple/idna/
    Downloading https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl#sha256=b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0
    Best match: idna 2.10
    Processing idna-2.10-py2.py3-none-any.whl
    Installing idna-2.10-py2.py3-none-any.whl to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs

    Installed /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/idna-2.10-py3.6.egg
    Searching for chardet<4,>=3.0.2
    Reading https://pypi.org/simple/chardet/
    Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl#sha256=fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691
    Best match: chardet 3.0.4
    Processing chardet-3.0.4-py2.py3-none-any.whl
    Installing chardet-3.0.4-py2.py3-none-any.whl to /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs

    Installed /tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/chardet-3.0.4-py3.6.egg
    Traceback (most recent call last):
      File "/tmp/pip-install-4f2q6lq8/elasticsearch-curator/setup.py", line 39, in <module>
        from cx_Freeze import setup, Executable
    ModuleNotFoundError: No module named 'cx_Freeze'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-4f2q6lq8/elasticsearch-curator/setup.py", line 175, in <module>
        tests_require = ["mock", "nose", "coverage", "nosexcover"]
      File "/home/jonathan/.pyenv/versions/3.6.8/envs/cucu/lib/python3.6/site-packages/setuptools/__init__.py", line 142, in setup
        _install_setup_requires(attrs)
      File "/home/jonathan/.pyenv/versions/3.6.8/envs/cucu/lib/python3.6/site-packages/setuptools/__init__.py", line 137, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/home/jonathan/.pyenv/versions/3.6.8/envs/cucu/lib/python3.6/site-packages/setuptools/dist.py", line 586, in fetch_build_eggs
        replace_conflicting=True,
      File "/home/jonathan/.pyenv/versions/3.6.8/envs/cucu/lib/python3.6/site-packages/pkg_resources/__init__.py", line 789, in resolve
        raise VersionConflict(dist, req).with_context(dependent_req)
    pkg_resources.ContextualVersionConflict: (urllib3 1.24.3 (/tmp/pip-install-4f2q6lq8/elasticsearch-curator/.eggs/urllib3-1.24.3-py3.6.egg), Requirement.parse('urllib3<1.26,>=1.25.4'), {'botocore'})
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
MadDoxXimba picture MadDoxXimba on 20 Oct 2020

did anyone find a workaround for that?

schosterbarak picture schosterbarak on 20 Oct 2020

+1

paveli-d9 picture paveli-d9 on 20 Oct 2020

+1

ArturKwiatkowski picture ArturKwiatkowski on 20 Oct 2020

Note the traceback i posted above has a compilation error with pyyaml and a missing module cx_Freeze:

This can effectively be fixed by using this command inside a virtualenv (3.6.8):

pip install -U pip setuptools wheel cx_Freeze --no-cache-dir

now by using:

pip install elasticsearch-curator==5.7.6 --no-cache-dir # same behavior for other releases

we get a new traceback:

Collecting elasticsearch-curator==5.7.6
  Downloading elasticsearch-curator-5.7.6.tar.gz (218 kB)
     |████████████████████████████████| 218 kB 4.5 MB/s 
Requirement already satisfied: elasticsearch<8.0.0,>=7.0.0 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from elasticsearch-curator==5.7.6) (7.9.1)
Requirement already satisfied: urllib3<1.25,>=1.24.2 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from elasticsearch-curator==5.7.6) (1.24.3)
Requirement already satisfied: requests>=2.20.0 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from elasticsearch-curator==5.7.6) (2.24.0)
Requirement already satisfied: boto3>=1.9.142 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from elasticsearch-curator==5.7.6) (1.16.0)
Requirement already satisfied: requests_aws4auth>=0.9 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from elasticsearch-curator==5.7.6) (1.0.1)
Requirement already satisfied: click<7.0,>=6.7 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from elasticsearch-curator==5.7.6) (6.7)
Requirement already satisfied: pyyaml==3.12 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from elasticsearch-curator==5.7.6) (3.12)
Requirement already satisfied: voluptuous>=0.9.3 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from elasticsearch-curator==5.7.6) (0.12.0)
Requirement already satisfied: certifi>=2019.3.9 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from elasticsearch-curator==5.7.6) (2020.6.20)
Requirement already satisfied: six>=1.11.0 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from elasticsearch-curator==5.7.6) (1.15.0)
Requirement already satisfied: chardet<4,>=3.0.2 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from requests>=2.20.0->elasticsearch-curator==5.7.6) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from requests>=2.20.0->elasticsearch-curator==5.7.6) (2.10)
Requirement already satisfied: botocore<1.20.0,>=1.19.0 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from boto3>=1.9.142->elasticsearch-curator==5.7.6) (1.19.0)
Requirement already satisfied: s3transfer<0.4.0,>=0.3.0 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from boto3>=1.9.142->elasticsearch-curator==5.7.6) (0.3.3)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from boto3>=1.9.142->elasticsearch-curator==5.7.6) (0.10.0)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/lib/python3.6/site-packages (from botocore<1.20.0,>=1.19.0->boto3>=1.9.142->elasticsearch-curator==5.7.6) (2.8.1)
Building wheels for collected packages: elasticsearch-curator
  Building wheel for elasticsearch-curator (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/bin/python3.6 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-i8gllchw/elasticsearch-curator/setup.py'"'"'; __file__='"'"'/tmp/pip-install-i8gllchw/elasticsearch-curator/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-_0okt_9w
       cwd: /tmp/pip-install-i8gllchw/elasticsearch-curator/
  Complete output (72 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib
  creating build/lib/curator
  copying curator/singletons.py -> build/lib/curator
  copying curator/__init__.py -> build/lib/curator
  copying curator/utils.py -> build/lib/curator
  copying curator/indexlist.py -> build/lib/curator
  copying curator/__main__.py -> build/lib/curator
  copying curator/snapshotlist.py -> build/lib/curator
  copying curator/repomgrcli.py -> build/lib/curator
  copying curator/cli.py -> build/lib/curator
  copying curator/actions.py -> build/lib/curator
  copying curator/logtools.py -> build/lib/curator
  copying curator/_version.py -> build/lib/curator
  copying curator/exceptions.py -> build/lib/curator
  copying curator/curator_cli.py -> build/lib/curator
  copying curator/config_utils.py -> build/lib/curator
  running egg_info
  writing elasticsearch_curator.egg-info/PKG-INFO
  writing dependency_links to elasticsearch_curator.egg-info/dependency_links.txt
  writing entry points to elasticsearch_curator.egg-info/entry_points.txt
  writing requirements to elasticsearch_curator.egg-info/requires.txt
  writing top-level names to elasticsearch_curator.egg-info/top_level.txt
  reading manifest file 'elasticsearch_curator.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no files found matching 'Changelog.rst'
  warning: no previously-included files matching '__pycache__' found under directory '*'
  warning: no previously-included files matching '*.py[co]' found under directory '*'
  warning: no previously-included files matching '*.pyc' found under directory 'curator'
  warning: no previously-included files matching '*.pyo' found under directory 'curator'
  warning: no previously-included files matching '*.pyc' found under directory 'docs'
  warning: no previously-included files matching '*.pyo' found under directory 'docs'
  warning: no previously-included files matching '*.pyc' found under directory 'test'
  warning: no previously-included files matching '*.pyo' found under directory 'test'
  no previously-included directories found matching 'docs/_build'
  no previously-included directories found matching 'docs/asciidoc/html_docs'
  writing manifest file 'elasticsearch_curator.egg-info/SOURCES.txt'
  creating build/lib/curator/cli_singletons
  copying curator/cli_singletons/__init__.py -> build/lib/curator/cli_singletons
  copying curator/cli_singletons/alias.py -> build/lib/curator/cli_singletons
  copying curator/cli_singletons/allocation.py -> build/lib/curator/cli_singletons
  copying curator/cli_singletons/close.py -> build/lib/curator/cli_singletons
  copying curator/cli_singletons/delete.py -> build/lib/curator/cli_singletons
  copying curator/cli_singletons/forcemerge.py -> build/lib/curator/cli_singletons
  copying curator/cli_singletons/object_class.py -> build/lib/curator/cli_singletons
  copying curator/cli_singletons/open_indices.py -> build/lib/curator/cli_singletons
  copying curator/cli_singletons/replicas.py -> build/lib/curator/cli_singletons
  copying curator/cli_singletons/restore.py -> build/lib/curator/cli_singletons
  copying curator/cli_singletons/rollover.py -> build/lib/curator/cli_singletons
  copying curator/cli_singletons/show.py -> build/lib/curator/cli_singletons
  copying curator/cli_singletons/shrink.py -> build/lib/curator/cli_singletons
  copying curator/cli_singletons/snapshot.py -> build/lib/curator/cli_singletons
  copying curator/cli_singletons/utils.py -> build/lib/curator/cli_singletons
  creating build/lib/curator/defaults
  copying curator/defaults/__init__.py -> build/lib/curator/defaults
  copying curator/defaults/client_defaults.py -> build/lib/curator/defaults
  copying curator/defaults/filter_elements.py -> build/lib/curator/defaults
  copying curator/defaults/filtertypes.py -> build/lib/curator/defaults
  copying curator/defaults/option_defaults.py -> build/lib/curator/defaults
  copying curator/defaults/settings.py -> build/lib/curator/defaults
  creating build/lib/curator/validators
  copying curator/validators/__init__.py -> build/lib/curator/validators
  copying curator/validators/actions.py -> build/lib/curator/validators
  copying curator/validators/config_file.py -> build/lib/curator/validators
  copying curator/validators/filters.py -> build/lib/curator/validators
  copying curator/validators/options.py -> build/lib/curator/validators
  copying curator/validators/schemacheck.py -> build/lib/curator/validators
  running build_exe
  error: [Errno 2] No such file or directory: 'run_curator.py'
  ----------------------------------------
  ERROR: Failed building wheel for elasticsearch-curator
  Running setup.py clean for elasticsearch-curator
Failed to build elasticsearch-curator
Installing collected packages: elasticsearch-curator
    Running setup.py install for elasticsearch-curator ... error
    ERROR: Command errored out with exit status 1:
     command: /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/bin/python3.6 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-i8gllchw/elasticsearch-curator/setup.py'"'"'; __file__='"'"'/tmp/pip-install-i8gllchw/elasticsearch-curator/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-djvc2wy9/install-record.txt --single-version-externally-managed --compile --install-headers /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/include/site/python3.6/elasticsearch-curator
         cwd: /tmp/pip-install-i8gllchw/elasticsearch-curator/
    Complete output (6 lines):
    usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: setup.py --help [cmd1 cmd2 ...]
       or: setup.py --help-commands
       or: setup.py cmd --help

    error: option --single-version-externally-managed not recognized
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/bin/python3.6 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-i8gllchw/elasticsearch-curator/setup.py'"'"'; __file__='"'"'/tmp/pip-install-i8gllchw/elasticsearch-curator/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-djvc2wy9/install-record.txt --single-version-externally-managed --compile --install-headers /home/jonathan/.pyenv/versions/3.6.8/envs/another_test/include/site/python3.6/elasticsearch-curator Check the logs for full command output.

it seems either elasticsearch-curator or one of it's transitive dependencies is using a broken/removed API of setuptools !

option --single-version-externally-managed not recognized
MadDoxXimba picture MadDoxXimba on 20 Oct 2020
👍1

The cx_Freeze import error is not the real issue, that module is optional and only used if you want to create executables.

The real issue seems to be the impossible to resolve dependency conflicts between botocore and curator:
elasticsearch-curator 5.8.1 requires urllib3<1.25,>=1.24.2
botocore 1.19.0 requires urllib3<1.26,>=1.25.4

rbooysen picture rbooysen on 20 Oct 2020

Workaround

Tested on a python 3.6.8 (without any forced update of pip, wheel, setuptools)

# inside a virtualenv 3.6.8
# force installation of urllib and boto3 without dependencies resolution
pip install urllib3==1.24.3 boto3==1.9.142
# now we can install curator !
pip install elasticsearch-curator

cx_Freeze module is due to setup.py line 144

ImportError exception is not caught/detected, causing windows compilation process to go on... instead of linux one !

MadDoxXimba picture MadDoxXimba on 20 Oct 2020
👍14 🚀1

Apparently, the latest botocore release broke urllib3 dependency.
Since curator is not using a fixed version of boto3, its update alongside with botocore update, broke urllib3 dependency for curator

mikhailantoshkin picture mikhailantoshkin on 21 Oct 2020

I recommend installing like below to ensure you get the most up to date dependencies:

$ python -m pip install urllib3==1.24.3 boto3 'botocore<1.19' --use-feature=2020-resolver
$ python -m pip install elasticsearch-curator

$ python -m pip freeze
boto3==1.15.18
botocore==1.18.18
certifi==2020.6.20
chardet==3.0.4
click==6.7
docutils==0.15.2
elasticsearch==7.9.1
elasticsearch-curator==5.8.1
idna==2.10
jmespath==0.10.0
pkg-resources==0.0.0
python-dateutil==2.8.1
PyYAML==3.13
requests==2.24.0
requests-aws4auth==1.0.1
s3transfer==0.3.3
six==1.15.0
urllib3==1.24.3
voluptuous==0.12.0

Notice the boto3 version is 1.15.18 instead of the more aggressive pin of 1.9.142 suggested above.

sethmlarson picture sethmlarson on 21 Oct 2020
👍3

@sethmlarson i have the following in my pip file but it still fails with the same error:

```urllib3==1.24.3
boto3==1.15.18
botocore==1.18.18
elasticsearch
elasticsearch-curator==5.8.1
requests
validators

CruzanCaramele picture CruzanCaramele on 22 Oct 2020

@sethmlarson

Any idea when a fix can be done ?
This is breaking my company build chain....

As you suggested, this can easily be fixed by defining a max version of botocore in setup.py

I did prefer not to resort modifying myself the source code for a fix !

MadDoxXimba picture MadDoxXimba on 22 Oct 2020
👍1

@CruzanCaramele
We ended up locking all of our dependencies in requirements.txt with pip-compile and using --no-dependencies during the build of our docker image specifically for curator as a separate command

pip3 install --no-cache-dir --no-dependencies elasticsearch-curator==5.8.1

It fixed our CI build for now but it would be great if we wouldn't have to resort to tricks like this

mikhailantoshkin picture mikhailantoshkin on 22 Oct 2020

@mikhailantoshkin thank you. So please what does your requirements.txt look like now ? or what changes do i have to make to mine to make it work as well ?

CruzanCaramele picture CruzanCaramele on 22 Oct 2020

Same issue here with my CI builds failing at curator install (eg https://travis-ci.org/github/geerlingguy/ansible-for-devops/jobs/738057564). Seems like the PR that updates the lib results in some other new errors :(

geerlingguy picture geerlingguy on 23 Oct 2020

@CruzanCaramele it looks like this
```
attrs==19.1.0 # via jsonschema
backoff==1.9.0
beautifulsoup4==4.4.1
boto3==1.9.155 # via elasticsearch-curator
botocore==1.12.155 # via boto3, s3transfer
certifi==2020.4.5.2
chardet==3.0.4 # via requests
click==6.7 # via clickclick, elasticsearch-curator, flask
clickclick==1.2.2 # via connexion
configobj==5.0.6
connexion==1.1.13
cvss==1.9
docutils==0.14 # via botocore

elasticsearch-curator==5.8.1

elasticsearch==7.7.1
flask==1.0.3 # via connexion
idna==2.8 # via requests
inflection==0.3.1 # via connexion
itsdangerous==1.1.0 # via flask
jinja2==2.10.1 # via flask
jmespath==0.9.4 # via boto3, botocore
jsonschema==3.0.1 # via connexion, swagger-spec-validator
markupsafe==1.1.1 # via jinja2
netaddr==0.7.18
pathlib==1.0.1 # via connexion
pyrsistent==0.15.2 # via jsonschema
python-dateutil==2.8.0
pyyaml==3.13 # via clickclick, connexion, elasticsearch-curator, swagger-spec-validator
requests-aws4auth==0.9 # via elasticsearch-curator
requests==2.22.0 # via connexion, elasticsearch-curator, requests-aws4auth
s3transfer==0.2.0 # via boto3
six==1.15.0
swagger-spec-validator==2.4.3
typing==3.6.6 # via connexion
urllib3==1.24.3
voluptuous==0.11.5 # via elasticsearch-curator
werkzeug==0.15.4 # via flask
xmltodict==0.12.0
```
Mind you it is quite old, so adjust versions accordingly

mikhailantoshkin picture mikhailantoshkin on 23 Oct 2020
👍1

My requirements.txt file looks like this:

urllib3==1.24.3
boto3==1.9.155
botocore==1.12.155
elasticsearch-curator==5.8.1

But I still face the issue:

22:37:16  Collecting urllib3==1.24.3
22:37:16    Downloading urllib3-1.24.3-py2.py3-none-any.whl (118 kB)
22:37:17  Collecting boto3==1.9.155
22:37:17    Downloading boto3-1.9.155-py2.py3-none-any.whl (128 kB)

22:37:18  Collecting botocore==1.12.155
22:37:18    Downloading botocore-1.12.155-py2.py3-none-any.whl (5.4 MB)

22:37:18  Collecting elasticsearch-curator==5.8.1
22:37:18    Downloading elasticsearch-curator-5.8.1.tar.gz (225 kB)

22:37:50      ERROR: Command errored out with exit status 1:
22:37:50       command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-fc3s1k1z/elasticsearch-curator/setup.py'"'"'; __file__='"'"'/tmp/pip-install-fc3s1k1z/elasticsearch-curator/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-nkq8afr5
22:37:50           cwd: /tmp/pip-install-fc3s1k1z/elasticsearch-curator/
22:37:50      Complete output (20 lines):
22:37:50      Traceback (most recent call last):
22:37:50        File "/tmp/pip-install-fc3s1k1z/elasticsearch-curator/setup.py", line 39, in <module>
22:37:50          from cx_Freeze import setup, Executable
22:37:50      ModuleNotFoundError: No module named 'cx_Freeze'
22:37:50      
22:37:50      During handling of the above exception, another exception occurred:
22:37:50      
22:37:50      Traceback (most recent call last):
22:37:50        File "<string>", line 1, in <module>
22:37:50        File "/tmp/pip-install-fc3s1k1z/elasticsearch-curator/setup.py", line 177, in <module>
22:37:50          tests_require = ["mock", "nose", "coverage", "nosexcover"]
22:37:50        File "/usr/local/lib/python3.7/site-packages/setuptools/__init__.py", line 143, in setup
22:37:50          _install_setup_requires(attrs)
22:37:50        File "/usr/local/lib/python3.7/site-packages/setuptools/__init__.py", line 138, in _install_setup_requires
22:37:50          dist.fetch_build_eggs(dist.setup_requires)
22:37:50        File "/usr/local/lib/python3.7/site-packages/setuptools/dist.py", line 698, in fetch_build_eggs
22:37:50          replace_conflicting=True,
22:37:50        File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 792, in resolve
22:37:50          raise VersionConflict(dist, req).with_context(dependent_req)
22:37:50      pkg_resources.ContextualVersionConflict: (urllib3 1.24.3 (/tmp/pip-install-fc3s1k1z/elasticsearch-curator/.eggs/urllib3-1.24.3-py3.7.egg), Requirement.parse('urllib3<1.26,>=1.25.4'), {'botocore'})
22:37:50      ----------------------------------------
22:37:50  ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Any ideas? My OS is python:3.7-alpine3.10

ocofaigh picture ocofaigh on 26 Oct 2020
👍3

Any sign of a fix here? Or at least a workaround that works?

ocofaigh picture ocofaigh on 30 Oct 2020

The urllib3 dependency issue is currently causing major issues for us. Any more activity on this?

tlmurphy picture tlmurphy on 2 Nov 2020
👍7

There have been some recent vulnerabilities found in urllib3 that have been fixed in the latest versions. It'd be a good idea to fix this soon.

jsoucheiron picture jsoucheiron on 3 Nov 2020

Any other ideas how to fix? Any sign of a fix at some point?
Trying different versions of dependencies from the comments above but still the same error:

error: [Errno 2] No such file or directory: 'run_curator.py'
  ----------------------------------------
  ERROR: Failed building wheel for elasticsearch-curator
  Running setup.py clean for elasticsearch-curator
Failed to build elasticsearch-curator
Installing collected packages: elasticsearch-curator
    Running setup.py install for elasticsearch-curator ... error
    ERROR: Command errored out with exit status 1:
     command: /root/.bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_oj7i1rn/elasticsearch-curator/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_oj7i1rn/elasticsearch-curator/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-tmuirkgk/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.6m/elasticsearch-curator
         cwd: /tmp/pip-install-_oj7i1rn/elasticsearch-curator/
    Complete output (6 lines):
    usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: setup.py --help [cmd1 cmd2 ...]
       or: setup.py --help-commands
       or: setup.py cmd --help

    error: option --single-version-externally-managed not recognized
knoha picture knoha on 6 Nov 2020

From what I gather, this was closed because there was a fix added, but does anyone know what version the fix is in? I don't see any new release yet. 5.8.1 is still the latest.

ocofaigh picture ocofaigh on 9 Nov 2020

Actually looks like an integration test failed in the merge pipeline after the fix was added. I think this issue needs to be re-opened until the build passes the pipeline and there is a new version available with this fix. (I don't see an option to re-open).

ocofaigh picture ocofaigh on 9 Nov 2020

It doesn't seem to be related to the changes though. There seems to be
another problem with flaky tests

El dl., 9 de nov. 2020, 14:40, Conall Ó Cofaigh notifications@github.com
va escriure:

Actually looks like an integration test failed in the merge pipeline after
the fix was added. I think this issue needs to be re-opened until the build
passes the pipeline and there is a new version available with this fix. (I
don't see an option to re-open).

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/elastic/curator/issues/1560#issuecomment-724019654,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAN625SL66QHDUXXD2WONEDSO7WMJANCNFSM4SXYFVFQ
.

jsoucheiron picture jsoucheiron on 9 Nov 2020

Still an issue #1567

t3chn0m4g3 picture t3chn0m4g3 on 18 Nov 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

bemeyert picture bemeyert  ·  3Comments
myoung34 picture myoung34  ·  7Comments
tiempososcuros picture tiempososcuros  ·  8Comments
ppf2 picture ppf2  ·  4Comments
pickypg picture pickypg  ·  10Comments