Sceptre: cfn-lint incompatibility with sceptre 2.3.0

[n2taylor]

Hi, it seems the latest cfn-lint and sceptre 2.3.0 have an incompatibility around the networkx dependency.

Using a virtualenv with the following requirements.txt:

cfn-lint==0.33.2
sceptre==2.3.0

We get:

ERROR: sceptre 2.3.0 has requirement networkx==2.1, but you'll have networkx 2.4 which is incompatible.

Same happens if I use Poetry:

Because sceptre (2.3.0) depends on networkx (2.1)
 and cfn-lint (0.33.2) depends on networkx (>=2.4,<3.0), sceptre (2.3.0) is incompatible with cfn-lint (0.33.2).
So, because python-package-template depends on both cfn-lint (0.33.2) and sceptre (2.3.0), version solving failed.

Could the networkx version be bumped up?

[zaro0508]

Anything is possible @n2taylor. How about a PR for us to review?

[n2taylor]

Done!

[tomwwright]

Can we please get this issue reopened as the dependency upgrade was reverted in https://github.com/Sceptre/sceptre/pull/934

Hitting this currently, not great

[ngfgrant]

The problem we are facing with this is that networkx >=2.4 drops support for Python version <3.5 and it looks like they will continue to drop support for older versions of Python3 on a regular basis. https://github.com/networkx/networkx/issues/4027

I think probably the long term solution to this is to remove networkx from the project. We used it to get a quick and stable start on out DAG work but I think it is probably a bit much to have that project drive backward compatibility support for various versions of python3.

In the meantime @tomwwright can I suggest that you pin cfn-lint to a version before they introduced networkx 2.4 as a dependency unless there is something you really need from the latest version. We aim to have a stable, backward compatible api and I don't want to just go dropping support for python versions out the blue. I see that cfn-lint is still pre v1 so might be a good idea to pin it in any case.

[craighurley]

cfn-lint introduced networkx in v0.29.0: https://github.com/aws-cloudformation/cfn-python-lint/blob/v0.29.0/setup.py#L75-L76

The version previous of cfn-lint is v0.28.4: https://github.com/aws-cloudformation/cfn-python-lint/blob/v0.28.4/setup.py

[tomwwright]

Thanks for the suggestions!

I've ended up solving this for the interim simply by running my cfn-lint checks via a Docker container:

FROM python:3.8-alpine

RUN pip install cfn-lint
RUN pip install pydot

WORKDIR /path
ENTRYPOINT ["cfn-lint"]
CMD ["--help"]

docker run --rm -v ${PWD}:/path --name cfn-lint cfn-lint:latest '**/*.cfn.yml' '**/*.sam.yml' ...

[DG-SHeather]

Another option would be to use pipx to install cfn-lint in an isolated environment.