Kedro: Dependency conflict with commonly used AWS libraries

[njgerner]

Description

Dependency conflict with PyYAML.
Kedro requires PyYAML>=5.1, <6.0 while various AWS libraries require in the range PyYAML<4.3,>=3.10.

Context

I'm unable to run any kedro project commands due to the dependency conflict.

Steps to Reproduce

1. pip install kedro sagemaker

Actual Result

Your Environment

Include as many relevant details about the environment in which you experienced the bug:

• Kedro version used (pip show kedro or kedro -V): 0.14.3
• Python version used (python -V): 3.7.3
• Operating system and version: macOS Mojave 10.14.5

Checklist

Include labels so that we can categorise your issue:

• [ ] Add a "Component" label to the issue
• [ ] Add a "Priority" label to the issue

[yetudada]

@njgerner Thanks for flagging this! We have a fix on the way.

[tolomea]

@njgerner We've loosened the pinning of pyyaml in this commit https://github.com/quantumblacklabs/kedro/commit/c895dca2f50c91c3c5784a04df957787513a1f39 this will be included in the next release (0.14.4 or 0.15.0)
This should resolve the ContextualVersionConflict when running Kedro commands.
However you might still get an error at install time because pip doesn't have a dependency resolution engine https://github.com/pypa/pip/issues/988
You can either resolve these errors in your requirements file or use a third party dependency resolution system like pip-compile from the pip-tools package https://pypi.org/project/pip-tools/

[yetudada]

@njgerner We're working with the AWS team to produce a knowledge document on using Kedro and Sagemaker. Would we be able to talk to you about how you used them together?

[njgerner]

Sure, anything specific?

[yetudada]

@njgerner We made something happen and you can find it on the AWS blog: https://aws.amazon.com/blogs/opensource/using-kedro-pipelines-to-train-amazon-sagemaker-models/