Kedro: [KED-1358] conda install kedro

[marcusinthesky]

Description

Anaconda is a popular package manager for many data scientists cause of its ability to resolve non-python dependencies, like Cuda. Many tools like MLFLOW rely heavily on anaconda in order to ensure compatibility when serving model on Databricks or Azure.

Context

Many pypi packages can be automatically compiled to be install as conda packages using conda skeleton python kedro, conda convert --platform all /path/to/package/ and conda uploadpath/to/package/`. For kedro this fails, as the pypi .tar.gz does not have the test_requirements.txt.

I have made a workaround by building specifically from the GitHub repo, which I have made availible.

Possible Implementation

Change what is uploaded to pypi to ensure the test_requirements.txt is uploaded.

Possible Alternatives

meta.yml

{% set name = "kedro" %}
{% set version = "0.15.8" %}

package:
  name: "{{ name|lower }}"
  version: "{{ version }}"

source:
  git_rev: 0.15.8
  git_url: https://github.com/quantumblacklabs/kedro.git

build:
  number: 0
  script: "{{ PYTHON }} -m pip install . --no-deps --ignore-installed -vv "

requirements:
  host:
    - pip
    - python
  run:
    - python

test:
  imports:
    - kedro

about:
  home: "https://kedro.readthedocs.io/"
  license: "BSD"
  license_family: "APACHE"
  license_file: "LICENSE.md"
  summary: "A Python library that implements software engineering best-practice for data and ML pipelines."
  doc_url: "https://kedro.readthedocs.io/"
  dev_url: "https://github.com/quantumblacklabs/kedro/tree/develop"

[921kiyo]

@marcusinthesky Thank you for the feature request! We have a ticket in our backlog and will let you know once it is done :)

[d-chambers]

This is currently what is keeping me from using kedro seriously, I don't want the mix of conda and pip installs to result in a broken/unstable environment. Otherwise looks like an amazing project, thanks for open-sourcing your hard work!

[ColinLeongUDRI]

I feel similarly to d-chambers, regarding mixing of pip and conda. Thank you to the people working on the pull request on conda-forge!

[lorenabalan]

Hey guys, good news: we're on conda! 😃 (thanks @marcelotrevisani for the help in getting this pushed on conda-forge)

conda install kedro will install the latest Kedro version, 0.15.9, and we'll be ensuring that future releases will also be added as they come out.

Please be mindful that we haven't gone and retroactively added older versions, we recommend you use the new release, particularly as the last 3 or so releases have been hotfixes. If there's any reason why you can't update and you strongly need the conda installaton, rather than PyPI, let us know in a separate issue and we can have a look at getting that sorted for you.

[marcelotrevisani]

Just to let you all know that it is not necessary to manually create PR to update the recipe on conda-forge because we have some bots which check from time to time for new releases. The bot will automatically create a PR with the new version. :)

[marcelotrevisani]

This is currently what is keeping me from using kedro seriously, I don't want the mix of conda and pip installs to result in a broken/unstable environment. Otherwise looks like an amazing project, thanks for open-sourcing your hard work!

It is nice to have everything packaged with conda indeed. However, if you really need a package which is not (yet) on conda-forge you have some options:

• You can open a PR on conda-forge and package the missing project, or just open an issue there requesting to package that project.

• Or you can install the pacakge with pip but asking pip to not install the dependencies, in that way it will not mess with your environment. Something like pip install --no-deps PKG_NAME

[d-chambers]

Thanks, I didn't know about the no-deps flag.