Dvc: Any suggected way to use dvc from python?

Created on 3 Feb 2020  路  3Comments  路  Source: iterative/dvc

We want to use dvc commands right from our software.

As i see dvc dont have stable api for this.

what way u suggest?

i find this and here they use it like

from setuptools import setup
from setuptools.command.build_py import build_py
from dvc.main import main


class FetchError(Exception):
    pass


class AssetFetcher(build_py):

    def run(self):
        print('Start pulling dvc asset')

        ret = main(['pull', 'train.dvc'])
        if ret != 0:
            msg = 'DVC returned error code {} while pulling assets'
            raise FetchError(msg.format(ret))

        print('Asset pulled successfully !')
        build_py.run(self)


setup(
    name='vat_detection',
    # and many other arguments ...
    cmdclass={'build_py': AssetFetcher}
)

Is this way ok or it can breaks on new versions?

awaiting response question

Most helpful comment

@byaka , that's a good option, if you need more control over the operations, you can try using dvc.repo.Repo

It is not likely that we're going to introduce breaking changes in the near future since we value backwards compatibility, it is not guaranteed, tho :slightly_smiling_face:

All 3 comments

@byaka , that's a good option, if you need more control over the operations, you can try using dvc.repo.Repo

It is not likely that we're going to introduce breaking changes in the near future since we value backwards compatibility, it is not guaranteed, tho :slightly_smiling_face:

@byaka Your example uses CLI through a python entrypoint. That is pretty stable, almost no chance of us moving main() somewhere else anywhere in the future. So that (or just os.system("dvc ...") or Popen) totally works :slightly_smiling_face:

The Repo class mentioned by @mroutis is officially not as secure from changes as CLI, but you should be fine, as long as you use specific dvc version as a dependency in your project. :slightly_smiling_face:

Good to know this, thx!
We give it a try :)

Was this page helpful?
0 / 5 - 0 ratings

Related issues

danfischetti picture danfischetti  路  41Comments

drorata picture drorata  路  46Comments

JoeyCarson picture JoeyCarson  路  53Comments

ChrisHowlin picture ChrisHowlin  路  35Comments

Casyfill picture Casyfill  路  56Comments