Dvc: Misleading "update available" message due to lack of synchronization between distribution channels

[autayeu]

Bug Report

dvc checks for updates using "updater.dvc.org". However, distribution channels might have delays. This might lead to a situation when dvc signals update availability, but the newer version is not available yet via distribution channel used for this particular installation.

For example, currently my dvc installed through conda-forge signals 1.5.1 update availability, but the update is not available through conda-forge, therefore the update message is misleading.

Perhaps a more correct strategy would be to check for updates according to and via the same distribution channel that was used for original installation. This will avoid confusion due to lack of synchronization. This might also help users of private repositories which might follow a different update strategy rather than "latest greatest". This might also help users of locked down build environments with controlled or unavailable internet access. The last one can already be partially mitigated via disable update check altogether, but it resembles more of a workaround from the point of view of private, proxy, or cached repository users. Perhaps adhering better to the original distribution channel might be better.

Please provide information about your setup

Output of dvc version:

$ dvc version
DVC version: 1.5.0 (conda)
---------------------------------
Platform: Python 3.8.5 on macOS-10.15.6-x86_64-i386-64bit
Supports: azure, gdrive, gs, hdfs, http, https, s3, ssh, oss
Cache types: reflink, hardlink, symlink
Repo: dvc, git

[efiop]

Hi @autayeu !

Indeed, we used to only push updates once all channels have the latest version, but these days we have things like brew and chocolatey which we don't really have a control over, so it is hard to tell when they will be available. E.g. chocolatey is still waiting for approval for 1.3.0 https://chocolatey.org/packages/dvc/1.3.0 , but we are working on getting a "trusted package" status there... That's why we've started pushing updates earlier (pretty much once pypi, deb/rpm/snap are ready, conda is usually just an hour or two late), so most of the users could upgrade ASAP.

Approach with checking updates through native dist channels would be a bit fragile because we will have to mess with PATH, env, shims and so on, which might be buggy. We could, however, provide updates per package type through https://updater.dvc.org/ , where we currently only have one "version", but might have something like:

{
    "packages": {
        "pypi": {"version": "1.5.1"}
        "conda": {"version": "1.5.0"}
    },
}

so we could tailor our "update available" messages and so people using fast distribution channels could upgrade faster, while not misleading users of slow dist channels.

[autayeu]

Thank you for a great explanation. That might leave private repo users a bit out of luck, but fragility (and complexity) argument makes sense, especially with the pragmatic design proposal.

[dakl]

Yeah, this caused some confusion today:

 » dvc run                                                                                    😄
+-----------------------------------------+
|                                         |
|     Update available 1.5.1 -> 1.6.0     |
|         Run `brew upgrade dvc`          |
|                                         |
+-----------------------------------------+

ERROR: too few arguments. Specify at least one: `-d`, `-o`, `-O`, `-m`, `-M`, `-p`, `--plots`, `--plots-no-cache`, `--outs-persist`, `--outs-persist-no-cache`, `command`.

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
» brew upgrade dvc
Warning: dvc 1.5.1 already installed
»

[efiop]

Adding to the next sprint, will try to take a look ASAP. Thanks for the feedback guys!

[efiop]

Not able to take it in this sprint :( , but we are not rushing with the updater bump these days, until all packages are updated, so shouldn't be too annoying for now.

[efiop]

For the record: we now have a trusted status for our chocolatey package, so it will now keep up with the rest of the releases. https://chocolatey.org/packages/dvc