Dvc.org: dvc install --use-pre-commit doesn't install pre-push and post-checkout

[theelderbeever]

Bug Report

dvc install can't be used if a repository is already using pre-commit. While you can use dvc install --use-pre-commit, the pre-push and post-checkout hooks are not added to the git hooks. This behavior is not documented and feels misleading. It may be personal opinion but, I would expect there to be a way to add those hooks when using pre-commit and right now that doesn't appear to be the case. In order to get all of the hooks I had to delete the pre-commit hooks and run the following.

dvc install
pre-commit install
dvc install --use-pre-commit

This seems overly difficult and needless.

Please let me know if I am just missing something obvious.

DVC version: 1.10.2 (pip)
---------------------------------
Platform: Python 3.8.2 on macOS-10.16-x86_64-i386-64bit
Supports: http, https, s3
Cache types: reflink, hardlink, symlink
Caches: local
Remotes: s3
Repo: dvc, git

[pmrowla]

This is really a pre-commit behavior thing, and is not DVC specific.

The --use-pre-commit flag just modifies your existing .pre-commit-config.yaml and adds the DVC configuration. If you are already using pre-commit then you still have to tell pre-commit to install the hook stages you wish to use. By default pre-commit only ever installs the pre-commit hook, and will ignore any other stages unless you explicitly tell pre-commit to install them via the -t/--hook-type option.

So for a repo which is already using pre-commit you should be running:

pre-commit install --hook-type pre-push --hook-type post-checkout
dvc install --use-pre-commit

(this can be done in either order)

Also, where you are doing

dvc install
pre-commit install
dvc install --use-pre-commit

This will only use the pre-commit tool for the pre-commit stage. The post-checkout and push hooks will be run directly by the DVC installed hooks and not pre-commit (since you explicitly installed them with dvc install and have not run pre-commit install --hook-type ...)

[pmrowla]

This is something that we can note in the DVC docs though, so I'll transfer this issue to the dvc.org docs repo

[shcheklein]

@pmrowla

does/can dvc install --use-pre-commit fail or notify if some hooks are not installed?

[pmrowla]

@pmrowla

does/can dvc install --use-pre-commit fail or notify if some hooks are not installed?

It does not, since with pre-commit it is up to the user to pick and choose which hook stages they wish to install themselves (since with any pre-commit plugin there may be particular hooks/stages that a user does not actually want or need). Basically we assume that anyone that is already running pre-commit has configured their pre-commit environment in the specific way that they want.

[shcheklein]

@pmrowla got it! but can we show that hook has been installed or not in the CLI output?

[pmrowla]

pre-commit doesn't have a CLI command to see which hook stages are installed, so I think the only way we could do that is to examine in the actual files in .git/hooks/<stage> and do some string comparison against an expected comment/line to check that it's actually pre-commit's script.

But it seems like that would going a bit beyond the scope of what DVC should reasonably be expected to do in that situation.

[shcheklein]

Sorry, may be I don't understand how actually dvc install --use-pre-commit is implemented, that's why the idea is actually stupid. My point was that this command does something - installs our hooks via some mechanism. I got it that it's hard and fragile to analyze the state. But can we see if installation went successful or not during the process itself (e.g. if we run some command via shell analyze its exit code?).

[pmrowla]

Ah I see. So for pre-commit, you configure a list of plugins (like DVC, linters, etc) and associated hooks for each plugin in .pre-commit-config.yaml. Normally you would manually create that configuration, but dvc install --use-pre-commit can be used to add the DVC plugin section to that configuration file. The DVC section will include hooks for the pre-commit, pre-push, and post-commit stages.

Whether or not anything in that configuration file actually gets run during git commands depends on whether or not the relevant hook stages have been installed via pre-commit install [--hook-type ...], and by default pre-commit install only installs the pre-commit stage.

[shcheklein]

Okay, got it now. I see how can it be confusing - dvc install installs hooks, while dvc install --use-pre-commit does not, but only updates config. Definitely worth clarifying. Thanks @pmrowla !

[theelderbeever]

It sounds like this is being addressed via an update to the docs which seems more or less sufficient. However, when using pre-commit and attempting a dvc install it complains with an error about pre-commit already in use and then fails. Maybe consider specifically adding to the docks what the commands are that can be manually added to the pre-push and post-checkout hook files when a repo already depends on pre-commit? I believe the lines of code are...

# pre-push
exec dvc git-hook pre-push $@

# post-checkout
exec dvc git-hook post-checkout $@

Let me know if it isn't clear what I am saying...

[pmrowla]

If pre-commit is already in use, dvc install (with no args) is not the command you want to be running. In that situation you should use the pre-commit tool to trigger the pre-push/post-checkout hooks, and not the DVC shell scripts.

dvc install --use-pre-commit
pre-commit install --hook-type pre-push --hook-type post-checkout

Basically the whole point of using the pre-commit tool is to avoid ever needing to touch the files in .git/hooks yourself, and the pre-commit tool is not specific to only .git/hooks/pre-commit, it can be used to trigger other git hooks as well.

[theelderbeever]

@pmrowla ah understood. I am still newer to extensive use of pre-commit so you have taught me something new. Thanks.

[pmrowla]

Right, we will address this issue by making the docs are clearer on how to use the DVC hooks with the pre-commit tool.