Dvc: Create a troubleshooting entry for the lock error message

[shcheklein]

From time to time users are getting a message like this:

ERROR: cannot perform the command because another DVC process seems to be running on this project. If that is not the case, manually remove.dvc/tmp/lockand try again.

(and even removing .dvc/tmp/lock doesn't help in this case)

when they run DVC on some Network fs (NFS, Lustre, etc).

Example: https://discordapp.com/channels/485586884165107732/563406153334128681/727222314080665682

Usually solution is simple and user needs to enable the core.hardlink_lock, but it's hard to get that from the error message.

To save us and users time we should create a troubleshooting entry and include link to the error message.

[moinnadeem]

I'm getting this issue; how would I enable the core.hardlink_lock?

[efiop]

@moinnadeem

dvc config core.hardlink_lock true

https://dvc.org/doc/command-reference/config :slightly_smiling_face:

[shcheklein]

@moinnadeem hi! just to clarify and make sure that this ticket is the right one- have you tried to remove the .dvc/tmp/lock file (first make sure that no DVC commands are running) first? Do you use NFS or Lustre or some other network/distributed file system?

[moinnadeem]

@shcheklein Yes, and yes! I'm on a networked filesystem that doesn't allow file-level locking either. Using hardlinking solved it!

I appreciate the quick response!

[shcheklein]

@moinnadeem perfect! :) thanks for clarifying.

[harsh8398]

Hi @shcheklein I would like to take this issue. Could you please assign it to me? Also, where is this troubleshooting entry supposed to be added?

[shcheklein]

@harsh8398 sure! feel free to work on this. Here is the place https://dvc.org/doc/user-guide/troubleshooting . Here is the guide on how to deal with the docs updates - https://dvc.org/doc/user-guide/contributing/docs

[jorgeorpinel]

Hi!

Usually solution is simple and user needs to enable the core.hardlink_lock, but it's hard to get that from the error message.

so should the error message in this repo also improve a little? (and link to the new troubleshooting anchor in docs)

[harsh8398]

@shcheklein I'm having trouble with tests/func/test_lock.py::test_cli test case. The test case uses assert FAILED_TO_LOCK_MESSAGE in caplog.text check. The issue is that the FAILED_TO_LOCK_MESSAGE contains link with ascii escape codes added using one of the utility functions format_link() whereas the caplog.text only has the printed text which doesn't have the escape codes. So the test case will check two different strings and fail.

I am thinking to add assert statement for URL only like assert TROUBLESHOOTING_URL in caplog.text. Any suggestions? Thanks in advance 🙂.

[shcheklein]

@efiop @pmrowla @pared @skshetry could you please help @harsh8398 with the tests here ^^ 🙏

[pared]

@harsh8398
I am very sorry for such a long time to respond.

As you might see in other our tests, we are usually duplicating the error messages that we want to check.
For example:
https://github.com/iterative/dvc/blob/b11fe83828fd5c65973a5ad7da3d95a5ea2c60e9/tests/func/test_data_cloud.py#L219

So in this case, what would be sufficient, would be to create the expected message and verify it is showing up in caplog.text.

[harsh8398]

Thanks @pared for your help. 🙂

@shcheklein I've raised the PR with the changes!