dvc run's auto-add functionality misplaced directory .dvc file

[eliorc]

Setup information
DVC version: 0.30.1
Platform: Ubuntu 18.04
Method of installation: pip

When using dvc add to track a directory, a .dvc file will be added under that directory, which is the expected behaviour

For example, dvc add data/test (test is a directory) has resulted in a data/test.dvc file.

But when you use the auto adding functionality of dvc run, the resulting .dvc file is created in the current directory, and not in the same level of the directory

For example
dvc run -d data/train/train.csv -d data/test -d scripts/create_embeddings.py -o embeddings/56-256 python scripts/create_embeddings.py
(where 56-256 is a directory) has created a 56-256.dvc in the current directory (one above embeddings).

The expected behaviour is for the .dvc file to be created under the embeddings directory.

[shcheklein]

Hi, @eliorc does -f (and potentially using wdir as well) solve the problem for you? In your case, you can do something like this:

dvc run -d data/train/train.csv -d data/test \
             -d scripts/create_embeddings.py -o embeddings/56-256 \
             -f embeddings/56-256.dvc \
             python scripts/create_embeddings.py

[eliorc]

Yeah, it does, and great to know

Though I believe the expected behavior of this scenario, is for the file to be created alongside the directory, wouldn't you agree?

[shcheklein]

@eliorc it's a good question indeed! Analogy with dvc add is a good one, but there is one difference there - you can specify multiple outputs in dvc run and it's not clear why should we put the DVC file along the first one. I think it was the reason for the current way of placing stage files by default, but @efiop can give you more context on this.

[efiop]

Hi @eliorc ! @shcheklein pretty much covered the reasoning :) Also, one more point is that your dvc run stages should have a meaningful name that describes what they do. E.g. featurization.dvc, at which point they are more of a part of your code than just a metafile for the output.

[eliorc]

I understand, the multiple output case is a good reason

[pared]

@eliorc can we close this issue?