Dvc: no repro metrics error

[dmpetrov]

$ dvc -V
1.7.9+7b590e

I created a metrics file and added it to dvc.yaml without rerunning a pipeline. The metrics diff gives me a strange error:

$ dvc metrics diff
ERROR: unexpected error - while constructing a mapping
  in "<unicode string>", line 3, column 5
found duplicate key "metrics" with value "[]" (original value: "[]")
  in "<unicode string>", line 20, column 5

To suppress this check see:
    http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys

Duplicate keys will become an error in future releases, and are errors
by default when using the new API.


Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
DVC version: 1.7.9+7b590e
---------------------------------
Platform: Python 3.7.3 on Darwin-19.6.0-x86_64-i386-64bit
Supports: azure, gdrive, gs, hdfs, http, https, s3, ssh, oss
Cache types: reflink, hardlink, symlink
Cache directory: apfs on /dev/disk1s5
Workspace directory: apfs on /dev/disk1s5
Repo: dvc, git

$ dvc metrics diff -v
2020-10-05 09:00:38,005 DEBUG: Check for update is enabled.
2020-10-05 09:00:38,039 DEBUG: fetched: [(3,)]
2020-10-05 09:00:38,048 DEBUG: fetched: [(682,)]
2020-10-05 09:00:38,049 ERROR: unexpected error - while constructing a mapping
  in "<unicode string>", line 3, column 5
found duplicate key "metrics" with value "[]" (original value: "[]")
  in "<unicode string>", line 20, column 5

To suppress this check see:
    http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys

Duplicate keys will become an error in future releases, and are errors
by default when using the new API.

------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/dmitry/src/dvc/dvc/main.py", line 76, in main
    ret = cmd.run()
  File "/Users/dmitry/src/dvc/dvc/command/metrics.py", line 119, in run
    all=self.args.all,
  File "/Users/dmitry/src/dvc/dvc/repo/metrics/__init__.py", line 13, in diff
    return diff(self.repo, *args, **kwargs)
  File "/Users/dmitry/src/dvc/dvc/repo/metrics/diff.py", line 22, in diff
    old = _get_metrics(repo, *args, **kwargs, rev=(a_rev or "HEAD"))
  File "/Users/dmitry/src/dvc/dvc/repo/metrics/diff.py", line 9, in _get_metrics
    *args, **kwargs, revs=[rev] if rev else None
  File "/Users/dmitry/src/dvc/dvc/repo/metrics/__init__.py", line 8, in show
    return show(self.repo, *args, **kwargs)
  File "/Users/dmitry/src/dvc/dvc/repo/__init__.py", line 51, in wrapper
    return f(repo, *args, **kwargs)
  File "/Users/dmitry/src/dvc/dvc/repo/metrics/show.py", line 108, in show
    metrics = _collect_metrics(repo, targets, recursive)
  File "/Users/dmitry/src/dvc/dvc/repo/metrics/show.py", line 33, in _collect_metrics
    for stage in repo.stages:
  File "/Users/dmitry/virtenvs/ml/lib/python3.7/site-packages/funcy/objects.py", line 28, in __get__
    res = instance.__dict__[self.fget.__name__] = self.fget(instance)
  File "/Users/dmitry/src/dvc/dvc/repo/__init__.py", line 554, in stages
    return self._collect_stages()
  File "/Users/dmitry/src/dvc/dvc/repo/__init__.py", line 562, in _collect_stages
    new_stages = self.get_stages(os.path.join(root, file_name))
  File "/Users/dmitry/src/dvc/dvc/repo/__init__.py", line 254, in get_stages
    return list(dvcfile.stages.values())
  File "/Users/dmitry/src/dvc/dvc/dvcfile.py", line 228, in stages
    data, _ = self._load()
  File "/Users/dmitry/src/dvc/dvc/dvcfile.py", line 107, in _load
    d = parse_yaml(stage_text, self.path)
  File "/Users/dmitry/src/dvc/dvc/utils/serialize/_yaml.py", line 24, in parse_yaml
    return yaml.load(text) or {}
  File "/Users/dmitry/virtenvs/ml/lib/python3.7/site-packages/ruamel/yaml/main.py", line 343, in load
    return constructor.get_single_data()
  File "/Users/dmitry/virtenvs/ml/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 113, in get_single_data
    return self.construct_document(node)
  File "/Users/dmitry/virtenvs/ml/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 123, in construct_document
    for _dummy in generator:
  File "/Users/dmitry/virtenvs/ml/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 723, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/Users/dmitry/virtenvs/ml/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 440, in construct_mapping
    return BaseConstructor.construct_mapping(self, node, deep=deep)
  File "/Users/dmitry/virtenvs/ml/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 257, in construct_mapping
    if self.check_mapping_key(node, key_node, mapping, key, value):
  File "/Users/dmitry/virtenvs/ml/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 295, in check_mapping_key
    raise DuplicateKeyError(*args)
ruamel.yaml.constructor.DuplicateKeyError: while constructing a mapping
  in "<unicode string>", line 3, column 5
found duplicate key "metrics" with value "[]" (original value: "[]")
  in "<unicode string>", line 20, column 5

To suppress this check see:
    http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys

Duplicate keys will become an error in future releases, and are errors
by default when using the new API.

------------------------------------------------------------

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
DVC version: 1.7.9+7b590e
---------------------------------
Platform: Python 3.7.3 on Darwin-19.6.0-x86_64-i386-64bit
Supports: azure, gdrive, gs, hdfs, http, https, s3, ssh, oss
Cache types: reflink, hardlink, symlink
Cache directory: apfs on /dev/disk1s5
Workspace directory: apfs on /dev/disk1s5
Repo: dvc, git
2020-10-05 09:00:38,885 DEBUG: Analytics is enabled.
2020-10-05 09:00:39,063 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/var/folders/hq/h82mgrhx37nc89_1rcg6gm280000gn/T/tmpivw63jns']'
2020-10-05 09:00:39,065 DEBUG: Spawned '['daemon', '-q', 'analytics', '/var/folders/hq/h82mgrhx37nc89_1rcg6gm280000gn/T/tmpivw63jns']

PS: why there is a dvc version output in the error message?

[skshetry]

@dmpetrov, can you post the dvc.yaml file? From the error, it seems there are 2 "metrics" keys in the file.

[dmpetrov]

stages:
  train:
    cmd: python train.py
    deps:
    - users.csv
    - train.py
    - mymodel.py
    - elog
    params:
    - epochs
    - log_file
    - dvc_logs_dir
    - dropout
    outs:
    - logs
    - dvc_logs
    metrics:
    - summary.json:
        cache: false
    metrics:
    - cust_metr.json:
        cache: false
    plots:
    - logs.csv:
        cache: false

[skshetry]

There are 2 metrics key, just merge them under one "metrics".

    metrics:
    - summary.json:
           cache: false
    - cust_metr.json:
            cache: false

[efiop]

@skshetry Good catch! Could we catch exceptions like that and raise something friendlier so it is not an "unexpected error"?

EDIT: I see we do wrap them https://github.com/iterative/dvc/blob/52fe76cefcc7a1eafe6f8a11b5863a64425b5e26/dvc/utils/serialize/_yaml.py#L23 , but this exception simply doesn't inherit from YAMLError, so it just falls through the wrapper. :slightly_frowning_face:

[dmpetrov]

It would be great to handle the error properly. Ideally to say something like 'dvc.yaml' parsing error: 'metrics' duplication in 'train' stage

[efiop]

It would be great to handle the error properly. Ideally to say something like 'dvc.yaml' parsing error: 'metrics' duplication in 'train' stage

@dmpetrov Unfortunately ruamel's exception doesn't contain much info that we could reuse, so we can only reasonably extract the exc.problem and have something like:

ERROR: unable to read: 'dvc.yaml', found duplicate key "metrics" with value "[]" (original value: "[]")