Dvc: BUG: JSON parsing error for `dvc metrics show`

[rgasper]

Bug Report

I have a simple JSON of test metrics, in file scores.json:

{
    "test_1": {
        "precision": 0,
        "f1": 0.0
    },
    "test_2": {
        "fpr": 0.0
    },
    "notes": "if precision is zero, there were zero positive predictions; if f1 is zero, something went very wrong"
}

I expect the key "notes" to fail, but the numeric keys should show up. However, they don't:

$ dvc metrics show -v scores.json
2020-09-28 17:16:09,022 DEBUG: Check for update is enabled.
2020-09-28 17:16:09,024 DEBUG: fetched: [(3,)]                        
2020-09-28 17:16:09,029 DEBUG: Could not parse 'precision' metric from 'scores.json' at '' due to its unsupported type: 'int'
2020-09-28 17:16:09,029 DEBUG: Could not parse 'f1' metric from 'scores.json' at '' due to its unsupported type: 'float'
2020-09-28 17:16:09,029 DEBUG: Could not parse 'test_1' metric from 'scores.json' at '' due to its unsupported type: 'dict'
2020-09-28 17:16:09,029 DEBUG: Could not parse 'fpr' metric from 'scores.json' at '' due to its unsupported type: 'float'
2020-09-28 17:16:09,029 DEBUG: Could not parse 'test_2' metric from 'scores.json' at '' due to its unsupported type: 'dict'
2020-09-28 17:16:09,029 DEBUG: Could not parse 'notes' metric from 'scores.json' at '' due to its unsupported type: 'str'
2020-09-28 17:16:09,030 DEBUG: fetched: [(11243,)]
2020-09-28 17:16:09,031 ERROR: failed to show metrics - Could not parse metric files. Use `-v` option to see more details.

Please provide information about your setup

Output of dvc version:

$ dvc version
DVC version: 1.7.9 (pip)
---------------------------------
Platform: Python 3.7.9 on Darwin-19.6.0-x86_64-i386-64bit

Additional Information (if any):

Running dvc metrics diff -v HEAD^1 simply produces the same set of errors twice.

In the situation where this was found, I have other, longer scores_XXX.json files in the repo that are working just fine!

I think having a new feature when running dvc metrics show without -v that mentions the keys which failed to parse would be a user-friendly addition!

[shcheklein]

Context: https://discordapp.com/channels/485586884165107732/485596304961962003/760249329637916682

[skshetry]

This is quite an interesting bug. We do support reading only partial values, but it seems we are not making a distinction between None and 0/0.0 (falsy values). :slightly_smiling_face:

https://github.com/iterative/dvc/blob/e3a7857b1352f1203504ed754482b9e308a48e2f/dvc/repo/metrics/show.py#L51

The fix is just to do if m not in (None, {}) on that line.
Still, there seems to be another bug when the metrics file contains a scalar value (i.e. json file with just 0 in it), so we might need similar checks there too.

https://github.com/iterative/dvc/blob/e3a7857b1352f1203504ed754482b9e308a48e2f/dvc/repo/metrics/show.py#L83

@rgasper, would you be interested in making a fix?

[rgasper]

If you can direct me to instructions on how to setup the codebase, sure.

[skshetry]

Sure, @rgasper, please follow the contributing guide. If you have any problems, please don't hesitate to ask in the #dev-talk channel in the Discord.

[rgasper]

@skshetry , @shcheklein , find the PR above and handle accordingly!