I did this by adding
"jest.debugCodeLens.showWhenTestStateIn": [
"skip",
"pass",
"fail",
"unknown"
]
to my .vscode/settings.json.
Took a bit of looking around and I finally found it by clicking the cogwheel as shown below below and then Copy Setting as JSON, in User Settings, search for Jest:

That's exactly the way it is supposed to work.
Thanks @eghernqvist !
Most helpful comment
I did this by adding
to my

.vscode/settings.json.Took a bit of looking around and I finally found it by clicking the cogwheel as shown below below and then Copy Setting as JSON, in User Settings, search for Jest: