Vscode-jest: Commented code is unknown state, removed commented code is good

[exapsy]

Environment

1. node -v: v14.13.1
2. npm -v: 6.14.8
3. npm ls jest or npm ls react-scripts (if you haven’t ejected): (not sure

$ yarn list jest
[email protected]

$ yarn list react-scripts
[email protected]

1. your vscode-jest settings if customized:

   • jest.pathToJest? "jest.pathToJest": "./node_modules/.bin/react-scripts test"

2. Operating system: Manjaro Gnome Linux

Steps to Reproduce

I've made a codesandbox so you can get the exact files - it shall not be used for reproducing the issue as the vscode-jest obviously is not part of codesandbox' ecosystem (and it has an issue with jest.useRealTimers anyways).
You should rather use it to just copy-paste:

• Headers.test.tsx
• Headers.tsx
• types.ts
• jest.config.ts
• tsconfig-jest.json

https://codesandbox.io/s/jest-fail-wbktj?file=/src/Headers.test.tsx

Relevant Debug Info

Tests are all successful when running jest and Output: jest also seems to be working correctly and with all the relevant details

Expected Behavior

Show an error or a successful status in gutter

Actual Behavior

Shows ? in gutter if I add more than 1 tests in describe('TableCollection::Headers::HeaderInput'
Shows ? in gutter in every single test after the commented test in a predictable way, but the behavior seems random:

• When I have commented the code (or when its uncommented), ? appears.
• when I remove the commented (or not commented) code, it seems that now all of sudden I get normal gutter statuses

https://user-images.githubusercontent.com/9019120/115970717-e56aa080-a54c-11eb-97f1-93660548b15f.mp4

It really looks like a quite random behavior, I dont know what could remotely be the issue.

[connectdotz]

@exapsy looks like you are using v4 alpha and you have parameterized tests, in this case, you will need jest >= 26.5.0. You seem to be using [email protected].

I copied your sandbox and run it with v4.0.0-alpha.5 and jest 26.6.x and it seems to be working fine... Free free to give it a try and let us know if it works.

[exapsy]

Yeah that seems to be it. I wonder what was making this happen, seems like an interesting issue.

[connectdotz]

I wonder what was making this happen, seems like an interesting issue

Yeah, unlike the earlier versions that use test names to find test results for each test block, v4 relies on relative position instead. Prior to 26.5.0, jest had a bug ( facebook/jest#10412 ) that output all parameterized test results with the incorrect (hardcoded) position. The combination of the two is what caused the behavior you were seeing.