Steps to Reproduce:

Does this issue occur when all extensions are disabled?: Yes
I think this is a regression in 1.51.
I can reproduce this only on Windows. On macOS things seems to work just fine.
The issue seems to be that this check here fails.
So the contextService.isInsideWorkspace wrongly returns false here.
I believe the issue is with the uri when it comes into VS Code via the Open With command, it has the following structure. Notice the leading upper case C. And I believe all URI internally have the lower case driver letter.
So the gist is: "Open With" command should normalize drive letters when feeding uri's into VS Code.
Also assigning to @bpasero and @sandy081 for ideas
Assigning candidate label since it is a regression. Not sure if we should ship as part of recovery but for discussion.

Could this be https://github.com/microsoft/vscode/commit/b228267d50d474de56cb56c19d151bd8f61b59f0 too, see https://github.com/microsoft/vscode/issues/108535 @jrieken
Yeah - likely because the uri search tree is not doing implicit normalisations anymore. I'd say the workspace service or workspace-object should make sure to use a canonical uri before looking something up. Not sure this is a candidate tho.
The smallest fix is to create the search tree with the ignorePathCasing option here: https://github.com/microsoft/vscode/blob/960205b8a06e3fdc283b9a476952620041cba84a/src/vs/platform/workspace/common/workspace.ts#L147
I am perfectly fine with not doing this is a candidate. Just added label for awarness.
For users coming to this issue to find more details: we will ship a fix for this issue in vscode stable this or next week.
If you are interested in how we fixed it you can checkout this PR https://github.com/microsoft/vscode/pull/110247
merged my PR and cherry-picked the change to master. Also, driving the adoption of the better way of doing this: https://github.com/microsoft/vscode/issues/110241
Most helpful comment
For users coming to this issue to find more details: we will ship a fix for this issue in vscode stable this or next week.
If you are interested in how we fixed it you can checkout this PR https://github.com/microsoft/vscode/pull/110247