Steps to Reproduce:
Is your current tab set to stay open? (It probably isn't if you just opened a file and haven't made any changes.)
Try pressing Ctrl+K, Enter (or Ctrl+Shift+P and search for 'keep') to keep that editor open. Then, try to Ctrl+P and open another file--it should appear in a new tab.
"workbench.editor.enablePreviewFromQuickOpen": false
Most helpful comment
"workbench.editor.enablePreviewFromQuickOpen": false