This only happens when "disablePublicRepoRedirects": false, which only affects sourcegraph.com (the only known instance has altered the value to false).
@asdine This is a good first issue for you to get some familiarity with the Sourcegraph codebase and investigating Sourcegraph.com issues.
@asdine FYI, #10112 fixes this specific kind of problem (i.e. only has hostname), but doesn't propose a systematic way to help us response 404 vs 500.