elementary OS 5.0, Files, Dark Mode (via System Settings > Tweaks), no labels:

Dark Mode with labels:

Suggestion: to use dark font color for filenames on color labels in Dark Mode.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
As the available color tags are all pale, a quick fix would be to always use black text for colored labels.
@jeremypw I am working on https://github.com/elementary/granite/pull/284, but I agree since we know the tag colors, we could just always use a constant suitably-contrasting foreground color as well.
Unless this is considered urgent I'll wait for the Granite utility. That would allow for possible future use of darker tag colors.
@jeremypw the util is in Granite master, but awaiting a new release.