Brave-browser: Create an exclusion list for the Brave User Agent

[bsclifton]

Description

This is a pre-requisite to providing our our UA https://github.com/brave/brave-browser/issues/1052

• If site is in the exclusion list, send the regular Chromium UA
• If the site is NOT in exclusion list, send Brave UA

Implementation

• Initial implementation could be a hardcoded list
• Later implementations (issue needs to be created) can use a proper list, maintained similar to:
  
  
  
  • https://github.com/brave/autoplay-whitelist
  
  
  • https://github.com/brave/extension-whitelist
  
  
  • https://github.com/brave/referrer-whitelist
  
  

We can look at existing lists that other browsers have (if applicable)

Examples this will fix

These are the known issues which would be fixed (if the sites are included in this exclusion list)

• WhatsApp Web not working on nightly (works on dev) #4608

For other possible examples, check the webcompat project board

[pilgrim-brave]

https://lobste.rs/ is explicitly blocking Brave in User-Agent (confirmed with site owner)

[WorldLanguages]

I've seen on Twitter that Whatsapp Web rejects to work with the new useragent.

[bsclifton]

Closing as wontfix; we're going to go with an inclusion list (similar to partner headers). Once https://github.com/brave/brave-browser/issues/1052 is completed and partners move over, we can deprecate the partner headers