Android: Broken site

[JeffAment]

App version: 53602

Description:
I tried to go to https://www.cvs.com this morning, and it appears to be broken with no change when requesting the desktop version. Please see the pics below of what the page currently looks like, and what it's supposed to look like.

"Mistaken identity" theory:
This appears to be a user agent issue, and I noticed that any suffix starting with "DuckDuck" (case insensitive) loads the site incorrectly, while other suffixes such as "DuckDuccGo/5.0" will work just fine. I have a feeling that the site rendered itself differently since it thought that it might be DuckDuckGo's web crawler visiting. Since DDG has multiple crawlers that start with "DuckDuck", I am guessing that cvs.com has some kind of logic on their end to render the pages different if "DuckDuck*" is in the user agent string. I further corroborated this theory by adding "Googlebot/1" (Google's web crawler) to the user agent, and the page loaded incorrectly again. It also loaded incorrectly with "Googlebottdsgdg/1", but it loaded correctly with "Googlbot/1" and "Googlebo/1" which further supports the wildcard theory. This is almost all conjecture, but it sounds logical to me.

If my theory is correct, it sounds to me that it's really an issue that CVS needs to resolve on their end, but I am worried that other websites may employ similar logic. Feel free to ping me for more information :bell:

[subsymbolic]

Fixed by https://github.com/duckduckgo/Android/pull/853. Thanks again for reporting this @JeffAment!

[JeffAment]

Thank you for linking to the PR @subsymbolic. I am happy to see a fix that doesn't omit the DDG component of the UA altogether. I think that having a dynamic UA to accommodate the small subset of sites that cause problems is an excellent idea, and I am very satisfied with the solution here. Great work!! :rocket: :+1: :fire:

[JeffAment]

Sorry for the double ping @subsymbolic.

Just a quick heads up, it seems that after the latest update (5.58.1), tirerack.com no longer automatically loads its mobile version :no_mobile_phones:. I have an inkling that the changes to the UA string in PR 853 could have something to do with it, but I have not verified this. It still loads correctly on other browsers, so I doubt that the problem emanated from new logic on tirerack's end.

I wasn't sure if it was better to post here or to start a new issue; please let me know if you would like me to open a formal issue on this.

[subsymbolic]

Thanks for reporting this @JeffAment. I've opened an internal issue to investigate and fix this. I'll comment once I have an update.

Internal issue: https://app.asana.com/0/72649045549333/1183556013880866