Uassets: french list too broad regex filter breaking domain names with a final dot.

[dascritch]

URL(s) where the issue occurs

Any domain name with a final . in the name

Describe the issue

A new regex is preventing to visit websites when the entered domain name is written with a final dot, with is legit and sometimes used by webdevs.

The offending regex is :

/^(https?|wss?):\/\/([0-9a-z\._-]+)\.(accountant|bid|cf|click|club|com|cricket|date|download|faith|fun|ga|gdn|gq|info|link|loan|men|ml|net|network|ovh|party|pro|pw|racing|review|rocks|ru|science|site|space|stream|tk|top|trade|webcam|win|xyz|zone)\.\/(.*)/$document

the problems is probably in the \.

The regex is from : FRA: EasyList Liste FR

Versions

• Browser/version: Firefox Nigthly 80
• uBlock Origin version: uBlock Origin v1.27.10

Settings

• only standard settings

[mapx-]

french list issues => https://forums.lanik.us/viewforum.php?f=91

@smed79

[dascritch]

Why there is no issue report howto file in the original github repo linked from the addon ?

I'm fed of, it's the third time

[dascritch]

OK, I note the very offensive way to treat an issue, especially if you are not french speaking.

[mapx-]

You should provide test cases , sites broken by that filter, steps to reproduce the issue

[dascritch]

Easy :
google.com. is blocked.

It should not, as writing a final dot in a domain name is absolutely legit.

[uBlock-user]

I added /^(https?|wss?):\/\/([0-9a-z\._-]+)\.(accountant|bid|cf|click|club|com|cricket|date|download|faith|fun|ga|gdn|gq|info|link|loan|men|ml|net|network|ovh|party|pro|pw|racing|review|rocks|ru|science|site|space|stream|tk|top|trade|webcam|win|xyz|zone)\.\/(.*)/$document to my filters, opened google.com and it works as expected. Google is not blocked.

[dascritch]

https://user-images.githubusercontent.com/482505/86326199-a2e47a00-bc41-11ea-827a-7c32a9cb9429.png

[mapx-]

@smed79 you should remove such filters, lots of issues were reported using french list

[mapx-]

@dascritch already fixed

[uBlock-user]

You're going to google.com./ instead of google.com/. Why ?

[dascritch]

Why not ?

As I told, it is a really used trick by webdevs on websites to use many profiles on browsers. And use the _Fully Qualified Domain Name_ notation, instead of the _Relative_ as your usually do, may be really useful in some DNS issues.

For more details : https://webmasters.stackexchange.com/questions/73989/how-should-websites-handle-hostname-with-trailing-dot

[uBlock-user]

Why not ?

Because google.com./ redirects to google.com/ in the end. So not a use case.

[dascritch]

It was an example : numerous websites don't redirect their FQDN to their RDN , and it permits to go thru misconfigured DNS servers.

If you cannot accept tech people are using a final dot in domain names for testing purposes, you're just over complicating their job. So mine.

[dascritch]

Well, even after refreshing all the caches and filters, there is too much limitations when we have FRA: EasyList Liste FR activated for my team : by example any JS or CSS are still blocked. As we usually use FQDN for multiple account testing on websites, I recommended to not uninstall uBlock, but to remove FRA: EasyList Liste FR.

I am not able to tell which rule is blocking, as i don't have enough information in dev tools (capture of a personal website, due to NDA).

I still cannot understand why final dotted domain names are blocked, this is really problematic.

poke @uBlock-user

[uBlock-user]

Did you update the uBlock unbreak list ? The RegExp filter was disabled in that list.

Nevermind, all other network types are also blocked.

[gorhill]

That is such a bad generic filter that we should fix this with no question asked really -- a trailing dot is entirely valid, indiscriminately blocking any hostname which has a trailing dot is really bad. I would like to know what was the reason to add such filter in a widely used list.

[gwarser]

I think this filter was created to block mining scripts using this FQDN trick.

https://hg.adblockplus.org/listefr/rev/b7608d8b8bbb
https://github.com/hoshsadiq/adblock-nocoin-list/commit/e2c961fca4834e653ecb5faa4e7bd5c4ec4a4540

Is this really used for good purposes in the "normal" web? Or just one developer is "exploiting" this in local development environment?

[dascritch]

It is really used in generic environment as it is a legal way to force a DNS request, or to make cleaned request without cookies or cleaning cache.

And it seems also used here by malicious code to circumvent a poorly written regex.

[gorhill]

A trailing dot would not work around the filters which were replaced, not in uBO as far as I know, and I would be surprised other blockers would not handle properly the trailing dot, so I still fail to understand what issue this solves.

[gorhill]

Ok so it appears that filter was added back then to accomodate ABP, which was not properly dealing with the trailing dot (and as a result this shortcoming was being leveraged by miners), this was eventually fixed in ABP, so that bad regex filter should be reverted back to the original set of filters or outright removed.

[gwarser]

To block only the requests with trailing dot? (regex having good or bad syntax is not important) I don't know.

---

In uBO .com blocks also .com.. Highlighting in logger is missing

In ABP match is exact.

[gorhill]

uBO drops the trailing dot, since it's essentially the same, i.e. example.com is same as example.com.. I don't think filters should care about the trailing dot. Missing highlighting in the logger should be fixed.

Edit: I need to clarify, when I say uBO drops the trailing dot, I mean for hostname-matching purpose (which is what the original filters were), the trailing dot is not dropped in case of URL pattern-matching purpose.

[maxxyme]

If you cannot accept tech people are using a final dot in domain names for testing purposes, you're just over complicating their job. So mine.

Or maybe "tech people" (I am) shouldn't clutter their web browser with so much extensions when "testing"... 🤔