Ublock: Feature Request: Whitelist wildcard IP addresses

[mnaiman]

Hello, I want to exclude filtering on 192.168.0.* or 10.0.* etc. How to make it possible?

[timthelion]

Just to make it clear, do you want to develop websites that serve ads, while at the same time use an ad blocker ;)?

[mnaiman]

No, I have sites in intranet, that for sure dont serve ads. But there are sometimes false positives which prevent these application from working after subscription update.

[gorhill]

This will need careful implementation -- using wildcard with IP-address-based whitelist directives will apply only to URLs with an IP-address-based host, otherwise this could lead to the unexpected whitelisting of sites (192.168.0.evil.biz).

[RoxKilly]

Another thought: opening this up to internet (rather than intranet) IPs would expose users to unpredictable content online. Is this a concern? Or would you be comfortable giving advanced users this power?

[lewisje]

It's not common for Internet sites to use bare IP addresses, and if the URL doesn't use a bare IP address, uBlock Origin can't make a decision on the basis of the IP address.

[gorhill]

IPs would expose users to unpredictable content online

We are talking about whitelisting directives here, nothing else. One can already use *.com as whitelist directive -- so I fail to see why having the ability to enter 192.* would suddenly be a concern. All those whitelist directives have to be created explicitly by users.

[krisweir]

This would also be very useful for me. Specifically for management UI pages for network switches, routers, NAS devices, UPnP media servers and renderers etc, etc.

At the moment I have to disable Ublock (now Origin) for each IP address or DNS name individually.
This is a total of over 300 entries...
The IP address list is not too hard. I have had to create a similar one for Java exception lists already, as Java is blocked by default in browsers and I need to enable it for many switch management UIs (e.g. HP Procurver switches.) And Java do not allow *_any *_wildcards :(

The DNS names are more labour-intensive though. For Intranet addresses, these are normally unqualified (i.e. no domain specified) e.g. https://intranet so for example if you could whitelist an address with no dots in it, that would work.

Cheers,
Chris

[gorhill]

e.g. https://intranet

Did you try intranet as a whitelist directive? I would expect this to work.

[krisweir]

Yes, it does work, but I'd have to whitelist all the internal web servers and other web-enabled devices on my LAN individually (about 300 in total) - on every PC or Mac that I use for management (at least 10)

And do it all again every time I get a new PC, or have to reformat, etc.

It would be great if there was an option to disable uBlock (Origin) for all urls containing local subnet IP addresses(e.g. http_//10.10.3._) or unqualified urls - i.e. addresses that contain only http://hostname (or https://hostname)

[gorhill]

If you add hostname as a whitelist directive, this will disable uBO for any document which URL is like http://hostname/[...], https://foo.hostname/[...].

[lewisje]

It sounds like what he wants to do is whitelist just those URLs that have a single label in their hostnames, which I think would need regex support, and because you explicitly don't support $document in ordinary whitelist rules (which, unlike directives, do support regexes), there's no way to do that.

If it were supported, here's what it might look like, if supporting the old syntax with a username and password in the URL (also, using aa* instead of a+ because the former is faster to test, and using ?: throughout for non-capturing groups, to reduce memory usage):

@@/^http(?:s)?\:\/\/(?:[\w_-][\w_-]*(?:\:[\w_-][\w_-]*)?@)?[\w_-][\w_-]*(?:\:\d{,5})?\//$document

For no password...

@@/^http(?:s)?\:\/\/(?:[\w_-][\w_-]*@)?[\w_-][\w_-]*(?:\:\d{,5})?\//$document

No username...

@@/^http(?:s)?\:\/\/[\w_-][\w_-]*(?:\:\d{,5})?\//$document

No explicit port...

@@/^http(?:s)?\:\/\/[\w_-][\w_-]*\//$document

I believe all of these will work in Adblock Plus and AdBlock; however, those rules, without the $document specifier, might do the job in uBlock Origin, as long as the intranet pages themselves weren't blocked to begin with, just subresources thereon.

[gorhill]

single label in their hostnames

Single label-hostname works as whitelist directive.

[lewisje]

_Particular_ single-label hostnames work, but there's no way to specify _all_ single-label hostnames in a single directive or in a reasonably sized number of directives.

[krisweir]

Hi - and thanks for the responses,

If regular expressions are a possibility, how about just testing for the absence of a “.” character in the host-only url?

I’m no expert, but I believe ^((?!.).)*$ should do this. Certainly seems to work on this regex test site: http://www.regextester.com/15

What do you think? I wonder how i.e. does it when you choose “bypass proxy server for local addresses” - I suspect it must be a similar test, as it will only do so for an address containing hostname with no domain (i.e. FQDN or IP address will not work, even if they are local.)

Chris.

From: James Edward Lewis II [mailto:[email protected]]
Sent: 12 May 2016 00:32
To: gorhill/uBlock
Cc: Chris Weir; Comment
Subject: Re: [gorhill/uBlock] Feature Request: Whitelist wildcard ips (#1578)

It sounds like what he wants to do is whitelist just those URLs that have a single label in their hostnames, which I think would need regex support, and because you explicitly don't support $document in ordinary whitelist rules (which, unlike directives, do support regexes), there's no way to do that.

If it were supported, here's what it might look like, if supporting the old syntax with a username and password in the URL (also, using aa* instead of a+ because the former is faster to test, and using ?: throughout for non-capturing groups, to reduce memory usage):

@@/^http(?:s)?://(?:[w_-][w_-]_(?::[w_-][w_-]_)?@)?[w_-][w_-]*(?::d{,5})?//$document

For no password...

@@/^http(?:s)?://(?:[w_-][w_-]_@)?[w_-][w_-]_(?::d{,5})?//$document

No username...

@@/^http(?:s)?://[w_-][w_-]*(?::d{,5})?//$document

No explicit port...

@@/^http(?:s)?://[w_-][w_-]*//$document

I believe all of these will work in Adblock Plus and AdBlock; however, those rules, without the $document specifier, might do the job in uBlock Origin, as long as the intranet pages themselves weren't blocked to begin with, just subresources thereon.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHubhttps://github.com/gorhill/uBlock/issues/1578#issuecomment-218620296

[gorhill]

What do you think?

I want to know _exactly_ what you are trying to solve, it still is not clear to me. Use a real example of what exactly is the issue to make me understand (I do not intend to allow regexp as whitelist directives.)

[lewisje]

krisweir, that would fail to match URLs that have dots in path components, filenames, and query strings, even if they're on servers with single-label hostnames.

[krisweir]

Yes, I know - this could be a problem.

For network hardware there is almost never a dot in the pathname, but other stuff like our Intranet and PC helpdesk system does tend to use .aspx and .php endings unfortunately.

I wonder how Microsoft do it for the “bypass proxy-server for local addresses” option in i.e?

Chris.

From: James Edward Lewis II [mailto:[email protected]]
Sent: 12 May 2016 14:39
To: gorhill/uBlock
Cc: Chris Weir; Comment
Subject: Re: [gorhill/uBlock] Feature Request: Whitelist wildcard ips (#1578)

krisweir, that would fail to match URLs that have dots in path components, filenames, and query strings, even if they're on servers with single-label hostnames.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHubhttps://github.com/gorhill/uBlock/issues/1578#issuecomment-218759626

[krisweir]

OK,

uBlock (origin) is very useful for stopping annoying ads, popups etc on Internet websites - for which, many thanks! ☺

However, about half the time I’m using a browser (on PC or Mac mainly) it is to administer network equipment (e.g. routers and switches) through a Web UI. This is normally on a local LAN - or a VPN-connected private network.

Also, we have an Intranet which also links into our Oracle database and several other Web-interfaced things, such as Wi-Fi management consoles etc, etc.

To ensure that some of the functionality of these internal UIs is not blocked by uBlock, I have to switch uBlock off for each address. Note - these are either naked IP addresses or a single hostname (with no spaces.)

This may not sound like a lot, but I do have 300+ of these network devices or other web-UI interfaces that I use, and around 10 different PCs/Macs that I may be using, so I may have to do this about 3000 times. And repeat this process for any new device or new PC etc…
The raw IP stuff is easy enough to handle. I’ve just added the 512 IP addresses (generated using Excel, from 10.2.3.1 to 10.2.4.254) that I use for network equipment to the white list. This works, as it’s really only network gear that I would normally connect to by IP address.
As I said, I have to do a similar thing in Java control panel to allow Java to work in the browser with these devices (and now Chrome won’t allow it at all…)

The unqualified hostnames, however I still have to do manually. As I mentioned, Microsoft offer the option in Internet Explorer to bypass a proxy server for local addresses. I imagine this is for similar reasons - i.e. so stringent rules concerning what is permitted on Internet websites is not imposed on the company Intranet, for example.

I guess I could just cut and paste all the fixed DNS entries from our DNS server and paste the hostnames into the uBlock whitelist.

Cheers,
Chris.

[monolifed]

CIDR notation might be better than wildcard for IP blocks

[lewisje]

That would definitely require new syntax, compared to the regexes I posted earlier.

[gorhill]

I went with the most generic solution for all those cases which are not covered by the existing directive syntax, so that I don't have to revisit again in the future.

A directive which starts and ends with a / will be treated as a regex-based directive. uBO will never use such directive, so it's all on the user to make good use of it. Using OP's issue as examples:

For 192.168.0.*, use /^https?://192\.168\.0\.\d+// (no need to escape forward slashes).

For 10.0.*, use /^https?://10\.0\.\d+\.\d+//.

uBO typically caches and reuses the result of the 1st evaluation for a given page, so overhead of using a regexp should be a non issue. Still, typically _only_ advanced users should venture to use a regex, and _only_ when no other directive syntax will work, and be sure that the regex is efficient (for example, anchoring to the start of the string in the above regex makes it efficient).

[RoxKilly]

@gorhill Is this to be used in the My Rules tab? For instance:

* /.*//192\.168\.0\.\d+// * noop
/.*//192\.168\.0\.\d+//  * 1p-script noop

[gorhill]

Here is about whitelist directives. Rules will never be based on regex, only on plain hostnames/IP addresses.