Ublock: Add "Ignore generic cosmetic filters" option in the "3rd-party filters" pane

[gorhill]

Generic cosmetic filters are at the root of the largest overhead added by uBO to web pages. Being able to disable generic cosmetic filters without disabling entirely cosmetic filtering would be very useful to lower the overhead added by uBO to web pages, without sacrificing completely cosmetic filtering.

Low powered devices are likely the biggest benefactors of such feature.

[gorhill]

For translators:

[gwarser]

This works like no-cosmetic-filtering: * true?

My mistake, I was thinking about generichide (or ublock version of elemhide) behavior.

[gorhill]

This works like no-cosmetic-filtering: * true?

No. the no-cosmetic-filtering switch is to disable completely cosmetic filtering, specific and generic.

As requested by @Bushido1, I am trying to come up with benchmark results to show the effect of not having to enforce generic cosmetic filtering. Thing is, for mainstream desktop computers, the effect is probably not really noticeable on even bloated pages (I am still trying to find one for reference). I expect this will make a good difference for less powerful devices though. I could benchmark with my decade-old laptop to show how it helps on less powerful devices.

[gorhill]

Mainly, this option allows uBO to overall run more efficiently, _without_ sacrificing completely cosmetic filtering (which is what the no-cosmetic-filtering switch does).

Conceptually, this is the equivalent of applying the generichide filtering option to all web sites. So an incidental side-effect might be to foil anti-blocker code used on many sites (the generichide option is often used for that purpose).

I will drop here various results as I run benchmarks. Things to keep in mind:

• uBO caches generic cosmetic filters which are found to be relevant to a site, so better performance is common after loading more than one page on the same site.
• What seems to be completely marginal improvement for a mainstream desktop computer (as in, no user will ever notice a difference), is probably significant for less powerful devices, especially those running on batteries.
• Even when having to handle generic cosmetic filters, uBO's added memory/CPU overhead to web pages is lower than Adblock Plus' added overhead -- especially memory-wise, and especially on Chromium-based browsers.

The timings below are the extra time added by uBO's content script to page load completion in the browser (figures are approximate, timings vary slightly from one page load to the next).

• http://www.nytimes.com/2016/08/13/business/dealbook/bitcoin-blockchain-banking-finance.html?ref=technology
  
  
  
  • 4yr-old desktop computer Chrome 54:
  
  
  • With generic cosmetic filtering: uBO cost ( top): 72.0ms
  
  
  • Without generic cosmetic filtering: uBO cost ( top): 35.7ms
  
  
  • 4yr-old desktop computer Nightly:
  
  
  • With generic cosmetic filtering: uBO cost ( top): 98.3ms
  
  
  • Without generic cosmetic filtering: uBO cost ( top): 28.7ms
  
  
  • 10yr-old laptop Chromium 51:
  
  
  • With generic cosmetic filtering: uBO cost ( top): 728.7ms
  
  
  • Without generic cosmetic filtering: uBO cost ( top): 268.3ms
  
  
• http://people.mozilla.org/~bgirard/doxygen/gfx/annotated.html (unusually large DOM, no ads on this site)
  
  
  
  • 4yr-old desktop computer Chrome 54:
  
  
  • With generic cosmetic filtering: uBO cost ( top): 204.0ms
  
  
  • Without generic cosmetic filtering: uBO cost ( top): 80.7ms
  
  
  • 4yr-old desktop computer Nightly:
  
  
  • With generic cosmetic filtering: uBO cost ( top): 679.5ms
  
  
  • Without generic cosmetic filtering: uBO cost ( top): 284.1ms
  
  
  • 10yr-old laptop Chromium 51:
  
  
  • With generic cosmetic filtering: uBO cost ( top): 1138.1ms
  
  
  • Without generic cosmetic filtering: uBO cost ( top): 411.2ms
  
  
• https://news.ycombinator.com/
  
  
  
  • 4yr-old desktop computer Chrome 54:
  
  
  • With generic cosmetic filtering: uBO cost ( top): 29.0ms
  
  
  • Without generic cosmetic filtering: uBO cost ( top): 25.3ms
  
  
  • 4yr-old desktop computer Nightly:
  
  
  • With generic cosmetic filtering: uBO cost ( top): 27.0ms
  
  
  • Without generic cosmetic filtering: uBO cost ( top): 14.3ms
  
  
  • 10yr-old laptop Chromium 51:
  
  
  • With generic cosmetic filtering: uBO cost ( top): 356.2ms
  
  
  • Without generic cosmetic filtering: uBO cost ( top): 204.3ms
  
  
• http://blogs.wsj.com/cio/2016/08/12/is-there-a-stem-crisis-or-a-stem-surplus/
  
  
  
  • 4yr-old desktop computer Chrome 54:
  
  
  • With generic cosmetic filtering: uBO cost ( top): 81.0ms
  
  
  • Without generic cosmetic filtering: uBO cost ( top): 37.6ms
  
  
  • 4yr-old desktop computer Nightly:
  
  
  • With generic cosmetic filtering: uBO cost ( top): 131.2ms
  
  
  • Without generic cosmetic filtering: uBO cost ( top): 40.6ms
  
  
  • 10yr-old laptop Chromium 51:
  
  
  • With generic cosmetic filtering: uBO cost ( top): 585.5ms
  
  
  • Without generic cosmetic filtering: uBO cost ( top): 332.4ms
  
  

I will provide timings for an old laptop eventually -- to provide a feel of how this helps less powerful devices. Done.

Note that I realized after ward that for the 4yr-old desktop computer, I had uBO configured to block facebook.com and twitter.com everywhere. For the 10yr-old laptop, all default settings.

Keep in mind that the timings above represent the amount of time the browser spend executing uBO's content script code, there is no idle time in there, so this means for example that the CPU was working at 100% for the time value displayed. Hence, the shorter time the better, and this has a direct effect on battery life.

The consequence of not loading generic cosmetic filters is that _maybe_ some pages won't have some areas collapsed, due to the lack of generic cosmetic filters, but given what is gained in return for less powerful devices, this is a welcomed option. I did not notice any visual quirk for the tested web pages.

For an example of the effect of not loading generic cosmetic filters, there is a gap at the top of https://www.theguardian.com/international -- but easily removable with the element picker (which always create specific cosmetic filters -- though this manually crafted one is better: theguardian.*##.top-banner-ad-container).

This option will be enabled by default for Firefox for Mobile at install time (no change for an existing installation).

[Bushido1]

But Adguard user Avatar noted that this setting has a very small effect on old laptops: https://forum.adguard.com/index.php?threads/add-ignore-generic-cosmetic-filters-option.12747/#post-96803

So, is 0.2% - 0.8% worh it or it makes uBlock addon to use more RAM?

[gorhill]

This option is likely most useful for Firefox for Android -- which I can't benchmark, hence why I benchmarked an old laptop. The benchmarks do not tell the whole story: it measured the content-side overhead, not the extension-side overhead. These are in separate processes on Chromium and Nightly, and measuring them at the same time of the content script is not easy.

Here is an overview of how uBO roughly works with regard to generic cosmetic filtering:

1. uBO's content script injected into page.
2. uBO's content script surveys the page.
3. Survey data is sent to uBO's core.
4. uBO's core looks-up generic filters matching survey data.
5. uBO's core returns matching generic cosmetic filters to uBO's content script.
6. uBO's content script applies the matching generic cosmetic filters.

In my benchmarks, point 4 above was not measured. Also, possibly parts of points 3. and 5. was not measured (inter-process messaging). If not loading generic cosmetic filters, point 2. to 6. are skipped.

Also, uBO needs to keep surveying for when the DOM is mutated. Long-lived pages will benefit more than others from not loading generic cosmetic filters. Examples of such pages (more and more common): github.com, youtube.com, mail.google.com, etc.

When I make such decision, it's because of profiling uBO against itself: generic cosmetic filters are the largest overhead compared to other parts of uBO's content script. So I am just offering an option to remove that largest overhead.

Some users prefer to use _"EasyList without element hiding rules"_ because of the overhead of cosmetic filtering. I am offering a similar option for all filter lists, but only for the cosmetic filters which are expensive to process, the generic ones, so one can still benefit cosmetic filtering.

Blockers which inject unconditionally all generic cosmetic filters (17,000+ currently in EasyList) into every page and frames on a page would benefit much more than uBO of such a feature -- this includes AdGuard.

Also, @@|http$generichide does not prevent the loading in memory of all generic cosmetic filters -- this means they would be all residing into memory, never to be used.

[harshanvn]

Maybe 'about:performance' will help in analyzing the performance in firefox. Just an another data point.

Anyways, I will try this in the next week, when it is released.

[gorhill]

So, by using that rule, you've spent 250 extra CPU seconds on it. Which, in turn, is equivalent to about 10-20mAh. Old laptop battery is about 2500-4000 mAh. So you have saved about 0.2% - 0.8% of the battery.

I will admit I do not understand his calculation. If 250 seconds at 100% CPU draws 10-20 mAh, then this would mean in the _worst_ case a 2500 mAh battery can provide 2500 / 20 = 125 chunks of 250 seconds with CPU at 100%, meaning 125 x 250 = 31250 seconds = a CPU at 100% for more than 8 hrs before the battery runs out. Really? (best case would be 4000 / 10 * 250 = 27 hours).

[Bushido1]

Blockers which inject unconditionally all generic cosmetic filters (17,000+ currently in EasyList) into every page and frames on a page would benefit much more than uBO of such a feature -- this includes AdGuard.

Here it is stated that ,,Adguard has an algorithm of determining what to inject in frames to speed up browsing." So, it is not injecting unconditionally all, but determines via algorithm, what to inject?

[gorhill]

So, it is not injecting unconditionally all, but determines via algorithm, what to inject?

Ok, I admit I do not know exactly how AdGuard works internally, didn't look at the code. I know though that for the web pages I opened, there were two style tags with a whole lot of generics in them. For example: https://news.ycombinator.com/ (no ads in there):

Same thing on github.com, google.com, etc.

You can test for yourself, just open a page, open dev inspector, see shadow-root of html tag.

[ameshkov]

Hi @gorhill, long time no see:)

I will admit I do not understand his calculation. If 250 seconds at 100% CPU draws 10-20 mAh, then this would mean in the worst case a 2500 mAh battery can provide 2500 / 20 = 125 chunks of 250 seconds with CPU at 100%, meaning 125 x 250 = 31250 seconds = a CPU at 100% for more than 8 hrs before the battery runs out. Really? (best case would be 4000 / 10 * 250 = 27 hours).

Let me elaborate a bit on my calculation. It is based mostly on my Android development experience.

Battery usage stats in Android are built according to the so-called "power profile" values:
https://source.android.com/devices/tech/power/

Here are some examples:
https://source.android.com/devices/tech/power/values.html

The formula for battery usage calculation looks like this (cpu.active is a power profile coefficient):
"CPU TIME (ms)" X "cpu.active" / (60 * 60 * 1000) = "POWER USE mAh"

So, for 250 seconds cpu time and cpu.active= 200mA it will be:
250000 * 200 / 60 / 60 / 1000 = 14mAh

It appears that for Android CPU time is very-very cheap comparing with the spendings on network (cellular data is the main battery hog).

[ameshkov]

Regarding the original issue with the generic filters you've implemented. We've tried it in AG for iOS and, frankly, I didn't like the outcome. Generic filters are widely used in some filters and disabling it has a huge impact on how page looks. For instance, fanboys social/annoyances and prebake consist mostly of generic filters.

If you want, we could provide you with the "optimized" filters we use for mobile apps. Here is a more detailed description:
https://forum.adguard.com/index.php?threads/add-ignore-generic-cosmetic-filters-option.12747/#post-97095

We'll need some time to prepare these filters for public usage though.

[vlakoff]

Just to report my experience, disabling the generic cosmetic filters diminishes a lot the hiding of the (infamously stupid and annoying) EU cookie warnings, as I discovered during #1609.