Adguardfilters: uBO version of Adguard filters

[ameshkov]

Hi @gorhill, as I see you're considering adding AG English filter to uBO stock lists: https://github.com/gorhill/uBlock/issues/2624

We'd like to make some preparations from our side. We have different "versions" of all AG filters for different platforms. For instance, this is the version for Chromium extension -- https://filters.adtidy.org/extension/chromium/filters/2.txt and this is for Android app -- https://filters.adtidy.org/android/filters/2.txt

I suppose it would be better if we prepare a version for uBO specifically, and cover possible incompatibilities there. Something like /ublock/filters/2.txt. With the hints support it'd be quite easy to do.

@gorhill is there anything you see which should be handled right now? Nested :has or :not(:has) for instance.

[gorhill]

I suppose it would be better if we prepare a version for uBO specifically, and cover possible incompatibilities there.

I appreciate the thought, but really you shouldn't worry about compatibility, uBO will discard what it can't understand. I don't want to incur any extra workload on anybody, so if ever there is an issue with uBO proper as a result of importing an Adguard list, this will be for me to deal with.

If anything, it's me worrying about incurring a hit on remote servers serving lists, I try to be considerate about this, by using minimal number of days between update, if you have a preference for this, just tell me (currently default is ~5 days).

I didn't realize you were supporting :not(:has(...)), but it seems to me I could simply translate this on my side to :if-not(...), so really no need to change anything on your side.

[ameshkov]

@gorhill

It's not just about one-sided compatibility. It is possible that if an issue is reported by a uBO user, it might be resolved with a uBO specific rule (something like redirect=noopjs for instance), and it'd be better for filters structure to have them in a separate filter file.

I don't want to incur any extra workload on anybody, so if ever there is an issue with uBO proper as a result of importing an Adguard list, this will be for me to deal with.

It is really not a big deal, the process is automated, just one more line in the config file.

Here it is already (extension/ublock/filters can be used for any AG filter):
https://filters.adtidy.org/extension/ublock/filters/2.txt

At first, we'll have there a copy of AG chrome extension filters, then if there is any uBO-specific rule, filters maintainers can place it there with a hint instruction.

If anything, it's me worrying about incurring a hit on remote servers serving lists, I try to be considerate about this, by using minimal number of days between update, if you have a preference for this, just tell me (currently default is ~5 days).

Yeah, it might be important. Current load is about 4-5TB daily, 2-3TB more and we are on the edge of the current tariff plan.

Can you take ! Expires: attribute into account so that we could manage it from the filter side?
Also, it's one more advantage of having a separate product/platform-specific directory, we can keep an eye on the servers load and report if something goes wrong.

[ameshkov]

@gorhill

Hi! So, after all, here are the filters for uBO, I'd be grateful if you use this location for all AG filters.

• English: https://filters.adtidy.org/extension/ublock/filters/2.txt
• Russian: https://filters.adtidy.org/extension/ublock/filters/1.txt
• Turkish: https://filters.adtidy.org/extension/ublock/filters/13.txt

Filters are the same as we use in Chrome extension + hints now support UBO:
https://github.com/AdguardTeam/AdguardFilters/issues/3708#issuecomment-303155314

[gorhill]

Thanks for doing this even if you do not have to.

Yes, uBO takes into account ! Expires:, but internally it uses a minimum of 2 days in order to be sure users can't abuse the directive to unduly force an update too often.

[gorhill]

Just so I understand correctly, is Adguard English filter list a generic filter list like EasyList which must be used for all locales, or is it just useful for English locale?

[ameshkov]

Yes, uBO takes into account ! Expires:, but internally it uses a minimum of 2 days in order to be sure users can't abuse the directive to unduly force an update too often.

Great, thank you!

Just so I understand correctly, is Adguard English filter list a generic filter list like EasyList which must be used for all locales, or is it just useful for English locale?

Yes, it is generic. We are actually considering renaming it into "Adguard Base filter + EasyList" just to make it clear.

[gorhill]

Would it be trivial to avoid appending EasyList filters to Adguard filters for https://filters.adtidy.org/extension/ublock/filters/2.txt specifically?

I would like to offer Adguard filters but without duplicating EasyList content, as it I don't want to wholly replace EasyList with Adguard English, but rather offer Adguard English on top of EasyList (which is selected by default and is even part of the package). EasyList is a high profile, well-known filter list and I would like to keep it clear this is what uBO uses by default.

Currently, when I import Adguard English, I get "10,062 used out of 84,979", so that means most of the parsing effort is wasted. Another nice side effect of not appending EasyList filters would be to consume less server bandwidth when the list is downloaded.

[ameshkov]

@gorhill Sure it's possible, but some of the EasyList rules are removed from the English filter. We should add some badfilter rules instead. Not many, but anyway. I'll report back once it's done.

[ameshkov]

@gorhill here you are:
https://filters.adtidy.org/extension/ublock/filters/2_without_easylist.txt

badfilter rules are not there yet, though. Leaving this issue open until they are.

[gorhill]

@ameshkov Thank you very much, I've now added the list as "Adguard Generic Filters" in uBO, but the list of lists update once every 13 days, so it will take that long before majority of users sees it in the choice of lists.

Out of curiosity, is there a lot of filters removed from EasyList, and typically what are the removed filters? If it's trivial and not crucial to the proper functioning of Adguard filters, I say do not bother with badfilter, at some points I don't want you guys to worry too much about making the list work with uBO, it's rather me who should make the efforts to work with your lists.

[ameshkov]

Out of curiosity, is there a lot of filters removed from EasyList, and typically what are the removed filters?

Not too much, as I recall it's about ~50-100 filters at the moment.

If it's trivial and not crucial to the proper functioning of Adguard filters, I say do not bother with badfilter, at some points I don't want you guys to worry too much about making the list work with uBO, it's rather me who should make the efforts to work with your lists.

It's not too important indeed, everything should be okay without them. We will anyway do the conversion of these exclusions into $badfilter in due course (once AG starts supporting the $badfilter (kudos for the idea btw)).

[happysurf]

I don't understand the difference between English Filter (about 80.000 strings) vs Generic Filter for UBO (only 10.000 string).
The efficiency is the same or what?
Is possible use both together or is useless?

[Alex-302]

@happysurf Generic Filter for UBO must be used with Easylist. Or you can add full Adguard's English filter
https://filters.adtidy.org/extension/ublock/filters/2.txt

[gorhill]

@happysurf

I don't understand the difference between English Filter (about 80.000 strings) vs Generic Filter

Just read the thread carefully, see what was said above. I used "Generic" instead of "Base", I changed the name.

[mikhaelkh]

Is there a complete list for all uBO-optimized filterlists? My understanding is I can change chromium to ublock in links from here. Since uBlock Origin doesn't allow to edit titles, these custom filterlists get mixed up with filterlists from other sources. Can you add Adguard to all titles in ublock subfolder?

[gmarcher]

is there any plan for adguard social media and spyware too? i like both, because it's minimal and enough for my language (IDN)

[uBlock-user]

You can also add Social Media Filter and Mobile Ads Filter under their respective sections available.

[ameshkov]

Is there a complete list for all uBO-optimized filterlists? My understanding is I can change chromium to ublock in links from here.

All filters have a uBO-specific version in the /ublock/directory so that filters maintainers could use hints: https://kb.adguard.com/en/general/how-to-create-your-own-ad-filters#hints

[ameshkov]

You can also add Social Media Filter and Mobile Ads Filter under their respective sections available.

Annoyances filter is also nice. The problem with it is that it relies a lot on JS rules (for anti-adblock scripts).

Mobile ads filter is more about blocking in-app ads, but it has some web-specific rules as well.

[gotitbro]

@ameshkov Is the Spyware filter based on EasyPrivacy? Does it differ significantly?

[gorhill]

@gotitbro Why not select both lists and see for yourself the amount of duplicates?

[ameshkov]

@gotitbro yeah, it is based on EasyPrivacy originally, we don't sync them tho

[gotitbro]

@ameshkov So any new filters added in EasyPrivacy won't benefit the Adguard Spyware List? Also any plans on offering the same without EasyPrivacy as with the English filter?

[ameshkov]

So any new filters added in EasyPrivacy won't benefit the Adguard Spyware List?

Yep, it won't.

Also any plans on offering the same without EasyPrivacy as with the English filter?

Back then when we hard-forked EasyPrivacy it was causing too many issues with websites functioning. I've tried using it recently and the situation has greatly improved, so you may be right about it.

[patrickdrd]

I'm trying to use the same filters both on chrome (ublock, desktop) and android (AG 2.12),
so, after reading this I added the following in chrome ublock (which should be the same as what I'm using in android):

`AdGuard Base filter
https://filters.adtidy.org/android/filters/2_optimized.txt

AdGuard Mobile Ads filter
https://filters.adtidy.org/android/filters/11_optimized.txt

AdGuard Spyware filter
https://filters.adtidy.org/android/filters/3_optimized.txt

AdGuard Annoyances filter
https://filters.adtidy.org/android/filters/14_optimized.txt

(Obsolete) Anti-Adblock Killer | Reek
https://raw.githubusercontent.com/reek/anti-adblock-killer/master/anti-adblock-killer-filters.txt

void.gr
http://www.void.gr/kargig/blog/greek-adblock-plus-filter/`

well, they're not the same,
for example in the page:
https://www.apkmirror.com/apk/dashlane/dashlane-1-password-manager/dashlane-1-password-manager-6-1848-1-release/dashlane-free-password-manager-6-1848-1-armeabi-v7a-android-apk-download/
adblock is detected, while it doesn't on android

I'm also using ublock annoyances, badware risks, privacy, resource abuse and unbreak and some of my own filters, all of them in both locations,
why is the behavior different?
what can I do to make them more of less, "same" (in order to deploy custom filters from desktop,
because it's impossible to do so from android)?

edit: it isn't detected if I use https://filters.adtidy.org/extension/chromium/filters/2.txt