Uassets: ABP Japanese filter breaks facebook posts' option menu

[toshiya44]

URL(s) where the issue occurs

Any Facebook post. (You have to be logged in)
Made some posts here for example purposes: https://www.facebook.com/NOTHINGNOTHINGNOTHING/

Describe the issue

Due to ABP Japanese Filter's https:*_ad_ rule, the menu of Facebook posts' are blocked.

Screenshot(s)

Without ABP Japanese Filter

With ABP Japanese Filter (loads forever)

Logger screenshots:

Versions

• Browser/version: Firefox 49, 51 (dev-edition), 52 (Nightly)
• uBlock Origin version: v1.9.16

Settings

• Default + JPN: ABP Japanese filters (日本用フィルタ)‎

Notes

I tested on a fresh firefox profile with only uBlock Origin installed.

Adding the following filters solves the issue, [Warning: Exception filters are prone to abuse.]

! unblock options menu in timeline
@@||www.facebook.com/timeline/feed/options_menu/$xmlhttprequest,first-party
! unblock options menu in facebook pages
@@||www.facebook.com/pages/feed/options_menu/$xmlhttprequest,first-party
! unblock options menu in facebook groups
@@||www.facebook.com/ajax/groups/mall/mall_post_menu.php$xmlhttprequest,first-party

Judging by the first two filters, if facebook creates more types of pages, the options menu request might follow the pattern. So maybe the following will be a better choice.

@@||www.facebook.com/*/options_menu/?button*$xmlhttprequest,first-party

@@||www.facebook.com/ajax/groups/mall/mall_post_menu.php$xmlhttprequest,first-party

It has been reported on their GitHub repo before. See issue #20.
The maintainer of the list has refused to solve the issue and directed users toward their support policy, which stated they won't fix any issues reported by anyone unrelated to Japan and any Facebook related issues.

[ghost]

https://github.com/gorhill/uBlock/issues/1537
Maybe a fix could be added here but it really should be fixed in their list. Maybe they'll do it now that you suggested the filters.

[ddraws]

This list has quite a few really wild filters.

and to top that off, they won't listen to any issues raised by non-Japanese and not reported via ABP's reporting system.

[gorhill]

it really should be fixed in their list

Absolutely. It's not realist to ask anybody else to take the burden of creating _exception_ filters just for that one filter list -- the maintainers of that list are completely responsible to ensure false positives in their list are taken care of. I do this for _Peter Lowe's_ because it is enabled by default, and its a hosts list which has no concept of exception filters.

At most, I am open to remove that list from uBO's set of filter lists.

[levicki]

I reported the original issue to them. They first closed the issue, but then reopened it, which in my opinion means they are at least considering it. The JPN adblock list maintainer doesn't use Facebook so he is not really motivated to fix it. I will try to ask them nicely to include the exclusion rules provided by toshiya44.

[gorhill]

I think a better solution would be to use the ~ domain option rather than an exception filter. Does that work?

https:*_ad_$domain=~facebook.com

This way, the filter won't apply on facebook.com, so a filter list maintainer does not need to test with facebook.com (since the filter was not meant to target facebook specifically), just need a confirmation from the reporter the filter is not applied on facebook.com.

[levicki]

@gorhill
I proposed the domain exclusion. If he disagrees with that as well it might be best to remove his list from subscription page entirely, since with that attitude sooner or latter it is bound to break something else more important than Facebook and then we are back to square one.

Maybe you could add Fanboy's Japan list instead?

[toshiya44]

I can confirm that https:*_ad_$domain=~facebook.com doesn't block any requests in facebook.com

[levicki]

@gorhill
Author of the list does not want to change it so I closed the issue. He suggested using one of the other lists. What now?

[gorhill]

What now?

The way I see it is that it's a very concrete case of supporting the ability to disable specific filter -- something I have declined to implement in the past. I need to think about what would be the best long-term solution code- and UI-wise.

[toshiya44]

Something like a filter blacklist to ignore specific filters would be great. That way we would be able get rid of very broad filters like ad_ easily. (Many false positives in websites that generate dynamic urls, such as webassign.net)
I doesn't seem realistic to contact every filter list owner and negotiate with them every time there's an issue.
(This should be discussed in a separate issue I think)

[levicki]

Frankly, I think not using bad lists is much better option. Implementing what you said would just encourage people to make and use more bad lists and then complain about memory and performance issues.

I would really prefer if you would reconsider adding regex support for uMatrix than waste time implementing workarounds for broken filter-lists.

[gorhill]

waste time implementing workarounds for broken filter-lists

This is something which has been asked before, that wouldn't be specifically to address the issue here. It is also useful for filter list authors, to quickly disable a filter to see the effect. I resisted adding this before because it does not fit well in uBO's static filtering engine. However meanwhile I have been thinking there could be a way to implement this with zero memory and CPU overhead to the static filtering engine, which is the main reason I refused.

adding regex support for uMatrix

This would kill uMatrix's filtering efficiency. Simple lookup are used for rules, something not possible with regexes. Also, what you see in uMatrix is what you get. With regexes, it's no longer possible, a cell could end up maybe being red, maybe being green, i.e. not being able to show a user exactly whether a cell will block or allow network requests.

[gorhill]

Due to ABP Japanese Filter's https:*_ad_ rule, the menu of Facebook posts' are blocked.

A new filter option is available as of 1.11.3b2: badfilter, to prevent a filter from being loaded in memory.

Using the current case as an example, one would use https:*_ad_$badfilter to prevent the filter https:*_ad_ from being loaded in memory. The badfilter option indicates to uBO to not load in memory the functional form of a filter. For example, https:*_ad_$badfilter would not prevent loading into memory the filter https:*_ad_$script, or https:*_ad_$domain=example.com.

Now I will use such filter in _"uBlock filters -- Unbreak"_ only if needed and only for the common set of default filter lists (a recent case would have been .php?ref= => .php?ref=$badfilter in EasyPrivacy, but it has been removed). This means you will still have to unbreak yourself the current case, using the badfilter option.

[uBlock-user]

I take it $badfilter cannot be used on cosmetic filters or can they ?

[gorhill]

No.

There is no need for such option anyways, cosmetic exception filter #@# have exactly the same effect -- to cancel the corresponding hide filter, except that this can be narrowed to a specific hostname.

[trlkly]

The new filter tech is good. But I think something should still be done about having this bad filter in the set. And, if this designed for ADP, it should be reported there, too.

This is a maintainer who is actively hostile to the purpose of an adblock list. If anyone reports a false positive, their job is to fix it. There is no call for deliberately keeping it broken because someone who speaks the wrong language reports it.

If they can't keep up to basic quality standards, their list should be removed. And possibly those who use it warned of the low quality.

[zcyzcy88]

The filter is designed for Japanese only, it pollutes non-Japanese.

So why not just provide a option:
[ don't | just ] load the whole blacklist for specially domains & languages

[smed79]

You should Live in Japan or Speak Japanese to get support.
Do not support websites which i do not have the time to create an account for.

If the maintainer don't care to fix issues caused by the bad too generic filters , the list should not enabled by default. And the user who want to manually subscribe to this list should be warned about the bad/low quality to avoid reporting issues on the wrong forum (uBO/uAssets in this case).