Uassets: Advanced anti-adblock techniques on many CZ local servers

Created on 6 Jul 2018 · 27Comments · Source: uBlockOrigin/uAssets

URL(s) where the issue occurs

https://www.idnes.cz/
https://www.novinky.cz/
https://www.aktualne.cz/
https://ihned.cz/
https://www.info.cz/
http://blesk.cz/
http://ahaonline.cz/
https://www.ahaonline.cz/
https://www.extra.cz/
https://www.super.cz/
https://www.zive.cz/
http://isport.blesk.cz/
http://www.auto.cz/
http://www.reflex.cz/
https://www.e15.cz/
https://avmania.e15.cz/
https://www.maminka.cz/
https://www.mobilmania.cz/
https://www.centrum.cz/
http://atlas.centrum.cz/
http://www.volny.cz/
http://tiscali.cz/
http://games.tiscali.cz/
http://nedd.tiscali.cz/
https://cestovani.tiscali.cz/

Describe the issue

Advanced anti-adblock techniques, elements and scripts names randomizing, delayed ads injection by js, broken page loading, never finish, ABP flag stored in cookie
Pages work with adblocker on 1st load without cookie
Ad blocker detected even when disabled

Screenshot(s)

Versions

FF 54.0.1 uBO 1.16.12
Dragon 67.0.3396.99 uBO 1.16.12

Settings

cloud on advuser on prevent leak IP via WebRTC on block CSP on

Notes

possibly utilized common framework of https://www.cpex.cz/

Source

trumpeta

👍5

Most helpful comment

if uBO is too weak to defend this

It's not a matter of being "weak", as already stated it's a matter of someone willing to invest the time to investigate and work out solutions on a regular basis -- this is also true of the user script approach.

gorhill on 17 Jul 2018

👍2

All 27 comments

Here's a list of cosmetic filters that should block most of the ads on concerned sites, but broken page loading and injecting scripts are still present.
Moreover, the element picker not available in Firefox due to DOM structure not ready for uBO.

cz##div.ads
cz###Leader-1
cz##div.leaderboard
cz###HalfPageTop
cz###HalfPageBottom
cz##[id^="Billboard-bottom-"]
cz##[id^="Skyscraper-"]
cz##[id^="Commercial-"]
cz##[id^=Square-"]
cz##[id^="PR-perex-"]
cz##[id^="PR-premium-"]
cz##[id^="Fullbanner-"]
cz##[id^="Leaderboard-"]
cz###Partner-statistik
cz##[id^="Mobile-rectangle-"]
cz##[id^="510x90-klub-"]
cz###Rectangle480-1
cz###Rectangle-left-1
cz###Rectangle-right-1
cz###Cileni-na-znacku-1
cz###Otvirak-1
cz###Komercni-tip
cz###idvert

cz##div[class]:has(> div#Leader-1)
cz##div[class]:has(> div#HalfPageTop)
cz##div[class]:has(> div#HalfPageBottom)
cz##div[class]:has(> div#Billboard-bottom-1)
cz##div[class]:has(> div.banner)
cz##div[class$="banner"]
cz##div[class$="column"]:has(> div[class$="items"])

@@||idnes.cz^*/$script
@@||1gr.cz/reklama/banner.js$script,domain=idnes.cz
idnes.cz##div#main > rale[class]
idnes.cz###ads-sticky
idnes.cz##:xpath(//div[@*[starts-with(name(), 'data-')]])
idnes.cz##script:inject(set-constant.js, Ads.ok, true)
/idnes\.cz\/[\w-_]{30,}/$domain=idnes.cz,script
||1gr.cz/data/selfboxy/Akcniceny/*$image,domain=idnes.cz

zive.cz##div#br-192-018
info.cz##div#topSite
info.cz##div#leader_board
info.cz##div[class]:has(> div[class~="sticky-content"])
info.cz##div.article-advertisement
||info.cz/*.mp4|
reflex.cz##div#topSite
maminka.cz##div#content-lead
blesk.cz##div#ads-iframe
ahaonline.cz##div[id]:has(> div[id$="wrapper"][class$="anim"])

super.cz##div[class]:has(> div[class] > iframe[id^="im-"])
super.cz##div[id^="adPlachta"]

extra.cz##.centered.ad-leaderboard.ad
extra.cz##div.container:has(> div > div > * + div.ads:last-child)
||extra.cz/*/hb/cpex-hb-sas-cnc.js$script,first-party

centrum.cz##div[class="dolni-amplion clearfix"]
aktualne.cz##body > div > div > div > div[class]:has(> div[class] > div[class]:has-text(r.klama))
atlas.centrum.cz,www.volny.cz##.row.eHKepp.sc-eMigcr
tiscali.cz##div.wrap-advert
tiscali.cz##div[class~="btitle-right"]
games.tiscali.cz##[class]:has(> div[class][style="display:block/*ahoj*/!important"])
games.tiscali.cz##div[class][style="display:block/*ahoj*/!important"]
osobnosti.cz##div.hide-screenbot
osobnosti.cz##div[class~="box"]:has(> div[class] > div.sticky)
osobnosti.cz##div[class^="box "]:has(> div[id][class])
osobnosti.cz##div[id^="leader-bottom-"]:has(> div#lbttm)

trumpeta on 6 Jul 2018

👍1

I don't think something can be done with this. Hiding rules cause high cpu usage because anti-adblock is trying hard to override them, uBO scriptles don't work (timming issue). Blocking cookies seems to work (not everyone will like this).

https://github.com/uBlockOrigin/uBlock-issues/issues/53

I will like to make this to be meta issue for these _cz_ sites.

gwarser on 7 Jul 2018

👍1

Hiding rules cause high cpu usage because anti-adblock is trying hard to override them

That's why I was asking to remove the ad network framework scripts rather than hiding the already created elements. Not even mentioning the performance issues all these sites having.

trumpeta on 7 Jul 2018

Blocking cookies seems to work

Possibly have a scriptlet to deal with such issues ? @gorhill

mapx- on 7 Jul 2018

Possibly have a scriptlet to deal with such issues ?

uBO scriptles don't work (timming issue)

Scriptlets can't fix this, those scripts need to be blocked fully to even browse those websites. I don't see any other way.

uBlock-user on 7 Jul 2018

Then selectively blocking cookies beginning with adb_cook_ and adb.key?

trumpeta on 7 Jul 2018

@gwarser Doesn't uBO support $cookie directive ? I vaguely remember gorhill discussing that some time ago.

uBlock-user on 7 Jul 2018

Does HTML filtering work for these?

gorhill on 7 Jul 2018

seems to work:

for example (but delete first all the cookies for those sites)

idnes.cz##^script:has-text(adbDetect)
aktualne.cz##^script:has-text(adbDetect)

mapx- on 7 Jul 2018

👍1

Doesn't uBO support $cookie directive ? I vaguely remember gorhill discussing that some time ago.

I might have floated this in the past, but this was formally proposed by @ameshkov:
https://github.com/gorhill/uBlock/issues/1930#issuecomment-370276156

However I have strong reservation for now for such option: it requires to add pattern-filtering in both the onBeforeSendHeaders and onHeadersReceived listeners, and the expensive kind of pattern-filtering because like with the csp= option, uBO would need to find _all_ the matching cookie= filters, not just the first match as is the case with request blocking.

gorhill on 7 Jul 2018

ABP + uBo detection ?

        function Y() {
            var n = Object.getOwnPropertyDescriptor(Element.prototype, "shadowRoot");
            if (n && -1 === n.get.toString().indexOf("native code")) {
                T = "ABP";
                var e = S(!1);
                return !(!t.ABPe && !e) || (T += "-off", !1)
            }
        }

        function x() {
            var n, e = t.querySelectorAll("head>style:empty");
            for (n = 0; n < o(e); n++) {
                var r = e[n],
                    i = r && r.sheet.cssRules;
                if (o(i) && "display: none !important;" === i[0].style.cssText) return T = "uBlock", !0
            }
        }

mapx- on 7 Jul 2018

👍1

idnes.cz##^script:has-text(adbDetect)
aktualne.cz##^script:has-text(adbDetect)

These filters working fine in latest Firefox, but not in FF below 57.x and other browsers, another method to block?

trumpeta on 7 Jul 2018

👍1

for now just block the cookies for all those sites

mapx- on 7 Jul 2018

but not in FF below 57.x and other browsers

HTML Filtering only works on Firefox Quantum. For Firefox-legacy , add -

idnes.cz,aktualne.cz##script:contains(adbDetect)

uBlock-user on 7 Jul 2018

idnes.cz,aktualne.cz##script:contains(adbDetect

For some reason not applied in legacy browsers, tried with negative result in Opera 54.0.2952.51 (latest), Chrome 67.0.3396.99 (latest) and FF 54.0.1.
Only works the html filter in Firefox Quantum

trumpeta on 7 Jul 2018

👍1

HTML filtering is supported only FF 57+
read
https://github.com/gorhill/uBlock/wiki/Static-filter-syntax#html-filters

mapx- on 7 Jul 2018

ABP + uBo detection ?

Yes, there is code in there to find out whether the blocker is ABP-like or uBO-like. However that code is sort of obsolete since now both ABP/uBO use user stylesheets to implement cosmetic filtering.

gorhill on 7 Jul 2018

👍1

How about:

idnes.cz##script:inject(addEventListener-defuser.js, /^(beforeunload|load)$/)

This seems to work on my side (at least for that one site) if you close all tabs for idnes.cz and remove all cookies for idnes.cz; after adding the filter.

Edit: added load event to defuser.

gorhill on 7 Jul 2018

🎉1 👍1

Yes and works on other sites as well

idnes.cz,novinky.cz,aktualne.cz,ihned.cz,info.cz,reflex.cz,zive.cz,mobilmania.cz,e15.cz,centrum.cz,volny.cz,tiscali.cz,blesk.cz,ahaonline.cz,extra.cz,super.cz,auto.cz,maminka.cz##script:inject(addEventListener-defuser.js, /^(beforeunload|load)$/)

Please give a short tutorial how to find the things.

trumpeta on 7 Jul 2018

Please give a short tutorial how to find the things.

There is no tutorial possible, each case is its own. It just take willingness to spend time trying to figure how the javascript code work using browser dev tools, and a trial/error steps, starting with broad measures and refining what seems to work.

Note that CZE maintainer(s) may want (or not) to further narrow the defusing using one or more patterns found in the function(s) to defuse (3rd argument in the injected scriptlet).

Now expect that this may not last as they will again work around the blocking -- though same thing can be said of the cookie approach that has been suggested. Just the usual cat and mouse approach.

gorhill on 7 Jul 2018

👍1

For some reason not applied in legacy browsers, tried with negative result in Opera 54.0.2952.51 (latest), Chrome 67.0.3396.99 (latest) and FF 54.0.1

BeforeScriptExecute/AfterScriptExecute is not present, so ofcourse it will not work in Chromium based browsers. As for legacy builds of Firefox, it should, if not, you will need to install Firefox-legacy build on FF 54.

uBlock-user on 8 Jul 2018

This filter was working well for a while but is already workarounded on all sites.

I would prefer a more personal solution not published anywhere (at least not in official country list) as the ads supplier seems counter act quickly.
It's desirable to embarrass detection of current ad blocking solution to ad network maintainer even at price of impaired accessibility for user community.

Now expect that this may not last as they will again work around the blocking -- though same thing can be said of the cookie approach that has been suggested. Just the usual cat and mouse approach.

Exactly as you predicted.
As for cookie blocking, not acceptable due to

there's site look personalisation
user account login remembering

For anyone knowing JS also a solution by userscript is acceptable if uBO is too weak to defend this.

Updated list of domains:

idnes.cz
novinky.cz
aktualne.cz
ihned.cz
info.cz
reflex.cz
zive.cz
mobilmania.cz
e15.cz
centrum.cz
volny.cz
tiscali.cz
blesk.cz
ahaonline.cz
extra.cz
super.cz
auto.cz
maminka.cz
iprima.cz
nasepenize.cz
osobnosti.cz
karaoketexty.cz

trumpeta on 17 Jul 2018

if uBO is too weak to defend this

gorhill on 17 Jul 2018

👍2

Just an idea, does this work for anyone?

/logo.png$script,domain=idnes.cz
idnes.cz##+js(nowrite.js, <img src=")

nowrite.js application/javascript
(function() {
    var needle = '{{1}}';
    if ( needle === '' || needle === '{{1}}' ) {
        needle = '.?';
    } else if ( needle.slice(0,1) === '/' && needle.slice(-1) === '/' ) {
        needle = needle.slice(1,-1);
    } else {
        needle = needle.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
    }
    needle = new RegExp(needle);
    realWrite = window.document.write;
    window.document.write = function(a) {
        if ( !needle.test(a.toString()) ) {
            console.log('uBO-write: ' + a);
            realWrite.call(window.document, a);//?
        }
    }.bind(window.document);
})();

https://gist.githubusercontent.com/gwarser/6176ec76f210178b611efee40e6e15f4/raw/f843bc3bbf090322780037552085c5fbb58f8178/resource.txt

gwarser on 21 Jul 2018

/logo.png$script,domain=idnes.cz
idnes.cz##+js(nowrite.js, https://github.com/uBlockOrigin/uAssets/issues/462#issuecomment-395998551 but not working atm.