Browser-laptop: YouTube showing ads

[bsclifton]

Did you search for similar issues before submitting this one?
Yes

Describe the issue you encountered:
Reported via support:

I’ve been trialing Brave for the last day or so. I was enjoying no ads on youtube until about the 3rd or 4th clip, then straight up I started getting ads. How do I fix this? The ad blocker is switched on in settings.

I tried some of the video links provided and was unable to reproduce. Maybe related to

Expected behavior:

• Platform (Win7, 8, 10? macOS? Linux distro?):
  Windows 10
• Brave Version:
• Screenshot if needed:
• Any related issues:

[srirambv]

One more from support

[neeklamy]

Silly question, but worth asking somehow, these are _definitely_ adverts getting through?

I ask because I have been using Brave for many months now, since 0.7, I frequent YouTube, and I’ve yet to see an advert slip through. I haven’t seen any “banner” ads pop up over videos, before the video begins or breaking up a longer video.

I would suggest this could be a few “sponsored by” sections that are part of the video (either at the beginning or end) being mistaken for adverts.

Might be an idea to ask anyone experiencing this to grab a screenshot.

[CliqueBait]

I can confirm for my install. I see no banner ads or ad breaks in the middle of a video. However, I have been seeing the occasional ad at the beginning of videos which need to be skipped in the usual way.

Ubuntu 16.04.1 LTS
Brave: 0.12.4
Electron: 1.4.12
libchromiumcontent: 53.0.2785.143
V8: 5.3.332.47
Node.js: 6.5.0
Update Channel: dev

[neeklamy]

@CliqueBait, I’ve checked out the playlist you’re watching (starts here: Does Political Correctness WORK?) I’m up to the 16th video – but nothing!

• Brave 0.12.4
• OS X 10.11.6
• HTML5
• Shields up
  
  
  
  • Show Brave ads
  
  
  • Block 3rd Party Cookies
  
  
  • Block Phishing / Malware
  
  
• Shields down – _adverts have to be showing now? Nope!_
• Using a private tab

Under a guest account in OS X, I managed to get the adverts to show immediately by dropping the shields or allowing ads and tracking. Maybe this is a stuck setting somewhere, something from a previous version? It might help to blow away Brave’s config files…

[bsclifton]

@neeklamy where are you physically? I see that @CliqueBait is in the UK (thanks for the screenshot, BTW)

I wonder if they are using different code based on the region. I know this is typical because many videos are blocked based on region (and also ads are region aware)

[neeklamy]

@bsclifton, I should have said, I’m in the UK too.

When I was toggling the shields, although it showed that the shields were down, the “Ads and Trackers Blocked” count was still going up, but, no ads were showing. Which as I said, makes me think it seemed like the “ads off” setting was stuck.

At some point the adverts started showing with the shields down, and now I can toggle the shields on and off and get the expected behaviour – _not terribly helpful, I know._

[Flo5701]

I have add on Youtube to (I'm in canada), i try with a VPN in sweden and no add

[CliqueBait]

@neeklamy "It might help to blow away Brave's config files..."

Um. That doesn't sound too safe/desirable if I'm going to lose anything I need but I'm willing to inspect the files and roll the dice (with backups) if it looks promising. There's a chance it might work because I've been installing each release over the prior release so there might indeed be a stuck setting somewhere.

Main thing I don't want to do is lose all of my bookmarks. I don't care about other settings. Which files do you want me to inspect/delete?

FYI, I upgraded to Ubuntu 16.10 last Friday 14th Oct.

[reesinkf]

I've always had YouTube ads showing up since I started using Brave (many months ago). I just figured they had not finished the code yet or something (since it's still a beta). It's very very annoying to say the least, and with every update I hoped they started "supporting" YouTube ads...

FYI, I'm in the Netherlands and on Windows 10.

[Dylvn]

I have the same problem for YouTube, the ads at the beginning pop! :)
I'm from France for info !

[tcr]

Using Brave, an ad plays on Youtube for me for nearly every video. Located in the US. Happy to perform any tests to help diagnose the problem.

[bbondy]

Where are you located? Would any of the about:adblock regional lists help?

[tcr]

@bbondy I'm located in Boston, MA. I assume that means none of the regional lists would be useful here?

I wonder if Youtube ads are cosmetic adblock ads (which aren't supported yet?) and I'm just unlucky that I get many ads served to me and others get few. If they're URL-based adblock ads, I should be able to type in custom rules in about:adblock and seem them succeed or fail in the Web Debugger.

[bsclifton]

@bbondy I'm curious- does YouTube use Flash if it's available? Is it possible that these are Flash ads getting through? Do you have Flash installed @tcr?

[tcr]

@bsclifton I have Flash installed; but it's not enabled for YouTube, and the ad element being displayed is a <video> element.

I can dump a HTTP log if it's helpful (and if I can figure out how to do it from the Web Console).

[bbondy]

If we can find the URLs of the ads that are serving I can check if they are in the adblock list.

[tcr]

Looking into it, it seems to be serving from the same servers as YouTube videos. It skips two doubleclick domain requests successfully, then originates a video request from YouTube's servers. It might be the case that adblock is working fine and that the current adblock rules just don't work if they are routed through the normal video servers.

Also, it only serves ads when I am logged in. Logging out or a new session tab works fine. So I believe the block is working fine. Maybe cosmetic (HTML selector) rule support will fix this.

This video reproduces the issue 80% of the time for me: https://www.youtube.com/watch?v=cAo9o7zV24A

[Flo5701]

with the new update i have no add on youtube !

[CliqueBait]

Still an issue for me unfortunately:

Ubuntu 16.10
Brave: 0.12.7
Electron: 1.4.20
libchromiumcontent: 53.0.2785.143
V8: 5.3.332.47
Node.js: 6.5.0
Update Channel: dev

[srirambv]

I had not experienced this till today but I suddenly see the Ad on YouTube. Took almost couple hours of running videos for the ad to show up.

[bsclifton]

Another user reported this issue with https://github.com/brave/browser-laptop/issues/5457

• Windows 10
• Brave 0.12.7

Screen cap of issue:

[KronemeyerJoshua]

Yep. Still an issue in 0.12.8
Windows 10
Please don't make me watch scientology ads. xD

[BrendanEich]

A friendly with ad-tech chops has analyzed what's going on, hoping he'll update here shortly.

Cosmetic filters is #344 but this bug has priority and can be fixed without those, we think.

[pixelass]

I just noticed ads in the search results

[bsclifton]

+1 from support

[bsclifton]

No assignee- pushing to 0.13.1

[lukemulks]

I've been taking a look at this, primarily from autoplay on page landing.

This URL was where I did a majority of review: https://www.youtube.com/watch?v=jremlZvNDuk&feature=youtu.be

What appears to be happening, is that YT has deployed a set "prototype" labeled features for the html5 player, which the ads are a component of (referenced as INNERTUBE).

Within the watch.js player resource, there are now ad blocking functions and messaging that weren't present in the past:

   Lya = function() {
        D0("AD_BLOCKER_MESSAGING_EVENT_TYPE_LEARN_HOW");
        E0();
        g.nn(F0, !1)
    }
    ;
    Mya = function() {
        D0("AD_BLOCKER_MESSAGING_EVENT_TYPE_DISMISSED");
        E0();
        g.nn(F0, !1)
    }
    ;
    D0 = function(a) {
        var b = g.lg();
        b && (a = {
            displayPeriodMs: (0,
            g.Yf)() - Nya,
            adBlockerMessagingEventType: a,
            clientScreenNonce: b
        },
        g.In("adBlockerMessaging", a))
    }
    ;

There are a few additional flags that are specific to ad blocking now appearing in certain playback errors and conversion events. Those are definitely new.

Call Sequence & Resolution
YT attempts to make the expected ad requests. When those fail, YT fails over into a separate sequence of request events.

This call is used for the Google API to get location and ads api data

Request URL:https://apis.google.com/_/scs/abc-static/_/js/k=gapi.gapi.en.FgPLF5SwqIU.O/m=gapi_iframes_style_slide_menu/exm=card,gapi_iframes/rt=j/sv=1/d=1/ed=1/rs=AHpOoo-9R8fkhlRsCMrG4wpDzgf1RI7BzQ/cb=gapi.loaded_1
Request Method:GET
Status Code:200 
Remote Address:127.0.0.1:8888

Response Headers 
age:613172
alt-svc:quic=":443"; ma=2592000; v="35,34"
cache-control:public, max-age=31536000
content-encoding:gzip
content-length:3509
content-type:text/javascript; charset=UTF-8
date:Mon, 12 Dec 2016 06:13:47 GMT
expires:Tue, 12 Dec 2017 06:13:47 GMT
last-modified:Fri, 02 Dec 2016 02:44:02 GMT
server:sffe
status:200
vary:Accept-Encoding
x-content-type-options:nosniff
x-xss-protection:1; mode=block

This call is made to autoplay

Request URL:https://s.ytimg.com/yts/jsbin/www-en_US-vfloDMEq8/watch_autoplayrenderer.js
Request Method:GET
Status Code:200 
Remote Address:127.0.0.1:8888

Response Headers
age:342888
alt-svc:quic=":443"; ma=2592000; v="35,34"
cache-control:public, max-age=691200
content-encoding:gzip
content-length:1852
content-type:text/javascript
date:Thu, 15 Dec 2016 09:18:31 GMT
expires:Fri, 23 Dec 2016 09:18:31 GMT
last-modified:Thu, 15 Dec 2016 08:48:14 GMT
server:sffe
status:200
timing-allow-origin:https://www.youtube.com
vary:Accept-Encoding, Origin
x-content-type-options:nosniff
x-xss-protection:1; mode=block

This YT JS API call is made to query & autoload the AD MODULE

Request URL:https://www.google.com/jsapi?autoload=%7B%22modules%22%3A%5B%7B%22name%22%3A%22ads%22%2C%22version%22%3A%221%22%2C%22callback%22%3A%22(function()%7B%7D)%22%2C%22packages%22%3A%5B%22content%22%5D%7D%5D%7D
Request Method:GET
Status Code:200 
Remote Address:127.0.0.1:8888

Response Headers
alt-svc:quic=":443"; ma=2592000; v="35,34"
cache-control:private, max-age=3600, must-revalidate
content-encoding:gzip
content-length:6147
content-type:text/javascript; charset=utf-8
date:Mon, 19 Dec 2016 08:33:19 GMT
expires:Mon, 19 Dec 2016 08:33:19 GMT
server:GSE
status:200
vary:Accept-Encoding
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-xss-protection:1; mode=block


Query String: 
autoload:{"modules":[{"name":"ads","version":"1","callback":"(function(){})","packages":["content"]}]}

OK - so the above call handles the AD MODULE (we should try blocking ^^^), the actual GPT (doubleclick ads) request is handled atypically when the ad blocking is enabled.

Instead of this being a standard request and response code, the GPT request is made as a 307 Internal Redirect, which responds with a base64 JS Application (embed):

Request URL:http://www.googletagservices.com/tag/js/gpt.js
Request Method:GET
Status Code:307 Internal Redirect

Location:data:application/javascript;base64,KGZ1bmN0aW9uKCkgeyB2YXIgcDsgdmFyIG5vb3BmbiA9IGZ1bmN0aW9uKCkgeyB9OyB2YXIgbm9vcHRoaXNmbiA9IGZ1bmN0aW9uKCkgeyByZXR1cm4gdGhpczsgfTsgdmFyIG5vb3BudWxsZm4gPSBmdW5jdGlvbigpIHsgcmV0dXJuIG51bGw7IH07IHZhciBub29wYXJyYXlmbiA9IGZ1bmN0aW9uKCkgeyByZXR1cm4gW107IH07IHZhciBub29wc3RyZm4gPSBmdW5jdGlvbigpIHsgcmV0dXJuICcnOyB9OyB2YXIgY29tcGFuaW9uQWRzU2VydmljZSA9IHsgYWRkRXZlbnRMaXN0ZW5lcjogbm9vcHRoaXNmbiwgZW5hYmxlU3luY0xvYWRpbmc6IG5vb3Bmbiwgc2V0UmVmcmVzaFVuZmlsbGVkU2xvdHM6IG5vb3BmbiB9OyB2YXIgY29udGVudFNlcnZpY2UgPSB7IGFkZEV2ZW50TGlzdGVuZXI6IG5vb3B0aGlzZm4sIHNldENvbnRlbnQ6IG5vb3BmbiB9OyB2YXIgUGFzc2JhY2tTbG90ID0gZnVuY3Rpb24oKSB7IH07IHAgPSBQYXNzYmFja1Nsb3QucHJvdG90eXBlOyBwLmRpc3BsYXkgPSBub29wZm47IHAuZ2V0ID0gbm9vcG51bGxmbjsgcC5zZXQgPSBub29wdGhpc2ZuOyBwLnNldENsaWNrVXJsID0gbm9vcHRoaXNmbjsgcC5zZXRUYWdGb3JDaGlsZERpcmVjdGVkVHJlYXRtZW50ID0gbm9vcHRoaXNmbjsgcC5zZXRUYXJnZXRpbmcgPSBub29wdGhpc2ZuOyBwLnVwZGF0ZVRhcmdldGluZ0Zyb21NYXAgPSBub29wdGhpc2ZuOyB2YXIgcHViQWRzU2VydmljZSA9IHsgYWRkRXZlbnRMaXN0ZW5lcjogbm9vcHRoaXNmbiwgY2xlYXI6IG5vb3BmbiwgY2xlYXJDYXRlZ29yeUV4Y2x1c2lvbnM6IG5vb3B0aGlzZm4sIGNsZWFyVGFnRm9yQ2hpbGREaXJlY3RlZFRyZWF0bWVudDogbm9vcHRoaXNmbiwgY2xlYXJUYXJnZXRpbmc6IG5vb3B0aGlzZm4sIGNvbGxhcHNlRW1wdHlEaXZzOiBub29wZm4sIGRlZmluZU91dE9mUGFnZVBhc3NiYWNrOiBmdW5jdGlvbigpIHsgcmV0dXJuIG5ldyBQYXNzYmFja1Nsb3QoKTsgfSwgZGVmaW5lUGFzc2JhY2s6IGZ1bmN0aW9uKCkgeyByZXR1cm4gbmV3IFBhc3NiYWNrU2xvdCgpOyB9LCBkaXNhYmxlSW5pdGlhbExvYWQ6IG5vb3BmbiwgZGlzcGxheTogbm9vcGZuLCBlbmFibGVBc3luY1JlbmRlcmluZzogbm9vcGZuLCBlbmFibGVTaW5nbGVSZXF1ZXN0OiBub29wZm4sIGVuYWJsZVN5bmNSZW5kZXJpbmc6IG5vb3BmbiwgZW5hYmxlVmlkZW9BZHM6IG5vb3BmbiwgZ2V0OiBub29wbnVsbGZuLCBnZXRBdHRyaWJ1dGVLZXlzOiBub29wYXJyYXlmbiwgcmVmcmVzaDogbm9vcGZuLCBzZXQ6IG5vb3B0aGlzZm4sIHNldENhdGVnb3J5RXhjbHVzaW9uOiBub29wdGhpc2ZuLCBzZXRDZW50ZXJpbmc6IG5vb3Bmbiwgc2V0Q29va2llT3B0aW9uczogbm9vcHRoaXNmbiwgc2V0TG9jYXRpb246IG5vb3B0aGlzZm4sIHNldFB1Ymxpc2hlclByb3ZpZGVkSWQ6IG5vb3B0aGlzZm4sIHNldFRhZ0ZvckNoaWxkRGlyZWN0ZWRUcmVhdG1lbnQ6IG5vb3B0aGlzZm4sIHNldFRhcmdldGluZzogbm9vcHRoaXNmbiwgc2V0VmlkZW9Db250ZW50OiBub29wdGhpc2ZuLCB1cGRhdGVDb3JyZWxhdG9yOiBub29wZm4gfTsgdmFyIFNpemVNYXBwaW5nQnVpbGRlciA9IGZ1bmN0aW9uKCkgeyB9OyBwID0gU2l6ZU1hcHBpbmdCdWlsZGVyLnByb3RvdHlwZTsgcC5hZGRTaXplID0gbm9vcHRoaXNmbjsgcC5idWlsZCA9IG5vb3BudWxsZm47IHZhciBTbG90ID0gZnVuY3Rpb24oKSB7IH07IHAgPSBTbG90LnByb3RvdHlwZTsgcC5hZGRTZXJ2aWNlID0gbm9vcHRoaXNmbjsgcC5jbGVhckNhdGVnb3J5RXhjbHVzaW9ucyA9IG5vb3B0aGlzZm47IHAuY2xlYXJUYXJnZXRpbmcgPSBub29wdGhpc2ZuOyBwLmRlZmluZVNpemVNYXBwaW5nID0gbm9vcHRoaXNmbjsgcC5nZXQgPSBub29wbnVsbGZuOyBwLmdldEFkVW5pdFBhdGggPSBub29wYXJyYXlmbjsgcC5nZXRBdHRyaWJ1dGVLZXlzID0gbm9vcGFycmF5Zm47IHAuZ2V0Q2F0ZWdvcnlFeGNsdXNpb25zID0gbm9vcGFycmF5Zm47IHAuZ2V0RG9tSWQgPSBub29wc3RyZm47IHAuZ2V0U2xvdEVsZW1lbnRJZCA9IG5vb3BzdHJmbjsgcC5nZXRTbG90SWQgPSBub29wdGhpc2ZuOyBwLmdldFRhcmdldGluZyA9IG5vb3BhcnJheWZuOyBwLmdldFRhcmdldGluZ0tleXMgPSBub29wYXJyYXlmbjsgcC5zZXQgPSBub29wdGhpc2ZuOyBwLnNldENhdGVnb3J5RXhjbHVzaW9uID0gbm9vcHRoaXNmbjsgcC5zZXRDbGlja1VybCA9IG5vb3B0aGlzZm47IHAuc2V0Q29sbGFwc2VFbXB0eURpdiA9IG5vb3B0aGlzZm47IHAuc2V0VGFyZ2V0aW5nID0gbm9vcHRoaXNmbjsgdmFyIGdwdCA9IHdpbmRvdy5nb29nbGV0YWcgfHwge307IHdpbmRvdy5nb29nbGV0YWcuZGVzdHJveVNsb3RzID0gZnVuY3Rpb24gKCkgeyB9OyB2YXIgY21kID0gZ3B0LmNtZCB8fCBbXTsgZ3B0LmFwaVJlYWR5ID0gdHJ1ZTsgZ3B0LmNtZCA9IFtdOyBncHQuY21kLnB1c2ggPSBmdW5jdGlvbihhKSB7IHRyeSB7IGEoKTsgfSBjYXRjaCAoZXgpIHsgfSByZXR1cm4gMTsgfTsgZ3B0LmNvbXBhbmlvbkFkcyA9IGZ1bmN0aW9uKCkgeyByZXR1cm4gY29tcGFuaW9uQWRzU2VydmljZTsgfTsgZ3B0LmNvbnRlbnQgPSBmdW5jdGlvbigpIHsgcmV0dXJuIGNvbnRlbnRTZXJ2aWNlOyB9OyBncHQuZGVmaW5lT3V0T2ZQYWdlU2xvdCA9IGZ1bmN0aW9uKCkgeyByZXR1cm4gbmV3IFNsb3QoKTsgfTsgZ3B0LmRlZmluZVNsb3QgPSBmdW5jdGlvbigpIHsgcmV0dXJuIG5ldyBTbG90KCk7IH07IGdwdC5kaXNhYmxlUHVibGlzaGVyQ29uc29sZSA9IG5vb3BmbjsgZ3B0LmRpc3BsYXkgPSBub29wZm47IGdwdC5lbmFibGVTZXJ2aWNlcyA9IG5vb3BmbjsgZ3B0LmdldFZlcnNpb24gPSBub29wc3RyZm47IGdwdC5wdWJhZHMgPSBmdW5jdGlvbigpIHsgcmV0dXJuIHB1YkFkc1NlcnZpY2U7IH07IGdwdC5wdWJhZHNSZWFkeSA9IHRydWU7IGdwdC5zaXplTWFwcGluZyA9IGZ1bmN0aW9uKCkgeyByZXR1cm4gbmV3IFNpemVNYXBwaW5nQnVpbGRlcigpOyB9OyB3aW5kb3cuZ29vZ2xldGFnID0gZ3B0OyB3aGlsZSAoIGNtZC5sZW5ndGggIT09IDAgKSB7IGdwdC5jbWQucHVzaChjbWQuc2hpZnQoKSk7IH0gfSkoKTs=
Non-Authoritative-Reason:Delegate

When the base64 js application is decoded, it reveals several GPT API functions and services that appear to null, but then also includes a destroySlots service (clearing the deck) which appears to be designed to throw an error on default, and follow with a legit GPT request for the ads:
note: these all pick up after the destroySlots service

var cmd = gpt.cmd || [];
    gpt.apiReady = true;
    gpt.cmd = [];
    gpt.cmd.push = function(a) {
        try {
            a();
        } catch (ex) {}
        return 1;
    }
    ;
    gpt.companionAds = function() {
        return companionAdsService;
    }
    ;
    gpt.content = function() {
        return contentService;
    }
    ;
    gpt.defineOutOfPageSlot = function() {
        return new Slot();
    }
    ;
    gpt.defineSlot = function() {
        return new Slot();
    }
    ;
    gpt.disablePublisherConsole = noopfn;
    gpt.display = noopfn;
    gpt.enableServices = noopfn;
    gpt.getVersion = noopstrfn;
    gpt.pubads = function() {
        return pubAdsService;
    }
    ;
    gpt.pubadsReady = true;
    gpt.sizeMapping = function() {
        return new SizeMappingBuilder();
    }
    ;
    window.googletag = gpt;
    while (cmd.length !== 0) {
        gpt.cmd.push(cmd.shift());
    }
})();

The GPT API sets new slots to replace the destroyed one, readies the command and pushes the function call. This appears to be YTs answer to ad blocking, by probing for failure and falling back to a last of line embed method that initiates a call from the embedded js application, which shows up as "no domain" (avoids domain-level blocklists)

For Blocking, I'd suggest trying these two things, if possible:

1. Block the ad module api call:

Request URL:https://www.google.com/jsapi?autoload=%7B%22modules%22%3A%5B%7B%22name%22%3A%22ads%22%2C%22version%22%3A%221%22%2C%22callback%22%3A%22(function()%7B%7D)%22%2C%22packages%22%3A%5B%22content%22%5D%7D%5D%7D
Request Method:GET
Status Code:200 
Remote Address:127.0.0.1:8888

1. I'm unsure if it is possible to detect/block this, but if it's possible to detect the gpt.js 307 internal redirect request and block it before it can embed the response, that would be the other half of the blocking equation:

Request URL:http://www.googletagservices.com/tag/js/gpt.js
Request Method:GET
Status Code:307 Internal Redirect

I would think that the googletagservices call would be blocked, but I believe it's being called in an atypical way that may be evading domain-level blocking. It appears that the JS API request and the watch.js YT player config are designed to work together to form and send the ad request, and uses the autoplay call as a trigger.

Apologies for the length here, but this one required a bit of unpacking to get to the bottom of. Definitely let me know if there are any questions, or anyone needs a set of eyes on reviewing an update.

[bbondy]

We're currently using a stub for googletagservices.com/tag/js/gtp.js
https://github.com/brave/browser-laptop/blob/master/js/data/siteHacks.js#L96

You could try modifying that to cancel: true instead to see if it helps in this case.
I think canceling globally across sites would probably cause some webcompat problems, but we could do it only here if that helps.

[lukemulks]

Thanks!

Oh, I see what you mean re: the stub (saw the post on ublock about it,
didn't connect that it was being used here too).

Will test with the call cancelled and report back.

On Dec 21, 2016 6:42 AM, "Brian R. Bondy" [e-mail] wrote:

We're currently using a stub for googletagservices.com/tag/js/gtp.js
https://github.com/brave/browser-laptop/blob/master/js/
data/siteHacks.js#L96

You could try modifying that to cancel: true instead to see if it helps in
this case.
I think canceling globally across sites would probably cause some
webcompat problems, but we could do it only here if that helps.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/brave/browser-laptop/issues/4693#issuecomment-268538969,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIkDKBiPI9KEj88zw_VTozhjgXV4PyCjks5rKTrrgaJpZM4KUPn2
.

[bbondy]

You might have to comment out the sitehack and then just use the custom adblock rules in about:adblock. Thanks!

[DivineOmega]

I can confirm this issue is still present in Brave 0.13.0 rev 9c55eca, running on Ubuntu 16.04.

[joekaiser]

I'm just adding another report of it.

Brave 0.12.15
Muon 1.4.31
libchromiumcontent 53.0.2785.143
V8 5.3.332.47
Node.js 6.5.0
Update Channel dev
os.platform win32
os.release 10.0.14393
os.arch x64

[luixxiul]

@lukemulks do you think the issue is caused by the issue concerning QUIC?

[lukemulks]

Yeah, @suguru the update in the 0.13 RC resolves in testing/QA as long as
cache and cookies are cleared in the browser. 3 testers yesterday observed
the issue as corrected in the RC, so we should be able to close this once
RC pushes to prod.

On Jan 26, 2017 8:09 AM, "Suguru Hirahara" [e-mail] wrote:

@lukemulks https://github.com/lukemulks do you think the issue is caused
by the issue concerning QUIC?

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/brave/browser-laptop/issues/4693#issuecomment-275429398,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIkDKOltAtOmsVqumhk8KO_InxZzW4j2ks5rWMUlgaJpZM4KUPn2
.

[luixxiul]

OK then let's close this thanks to #6831

[luixxiul]

I labelled QA/no-qa-needed but let's keep an eye on the issue on youtube and reopen if the problem would still persist.

[echosa]

I updated to 0.13.0 earlier today. I'm still seeing YouTube ads. Shields are up. I cleared all my cache/cookies/history, based on previous comments in this issue. Still seeing ads.

[lukemulks]

@echosa can you please provide us with a little more info on the ads you're seeing, and a screenshot (if possible)?

• Are you observing display (static) ads, or is this a video ad you're observing?

• Which OS are you observing this from?

• If possible, can you provide a rough geo-region (doesn't have to be extremely specific, you can even let us know if within the US or outside of the US, but would help).

• Can you also let us know if you were authenticated w/Google when you observed the ads, or if you viewed them w/o being signed in?

Thanks in advance.

[echosa]

It was this video: https://www.youtube.com/watch?v=gneBUA39mnI

I didn't get a screenshot, and viewing the video a second time doesn't show the ad. (That's normal.) When it happens again, I'll try to get a screenshot.

It was a video ad before the actual video. I was able to skip it after 5 seconds, as usual.

OS X 10.11.6

I'm in Texas, so inside the US.

[lukemulks]

Thanks for the info @echosa! Really appreciate it.

I've just attempted to repro across 20 clips, and am not seeing any ads so far.

Will try from the URL provided and see if we're able to repro from our end. I'm checking from Windows, so I will see if we can get some OS testers to attempt to repro as well.

Do you have Flash enabled in your browser?

[echosa]

I do have Flash enabled. However, I just tried a few videos myself, using links from other comments in this issue as well as some random videos I could fine, and it looks like the pre-roll ads are being blocked. It looked like, on a couple of videos, I saw the pre-roll ad flash for just a moment before the video started, so it seems like ads are being actively suppressed. Maybe just that one I saw got through somehow. I'll keep a look out for ads, and if I see any, I'll respond back here. I'm happy to answer any other questions you might have.

[lukemulks]

@echosa Thank you very much for all of the feedback and info.

I've probably gone through ~40 videos by now and have not been able to repro, but do not have flash enabled in Brave.

I'm going to see if I can get a tester in OSX w/flash enabled to repro.

It sounds like based on your description these ads are being actively suppressed over time, which is a good sign.

[echosa]

@lukemulks Ok, I just had an ad on YouTube again. Shields up. I got a screenshot that shows the URL, the ad, and the fact that shields are, indeed, up.

[lukemulks]

Thank you @echosa - we had some additional team members on OS X attempt to reproduce on Friday, and were unable to. I am going to dig in further and attempt to reproduce (and inspect the player config in case for some reason there may have been a change post-update).

Aside from your valuable feedback, we have only had one additional case where someone reported seeing a YT ad since the update pushed, which appears to have been resolved from the cache clearing.

Looks like these are the specific playback conditions, but please let me know if I am missing anything:

• Mac OSX
• Flash enabled in Brave (v0.13.0)
• Logged into Google/YouTube
• Autoplay is disabled
• Shields Up: Block Ads, Block Cookies

Two things I am wondering if you could provide if possible @echosa

1. Can you please screenshot the shield settings exactly how you have them enabled during the ad playback

1. Can you also please screenshot the About Brave version panel?

Our team has run through over 80 video clips across Windows, MacOS, Linux,, so I want to make sure we capture the exact conditions (as close as possible) to reproduce so we can find any potential outlying conditions.

Thanks again for your persistence and attention on this.

[echosa]

FYI: I just went back to the video to get a screenshot of the shield settings, and I got the same ad again, so it's happened twice.

All of your bullet points look good. I'd add one more in, just in case it matters. I'm watching the video in Theater Mode, as you can see. Also, keep in mind I'm on OS X 10.11.6, not macOS 10.12.x.

Here are the screenshots you asked for:

[lukemulks]

@echosa thanks for the screenshots and addt'l info. Will work to repro to these same conditions and follow up.

[reesinkf]

I still get ads aswell. I've put the images on imgur: http://imgur.com/a/JFZ8U

I didn't have flash enabled though. I just enabled it after reading the last few posts, and I just got another ad so I guess this doesn't change anything for me.

[lukemulks]

@reesinkf thanks for reporting.

Can you confirm if you'd cleared cache and cookies after installing v0.13.0.

Thanks!

[JamieHitGub]

Just to add, I'm having the same problem on Sierra. I am based in London, Uk. Tried VPN'ing around the world and I have the problem regardless of geo. Tried clearing cache/cookies after the update but it's the same.

[lukemulks]

Thanks @JamieHitGub

Can you confirm if you have Widevine & Flash enabled?

@reesinkf can you also confirm if you have cleared cookies and cache after updating, v0.13.0 (and are seeing ads after clearing and relaunch)

[reesinkf]

@lukemulks yep, still getting ads, just got 2 in a row.

[lukemulks]

Thanks for the confirmation @reesinkf
Going to see if we can reproduce for each set of conditions reported here.

[JamieHitGub]

they are both off for me

[JamieHitGub]

Tried clearing again with the following and then re-opened browser and now I can't make an ad appear, maybe before I didn't properly clear cookies or something, it was pretty late here. These are the options that seemed to fix it for me on version 0.13.

[JamieHitGub]

Ok this is interesting, logged into youtube so I could sub to channels/check my watchlists. Now adverts are back. If I log out they dissapear.

Edit: I can re-create this constantly, when logged out - no adverts, when logged in, adverts every time. I recorded my screen as an example if needed. The video I randomly clicked on and used for this test is this one. https://www.youtube.com/watch?v=pd7uo2BVjOA

Hope this helps.

[lukemulks]

thanks @JamieHitGub the info was helpful.

@reesinkf @echosa @JamieHitGub we just pushed v0.13.1. Can you try updating to v0.13.1, clearing cache/cookies and attempt to reproduce?

If that doesn't work, can you try backing up your bookmarks and brave wallets (if payments are enabled in your browser), uninstall and attempt a fresh Brave install?

[echosa]

First video I tried played without ad. The second video played an ad first. The only difference I can tell: I was logged in to Google/YouTube for the second. I updated and cleared everything here, restarting the browser when prompted:

I have not tried completely uninstalling/reinstalling the browser. I'll do that when I get a chance.

Additional screenshots:

[echosa]

Ok, I just completely* (in theory... see below) uninstalled Brave and reinstalled it from a fresh download from the website. I used the same video as shown in my previous comment. I first viewed it logged out. No ad. I then logged into Google (required setting up LastPass), and opened the same video again. I got an ad. Definitely seems related to being logged into Google. I didn't do anything else. No settings changes (everything is back to default: colored tabs, hovering over tabs previews it, etc. all stuff I'd previously turned off), all my stats show on newtab were back to zero (sad day), etc.

• When I say "completely (in theory)", one thing concerns me. When I went to set up LastPass, it remembered my email address. I had told it to remember email, so normally that's expected, but having "completely" uninstalled, I would have expected that information to not be there. :-/

Also, considering I'm on OS X, I should explain the uninstall process. I used an app called AppCleaner (version 3.4, the latest, available here: http://freemacsoft.net/appcleaner/) which lets you pick an app, and it detects matching/related files in your preferences, temp directories, etc. and of course the .app itself. I deleted every matching file and folder it found, then emptied the trash, just to be sure.

[lukemulks]

This is incredibly helpful information @echosa.

Also, apologies about the stats-loss, but know that the sacrifice has helped us narrow in on this (I had to do the same thing last week too, was a bummer)

I suspect that the issue is directly tied with authentication, given that Google uses YouTube for authentication for both Doubleclick (Google's primary ad server) and YouTube.

Going to work with our team on reproducing against the specific conditions you just mentioned, and will follow up. Thanks again, really appreciate your help with this.

[srirambv]

I see Ads on YouTube after I login to YouTube. This is on the first video which was played after logging into YouTube

[JamieHitGub]

I cleared all cookies/history etc then removed app via the 'drag to bin' method on the mac. Manually checked for other files and removed them too. Re-installed from a fresh download. Still adverts only when logged in.

[pixelass]

@srirambv
I can reproduce your report. I can also see ads (when signed in)

I always clear data when I close the browser but also did a manual clearing.

[lukemulks]

Thanks all, one more thing I could use some data/tests on here.

While logged into Google, can you go to the two ads settings pages (control
ads in Google, control ads while outside of google, Google search ads) and:

1. Let me know if the settings are on or off for all 3 switches.

2. If on, can you try turning off for all 3, signing out, signing in and
   attempt to reproduce?

https://www.google.com/settings/u/0/ads/authenticated?hl=en

I just noticed all of mine are set to off, and didn't consider that this
may impact what we are seeing here since authentication is required for
these settings to work.

Thanks again!

On Feb 1, 2017 9:53 AM, "JamieHitGub" [e-mail] wrote:

I cleared all cookies/history etc then removed app via the 'drag to bin'
method on the mac. Manually checked for other files and removed them too.
Still adverts only when logged in.

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/brave/browser-laptop/issues/4693#issuecomment-276729714,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIkDKCMdgh4KOdG_79-izZ8CygjJrbqRks5rYMaSgaJpZM4KUPn2
.

[pixelass]

@lukemulks I have no idea which "two ads setting pages" you are talking about.

[pixelass]

This one @lukemulks ?

Is it supposed to be ON or OFF ?
Mine are all ON and I see ads when logged in

[srirambv]

@lukemulks Turning off the Signin ads and Signout Ads doesn't show ads now.

[pixelass]

All set to OFF

Now I have ads when not signed in too (bottom left ad box)

[lukemulks]

Apologies, should have been more clear.

That page is correct. Try switching to OFF for that switch.

There is a blue button that says "Control Signed in Ads" which will take you to a second page that includes an additional switch.

We basically want to try switching all to off, then clearing cache and cookies, signing back in and attempt to reproduce.

I have not been able to reproduce at all with these settings off, but have not considered that despite all the blocking measures in place, Google has penetrated authentication this deep to where this could be a potential factor.

[lukemulks]

@pixelass understood re: overlay, but what about actual video ads?

[pixelass]

I am not sure if that box in my last screenshot is considered an ad.

I could not reproduce pre-video ads after the change

[lukemulks]

@pixelass TY for confirming!
I am going to file a separate issue to investigate the overlays, as those might fall under 1st-party, but good to hear regarding the preroll.

[pixelass]

Ad space not removed. I might have seen a ticket for this before, thought I'd mention it here. While the ad on the landingpage is not visible there is still the gap for it. (Reference from below: #344 thx @srirambv )

[lukemulks]

TY @pixelass will include that on the overlay issue that I'm going to file (and will double check existing ones). really appreciate the assistance!

[srirambv]

@pixelass @lukemulks This issue covers the cosmetic adblock https://github.com/brave/browser-laptop/issues/344

[pixelass]

@srirambv yes.. that's the one I meant, thx.

So I just clicked through 100 - 200 videos and reloaded about 20 of them ~10 times.

No video ads visible if the google ad settings (mentioned above) are turned OFF

[lukemulks]

Beautiful news. @pixelass what OS are you running?

[pixelass]

[JamieHitGub]

Still getting ad's with all this turned off, tried clearing browsing data and another re-install but still ad's. I'm using the same video for reference as it seems to proc an add every time, clicking other random videos still produces ad's with regularity, also on the most up to date osx release.

[lukemulks]

One note about the ad settings update within Google (the 3 switches). Google is a little opaque on when the changes to ad setttings take effect. This makes this a little tricky, because the boxes that pop up when you switch from ON to OFF mention that the changes _may not_ take place immediately.

We're going to continue w/attempting to repro in MacOS, but so far for Windows it is looking like this has helped resolve, and from @pixelass feedback, is helping resolve to a degree in MacOS.

[echosa]

@lukemulks I don't understand what "two ads settings pages" or "3 switches" you're talking about. I went to the page that you linked: https://www.google.com/settings/u/0/ads/authenticated?hl=en

There's only one switch, and I turned it off:

I've visited a few video and haven't gotten pre-video ads yet. Seems promising, but I'll keep my eye out for more ads.

Can you tell me what page you're talking about with the three switches? I'd like to turn off as many advertising options as I can.

[lukemulks]

No problem @echosa

I've put together an image to help w/this, as Google has basically made this into a 2-page process (I'm writing a blog post covering these types of things that will be available to read soon that puts some more backstory to this).

• The image above includes both ad settings pages that I'm referring to.

• The switches are outlined in red in the image above. From the screencap that you've showed me, it appears that you've shut one off, which is great.

• If you scroll down further in the page, there is the blue button that I've outlined in yellow. Both pages include this. You'll want to click on that button, and it will take you to the second page.

• The second page has 2 additional switches, one for ads beyond Google, and one for Google Search Ads.

• You'll want to shut both of those switches off as well.

Those are the specific "ads settings" - none of which stop Google from collecting data, but change the data profile that they collect about you.

A couple of other things - for your privacy, which may help with this particular ads issue:

Google Activity Controls: https://myaccount.google.com/activitycontrols

I've been unable to reproduce ads in YT playback with the following settings disabled, which also should help control some of the data that you're knowingly (or more often, unknowingly) letting Google collect (note: they never really stop collecting user data, but that's a separate discussion that I'm writing about in a blog post).

I have the switches for these settings in the OFF position (gray, with the switch in the left position)

• Web & App Activity (paused)
• Location History (paused)
• Device Information (paused)
• Voice & Audio Activity (paused)

There are two Youtube specific settings at the bottom of that page, but I have them both ON (blue) and am not getting any ads. You can try shutting those off if you'd like. I have kept them on since I'm not seeing ads w/them on.

_Just an FYI,_ before coming to work at Brave, I spent 5+ years in ad ops and ad products, a majority of which was spent working with Google on ad-related products.

In June 2016, Google updated their Privacy Policy to merge the user ad data with the user data profiles from their other products, and began to use non-ad URLs for ad-related delivery. This is why the issue at hand has been so difficult to resolve, because we have to take a strong but careful approach at blocking to ensure that we don't break core functionality that now has ad stuff merged in with it. Most people are unaware of this update.

Here's some info about the Google privacy policy update (which btw, went pretty much unreported by the press until Oct 2016).

http://www.slate.com/blogs/future_tense/2016/10/21/google_changed_a_major_privacy_policy_and_no_one_really_noticed.html

https://www.engadget.com/2016/10/21/googles-redefined-privacy-policy-lets-ads-follow-you-everywhere/

These types of changes by Google (+my personal desire for privacy protection) and the general deep-state surveillance complex that the ad industry has become is what led me to leave the ad world and come work at Brave on helping to protect privacy, and work on alternatives.

I know I put a lot in one comment here, but I hope it helps explain why this has been such a pain to get locked down.

[pixelass]

OFF TOPIC (sorry)

Just an FYI, before coming to work at Brave, I spent 5+ years in ad ops and ad products, a majority of which was spent working with Google on ad-related products.

@lukemulks "you monster" (good to have you here)

[lukemulks]

LOL @pixelass no worries (very happy to be here) ;-)

[echosa]

@lukemulks That screenshot will be helpful, but I still don't know how to get to those pages. Can you provide a link?

[cndouglas]

By the way, to generate a new Brave profile for testing but preserve your personal profile, just rename the Brave profile directory (at ~/Library/Application Support/brave) to something else (e.g., brave1). When you are done testing, delete the new directory (called brave) and rename the old one back to brave.

When I went to set up LastPass, it remembered my email address. I had told it to remember email, so normally that's expected, but having "completely" uninstalled, I would have expected that information to not be there.

@echosa: It is possible that LastPass has your email address stored in the macOS Keychain. You can find out by opening the Keychain Access utility.

[lukemulks]

@echosa no problem! Going to include them all here for quick reference:

Ads Settings:
For these two settings, try this URL: https://www.google.com/settings/u/0/ads/anonymous?hl=en

• Ads based on your interests on websites beyond google.com
• Google Search Ads based on your interests

For the following setting, try this URL: https://www.google.com/settings/u/0/ads/authenticated?hl=en

• Ads based on your interests (When you are signed in to Google sites)

Related Privacy Settings:
Google Activity Controls: https://myaccount.google.com/activitycontrols

I've been unable to reproduce ads in YT playback with the switches for these settings in the OFF position (gray, with the switch in the left position):

• Web & App Activity (paused)
• Location History (paused)
• Device Information (paused)
• Voice & Audio Activity (paused)

_Note:_ There are two Youtube specific settings at the bottom of that page, but I have them both ON (blue) and am not getting any ads. You can try shutting those off if you'd like. I have kept them on since I'm not seeing ads w/them on.

Hope this helps!

[echosa]

Neither of those URLs take me to pages that look like your image. :-/ When logged in to Google, both of those first two links take me to the same page. The page they take me to is the last screenshot I posted a couple of comments back; the one with a single toggle, which I turned off.

If I visit the first link in a new session tab (which makes me not logged in), I get this:

I turned them off, but when I went back to the page in another new session tab, they were both on again. I guess I have to actually logout and turn those off, because a new session tab's changes aren't permanent or something? It's a bit confusing.

Either way, as you can see, I'm not seeing any pages as what your screenshot showed. Hopefully the pages I am seeing are equivalent, or at least good enough. I wonder why we see different things, though.

[pixelass]

@echosa @lukemulks what kind of Google accounts do you have?
I have a free business account from the BETA-era of Google domains.
Maybe it is different with default google accounts ([email protected] vs. [email protected])

[echosa]

I don't know how to answer that. I signed up for a Gmail account many, many years ago, when it was still invite-only. The rest, as they say, is history.

[pixelass]

I was actually right.

If you have a business account (use your own domain) you get the pages @lukemulks added as screenshots.

If you have a default gmail account [email protected] you get the screen that @echosa posted

screenshot from my test account

[echosa]

That makes sense for logged in, but what about logged out?

[lukemulks]

Thanks, these pages were accessed using consumer (free) user accounts, but
will try from a business account too.

I apologize for the process being this way, Google is making this very
difficult. I don't understand why Google couldn't just put them all on the
same page, but I suspect the confusion is intentional given the amount of
money they bring in from advertising.

I will try to see if I can reproduce a path with the settings that is a
little easier.

What you could do, as an alternative, is attempt the URLs that I sent over
from Chrome or another browser, which will apply to your Google account
across any browser. Settings should stick (hopefully).

There might be some legacy account exceptions, but I also have had a Gmail
account since the invite-only days.

On Feb 1, 2017 1:20 PM, "Gregor Adams" [e-mail] wrote:

@echosa https://github.com/echosa @lukemulks
https://github.com/lukemulks what kind of Google accounts do you have?
I have a free business account from the BETA-era of Google domains.
Maybe it is different with default google accounts ([email protected] vs.
[email protected])

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/brave/browser-laptop/issues/4693#issuecomment-276785772,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIkDKFHjZTezBpE99BTwt9nnjEu8W3IXks5rYPcdgaJpZM4KUPn2
.

[pixelass]

The link is on the page that business accounts get.

[pixelass]

Maybe it's a feature that has not been ported to either of the two versions. They might not even use the same logic, framework, whatever. Google is big at making a lot of custom stuff that doesn't fit together, it's part of moving along and building new ideas. Just spitballing here though.

EDIT: There are a lot of domain speciffic settings available. You can add and remove different apps to your domain, so the settings page might also take that into account. Just looking at the description of what will or will not be personalized if the switch is toggled.

[echosa]

Well, I turned off what I could find, so hopefully that'll do some good. Sticking to the original plan: ads seem to not be playing, which is great, but I'll keep an eye out and report any that do. Thanks for this Google settings tangent. :+1:

I don't understand why Google couldn't just put them all on the same page

If it is easy to do so, more people will turn off the tracking and ads and such. Google doesn't want that but also wants to be able to say they provide a way to do so. So they provide a complicated way people are likely to avoid or miss. It's a win/win for them. :-P

[pixelass]

@lukemulks that confused me. Do you have a custom domain? Those were given in the invite and beta phase for free too.
My test account (@gmail.com) is about 1 year old and it has is a vanilla gmail account and I get the same screen as @echosa

My own account (@pixelass.com) has the same screen as you posted

[lukemulks]

No custom domain in my case.

At this point, I'd just suggest turning off the settings discussed in
whichever form they are presented, close the browser when completed,
relaunch, clear cache and cookies, sign in and attempt to check YouTube for
ad playback.

That should hopefully result in resolving the ads issue.

Thanks again for the persistence and assistance. :-)

On Feb 1, 2017 1:45 PM, "Gregor Adams" [e-mail] wrote:

@lukemulks https://github.com/lukemulks that confused me. Do you have a
custom domain? Those were given in the invite and beta phase for free too.
My test account is about 1 year old and it has is a vanilla gmail account
and I get the same screen as @echosa https://github.com/echosa

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/brave/browser-laptop/issues/4693#issuecomment-276792710,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIkDKExhSRVwgE36Q1O6vi0YB5GXjEHyks5rYP0LgaJpZM4KUPn2
.

[lukemulks]

With the latest set of settings adjustments (Google ads settings), and the attempts to repro by @srirambv @pixelass, @echosa and me that results in no ads displaying, and the lack of additional reports from users post v0.13.1, I'm going to close out the issue at this time.

If there are repeats of ads displaying for users that are signed in, please let us know and we'll triage those cases into a new issue to specifically investigate further. Thanks again to everyone that helped get us to this point.

[tcr]

For future travelers: following the instructions in this comment by @lukemulks resolved the issue for me in Brave 0.13.0.

[borntohonk]

I've had those google settings off forever @lukemulks

looking in the debugger / developer tools, it's apparent that google forces another video up instead of the video I intend to watch.

note: I have a pi-hole on my network aswell, which has successfully prevented any loading of ads in any browser on any device (including my ipad) until recently, probally february first. So this is a youtube thing. (https://github.com/pi-hole/pi-hole)

A friend links me this:

https://www.youtube.com/watch?v=ye24Qehidkg

and this gets forced before it (as an ad)

https://www.youtube.com/watch?v=Lpbda_NXRL0

"html5-video-player ad-created ad-showing ad-interrupting videoAdUiRedesign ytp-video-ad-learn-more-ui ytp-title-extra-info paused-mode"

I'm guessing this is a testpilot system being rolled out, and that it's being regionally tested.

Might be interesting to note, the videos that get forced as ads, don't have any ads themselves.

mashing F5 on any other video gives ads 100% of the time before loading the intended video I'm clicking (randomly).

but doing the same on the videos designated as ads, it jumps straight to the video. (probally some sort of whitelist to not get an ad ontop of a forced ad?)

here's a recording demonstrating it, brave adblocker enabled, pihole also active on a random video I got to by just clicking random trending videos.

https://a.pomf.cat/vqaxlm.webm

just for referance, there are ads on youtube that my pihole do prevent addition to the ones violating my eyes.

[lukemulks]

Thanks for the in-depth followup @borntohonk

A few questions, I know they're likely redundant, but have to go thru the motions.

1. Can you confirm which version of Brave you're running where this was observed?

2. Can you let me know if you cleared cache/cookies and then restarted the browser prior to this being observed? If not, can you try that and let me know if the issue persists for you?

v0.13.0 and higher w/cache & cookie clearing and the ad settings updated should resolve for the majority of cases.

You're right about this being a YT-specific thing. I suspect that this is likely what you're suspecting as well, a regional rollout test for a player update. I saw the same thing happening last fall prior to the last update.

OK so aside from the questions I asked above, this is why it's such a pain of an issue:

• DNS blocking for YT ads has been tricky, as Youtube (and Google in general) are packing advertising in youtube.com urls, and others that aren't historically ad-related.

• What you're observing from the DOM is how Google and YT serve video ads, except that you've caught one of a few of the fallback player configs that are part of the overall player configuration.
  Google and YT use the IMA SDK, which basically overlays a video player on top of the content player for the ad, which destroys after the ad completes. The player below is paused while the SDK is playing the ad. They've been experimenting more with the sdks lately as well. YT tries for flash first, then ultimately fails over into an embedded adsense viral vid player as a last stop.

For the pi-hole setup, can you let me know if there are any overrides applied on your end that would allow for any protocol exceptions over ports?

Thanks again for the details here, really appreciate it.

[borntohonk]

@lukemulks

1. version 0.13.1

1. clearing cookies (and by that logging out), the ads get cockblocked by my pihole (causing a tiny delay for intended video to load up), otherwise if I log in it returns to same pattern of 100% rate of ads played infront of any video.

note: ads still try to "load" if cookies (logged out) are cleared, but the pihole just sends it to a black-hole.

at this point being logged in at youtube seems to be a disadvantage, but at same time probally find whats coming in near future to "everyone". Considering browser adblocker and my network hardware adblocker have been bypassed/killed by this solution, I'd say that's more likely the case.

the pihole is just a dedicated dns

current setup is my computer (or other devices) have DNS server set as local address 192.168.1.85 (pihole) which in turn communicates with 8.8.8.8 and 8.8.4.4 (google DNS servers)

I query 192.168.1.85 for DNS resolves, if on filter list return 0.0.0.0, if not on filter list query 8.8.8.8/8.8.4.4 then return to device in question.

I have somewhat technical understanding of things, but only on a hobby level.

https://github.com/pi-hole/pi-hole/blob/2f3cf1c1b53514260636f3adea6511a0a86a3c37/adlists.default

logging out it's apparant it still tries to load ads, but nothing shows up visibly because the ads are routed to 0.0.0.0 and return 'nil' instead

only conclusion I can get out of this is to not be signed into youtube to not get ads until this solution finally comes over and annoys the average person who also has adopted an adblock solution.

probally interesting to note that the videos designated as ads have some sort of exception/whitelist to be affected by other ads.

it's obviously paid content / partner solution.

Norwegian ads aren't that bad, most of it seems innocent because of our laws are very strict, but it's still annoying.

[lukemulks]

Thanks @borntohonk

• Yeah what you've noted about being logged in v logged out is similar to what has been reported by other users.

• Got it re: the pi-hole cockblock. The thing is, Google has penetrated the cookie store so deep in order to keep serving ads that until users cleared cache and cookies, the issue appeared to persist post-update.

• If you were able to clear cache/cookies, then apologies for the redundancy, just letting you know because the youtube case in general has been really oddball.

• Reason I asked why there might be any overrides applied are due to the nature of the update that was required in order to block these effectively.

We're still investigating the usage, so just a forewarning there, but we discovered that QUIC over UDP protocol _was being used by default by Google in Chromium for ad requests_ that had been merged into Google non-ad domains (that are now considered fair game for ads by Google's documentation, unfortunately).

For this reason, YT in particular was not blocking ads until we went in and applied some specific blocking to QUIC. We're digging in on this further, as while Google had mentioned QUIC being used in Chromium by default for ~50% of Google's stack, it hadn't been mentioned that ads were involved with this, and until their recent update they would have silo'd the ad domains separately. Once we blocked QUIC this way, the issue was not reproducible on our end in a majority of attempts.

FYI - since you're a bit of a guru on the DNS side, and we can use all the help from good natured people that are DNS-inclined, here's the list of domains, which helps break out why DNS blocking has been such a pain:

• admob.com
• adsensecustomsearchads.com
• adwords.com
• doubleclick.net
• _google.com_
• googleadservices.com
• _googleapis.com_
• googlesyndication.com
• googletagmanager.com
• googletagservices.com
• googletraveladservices.com
• _googleusercontent.com_
• google-analytics.com
• _gstatic.com_
• urchin.com
• _youtube.com_

• _ytimg.com_

• When we tested blocking the highlighted domains above, it blocked core product functionality that wasn't ad related, so we're having to get pretty surgical.

• The v0.13.0 release resolved the majority of issues reported, but there were some residual cookie/cache cases where until the cookie/cache was cleared, users occasionally observed ads.

• The _ads settings_ was another outlying item that we discovered needed to be updated for some users, once they did, the ads were blocked.

All that said, what we're both seeing (which is reassuring) is _Google really going to some extremes to hack ads into the user experience_.

I think that what you reported could also be a prototype test as you mentioned. Google does this in YT with previews, and users aren't aware. I worked closely w/Google for 5+ years and we were often in a spot like this where we'd end up reverse engineering their stuff in the wild and it would only apply to some regions.

Also, thank you again for the info and feedback, we're really lucky to have you as a contributor here. :-)

[borntohonk]

@lukemulks the pi-hole github is probally a mecca for the blocking of dns kind.

I use the default compilation that the list, which the pi-hole installer compiles to a single file, because it breaks minimal amount of stuff so far.

it does prevent some minor things occasionally, mostly related to shopping and using searches where paid contributors get promoted with "ads".

the 98 608 entries in this hosts file are a result of combining all the lists from ( https://github.com/pi-hole/pi-hole/blob/2f3cf1c1b53514260636f3adea6511a0a86a3c37/adlists.default )

I'm just an end-user, so credit obviously would go to the pi-hole github contributors ( https://github.com/pi-hole/pi-hole )

note: this is more than just ads, this is a lot of fradulent/malware/tracker websites too.

hosts.txt

Quick rundown:

admob : 52 entries
adsense : 6 entries (adsensecustomsearchads.com : 1 entry)
adword : 11 entries
doubleclick : 12 551 entries
.google.com : 23 entries
*googleadservices : 0 entries
googleapis : 1 entry
googlesyndication : 12 entries
googletagmanager : 2 entries
googletagservices : 2 entries
googletraveladservices : 0 entries
googleusercontent: 2 entries
google-analytics: 4 entries -> analytics : 267 entries
gstatic 13 entries
*urchin : 5 entries
youtube : 9 entries
ytimg : 0 entries

[echosa]

Is there a reason this closed? I was just about to comment to say that I've been seeing ads again, lately.

[lukemulks]

@echosa opening a fresh issue to link to this one - will link here - seeing some additional items that we're running thru, and am linking to a separate repro issue that we're tracking for the blocklists.

[echosa]

Perfect. I'll keep an eye out for the new one. :-) Thanks!

[lukemulks]

@echosa here is the new one https://github.com/brave/browser-laptop/issues/7432
Breaks out timelines and other related issues. Thanks!

[quesada]

I still see ads. Linux.