Prebid.js: GPT instant load race condition against display() calls

[brondsem]

https://developers.google.com/doubleclick-gpt/reference#googletag.PubAdsService_refresh says:

For proper behavior across all browsers, calling refresh must be preceded by a call to display the ad slot. If the call to display is omitted, refresh may behave unexpectedly.

Using the instant load method, if a page is big enough and the client is slow enough, it is possible for the the GPT instant load setup to call refresh() before the inline display() calls are reached. Then when they are, nothing loads into them since they missed the refresh() call. This makes the default instant load approach not tenable for all situations.

Maybe workarounds could involve calling refresh() for a single ad unit, if display() ran too late. Or maybe calling enableServices() later? Not sure what that does exactly though.

[mkendall07]

Is this happening in practice? That's a long time to wait for the browser to not have executed some inline javascript.

[brondsem]

Yes, in a VM on a fast machine, which is probably representative of many slower machines out there. We worked on making prebid start executing as soon as possible, and gave it a 500ms timeout. On a heavy media site with lots of HTML and lots of external resources, it's definitely possible to burn through 500ms. I was getting it to happen about 5% of the time, roughly.

[Heray]

One potential solution is: Right after the first .display() call, there can be Javascript to check if a prebid.js initiated .refresh() call has been made or not. If yes, it will call .refresh() again.

[brondsem]

I think this is a bug, not just a question. Until a good solution is available, the docs should be updated to show the "old" (but more reliable) setup also. (Or just go back to the "old" setup if its better to keep the docs simple and only have one set of examples instead of two).

[mkendall07]

Hi @brondsem
Do you have any data on monetary performance between the 2 versions of the setup? I'd love to see actual data and let pubs decide themselves if the tradeoff is worth it. It also seems to be site/content specific.

[brondsem]

No, sorry, we don't have any data on revenue. We have used only the traditional approach since the "instant load" approach wasn't working all the time and we wanted to use something that did.

I agree it does depend on the page content and resources, but also the browser/client behavior e.g. slow cpu, slow network, etc.

[BartVB]

What browser were you using when you saw this behaviour?

[brondsem]

IIRC, I was testing both Chrome and IE in a windows VM.

Here is a straightforward fiddle to force reproducing the issue: http://jsfiddle.net/oxg6Luvz/1/ It is the current "instant load" example, with the timeout lowered to 500ms and a slow loading <script src="https://httpbin.org/delay/1"></script> added between prebid setup and the display() ad calls.

[sonemic]

Just FYI, we've had the same problem since implementing prebid.js (just noticed this issue after @brondsem mentioned it elsewhere) and we have actually been logging and reporting these instances back to our servers for over a month. We're currently seeing 5.2% of all pageviews failing to render, which means that no ads are displayed on the page, including non-prebid line items.

We've considered calling refresh() in these cases, which seems to work perfectly, but it's unclear how DFP would report this - we're concerned that it might appear to advertisers and/or to AdX that we're refreshing ads, even though we're not actually doing so. Does anyone know enough about DFP to know if this would be an issue?

Also, we'd be glad to A/B test the instant load vs non-instant, but the prebid.org site doesn't really make it clear what the difference is - it seems like the entire site heavily promotes instant load and there aren't any more clearly-marked examples of non-instant-load on the site.

[mkendall07]

@brondsem, @sonemic

We've been looking into this issue. We'll bring back the examples with the non-instant version so publishers can choose.

We've discovered a new API that might help with this, if you are interested in testing it out. It would essentially rely on a polling method in case that googletag is not ready to go and fire it once it's good.

  function initAdserver() {
    if (pbjs.initAdserverSet) {
      return;
    }
    if(!googletag.pubadsReady) {
      setTimeout(initAdserver, 50); // set polling to whatever is desired. 
      return;
    }
    googletag.cmd.push(function() {
      pbjs.que.push(function() {
        pbjs.setTargetingForGPTAsync();
      });
      googletag.pubads().refresh();
    });
    pbjs.initAdserverSet = true;
  }

[sonemic]

@mkendall07 : We actually already tried that mitigation approach - it doesn't work, and neither does waiting for googletag.apiReady.

The correct solution (I believe) is to wait until display() has been called on all placements, but there is no event in DFP (that I can find) that corresponds to that happening. There is the public "slotRenderEnded" event, but that is triggered after the refresh so it's not useful in this case.

[mkendall07]

@sonemic
Do you have example of it not working? In my tests, I added setTimeouts around the display() calls so it would be invoked later and the ads still loaded correctly.

[sonemic]

@mkendall07 : It is an intermittent problem, so it's impossible to create an example. But in production, this is how we test failure:

   var slotRendered = false;
   googletag.pubads().addEventListener('slotRenderEnded', function(event) {
       // if DFP loaded properly, then this will have been called at least once in the
       // 5 seconds following the refresh() call
       slotRendered = true;
   });

  function initAdserver() {
    if (pbjs.initAdserverSet) {
      return;
    }
    googletag.cmd.push(function() {
      pbjs.que.push(function() {
        pbjs.setTargetingForGPTAsync();
      });
      googletag.pubads().refresh();
      setTimeout(function() {
          if ( !slotRendered && !adsAreBlocked() ) {
             // send data back to server to indicate failed render
             // $.ajax('http://blabla.com/reportDFPfailure')
          }
       }, 5000);

    });
    pbjs.initAdserverSet = true;
  }

We see thousands of these events per day (and don't even see failures where the person was on the page for less than 5 seconds). We can also force this to occasionally happen by reloading a page over and over again, and can confirm that when ads don't load, the error event is properly logged.

Also want to point out that there are specific circumstances that affect the chances of display() being delayed. We significantly reduced the failure rate by eliminating some intensive JS calls that were called on page load, and reduced it further by removing some unneeded JS libraries. You might be able to create a more reliable repo case by placing load on the UI thread.

[mkendall07]

@sonemic
Thank you very much for posting an example. I believe the issue might be that you are not using googletag.pubads().enableSingleRequest(); mode. Is that correct?

[sonemic]

@mkendall07 : we are in fact using Single Request Mode.

[mkendall07]

Strange. Here is my test. I've forced googletag.display to be invoked after everything else (10 seconds after page load). If you open console window you can see that the render event is fired.

https://jsfiddle.net/8srz2zab/

[sonemic]

Yeah, but if I'm not mistaken, calling enableServices() triggers a series of events that leads to display() being called on its own. In this test, you're calling display() a second time for a unit that likely already has been displayed due to the enableServices() call, right?

[edit] Sorry, I meant enableServices(), not refresh(). I believe the race condition is refresh() being called between enableServices() and the display() call of the units.

[mkendall07]

I believe you are right. You can always noop display if you want in that case. Fiddle updated: https://jsfiddle.net/8srz2zab/1/

You shouldn't need it anymore since the page is loaded and subsequent refresh calls should do it automatically like you said.

[sonemic]

We just tried this live, and unfortunately it seems to increase the number of failures significantly. So perhaps I was wrong about display().

[mkendall07]

@sonemic
I just realized, as part of the this change, you should remove the on page timeout (here).

[mkendall07]

Closing as pubs can choice which integration method they prefer now (legacy or instant load).