Prebid.js: Improve ability to test bidder responses

Created on 13 Nov 2019 · 14Comments · Source: prebid/Prebid.js

Type of issue

Feature Request

Description

While debugging on a live inventory, ad-ops people need to request test bids from bidders.
We have a to modify bid responses but it needs changes in the config
http://prebid.org/dev-docs/troubleshooting-tips.html#modify-bid-responses-for-testing
DistrictM has its own way, it looks for dmTest=true in page URL, but bidder specific test-query-param is not a scalable solution.

Proposal

All bidders read a common query param pbjs_testbids=true and if present in page URL then each bidder will send respective test-bids flag to their server.

feature help wanted

Source

pm-harshad-mane

Most helpful comment

@pm-harshad-mane - you're right that the debug feature only works for bidders that return bids. It's structural. Rich suggested a 'mock-bidder' adapter, but we would want that built directly into core. Perhaps a new localstorage-based feature that causes core to skip actual bids altogether and just uses a supplied bidresponse?

e.g.

> pbjs.setConfig({
  debugging: {
    enabled: true,
    bidOverride: [{    // processed before the auction rather than after
      bidder: 'bidderA',
      adUnitCode: 'test-div',
      cpm: 1.5,
      adId: '111111',
      ad: '<html><body><img src="https://files.prebid.org/creatives/prebid300x250.png"></body></html>'
     },{
      bidder: 'bidderB',
      adUnitCode: 'test-div',
      cpm: 1.7,
      adId: '22222',
      ad: '<html><body><img src="https://files.prebid.org/creatives/prebid300x250.png"></body></html>'
    }]
  }
});

[Edit] - we would want to avoid firing analytics adapters in this debug scenario.

bretg on 11 Dec 2019

👍2

All 14 comments

Rather than each bidder reading the same param, this param can be made available through config.getConfig('testbids') // true/false

pm-harshad-mane on 13 Nov 2019

we can also support pbjs.setConfig({ testbids: true });

pm-harshad-mane on 13 Nov 2019

@bretg @mkendall07 Any thoughts on this new feature request

jaiminpanchal27 on 18 Nov 2019

Hello @bretg, @mkendall07
Could you please share your thoughts on this new feature request?

pm-harshad-mane on 19 Nov 2019

What is the bidder expectation here? Just avoid charging for the bid? Extra debug info? Fake creative? Really whatever the expectation, it's pretty impractical -- that would be a very long term goal.

You need to identify the actual use cases... what does the pub want to test:

header bidding ads high enough to win in the ad server?
ad server line items?
PBJS config like bidAdjustments?
a specific type of creative?

I don't think trying to herd 200+ bidder-cats into providing fake bid+creatives is going to happen anytime soon. And the requirements are too vague -- does the pub want a bid of a specific value to trigger a particular line item? Do they want to see bidderA win for once?

I think a much better approach would be to deepen the feature we already have documented at http://prebid.org/troubleshooting/troubleshooting-guide.html#modify-bid-responses-for-testing

One could imagine a Chrome extension that would help make it easier for AdOps people to identify an adunit and force things to happen... all within PBJS.

bretg on 20 Nov 2019

The expectation from the bidder is to serve a fake creative.
This can help test the end-to-end setup, not all bidders respond with bids always.
I agree that it will be difficult to have all bidders support it.
The existing approach is useful only in a test environment where one has access to the PBJS config to modify bid responses.

pm-harshad-mane on 20 Nov 2019

The existing approach is useful only in a test environment

Not true - the approach uses the javascript console to set local storage and modify future bid responses. It's pretty cool. Try it! However, it does have the shortcoming there has to be a bidresponse to modify.

What we need is a way to inject bids that never happened. @snapwich - do you think we could extend debugging.js to have some no-bid behavior that when debugging is turned on?

1) Always inject an event handler for noBid
2) If the bidRequest that resulted in noBid matches the bidderCode+AdUnit defined
3) Create and inject a bidResponse from the bid value

Then, from the console, someone could define a creative. e.g.

> pbjs.setConfig({
  debugging: {
    enabled: true,
    bids: [{
      bidder: 'bidderA',
      adUnitCode: 'test-div',
      cpm: 1.5,
      adId: '111111',
      ad: '<html><body><img src="https://files.prebid.org/creatives/prebid300x250.png"></body></html>'
    }]
  }
});

bretg on 25 Nov 2019

I kind of like the idea of having an actual test bid returned from the bidder; that makes it a true end-to-end test. While some bidders do accept returning test bids as a flag on bid.params, I do see an issue with it being global config as I would say the majority of bidders do not support test bids... which means, does it make sense to just leave that configuration as bidder specific on bid.params? I do see some value in making it global as a "build it and they will come" type approach, where maybe if the config exists more bidders might start implementing test bid support.

As for returning mocked bids, I almost feel that'd be better served by a mock bidder adapter. In-fact, I think we had one internally at rubicon... but maybe that's something worth adding to open source? The issue I have with stubbing bidders through the debugging module is while you can say what the bidder is returning, the bidder's implementation might return nothing like that (you hope they would, but you never know), so I think it could give you a false sense of security; getting an actual test bid back from the bidder seems a lot more valuable for an end-to-end test.

snapwich on 27 Nov 2019

I think the use case here is mainly to confirm the publisher in-page code and the line item setup.

A mock bidder adapter would be fine if there were a runtime way to force/override aliases from the javascript console.

e.g.

1) page adunits are setup with bidderA, bidderB, bidderC
2) the goal is to test the actual page code and line item setup
3) tester runs a command to alias bidderA-->mockBidder, bidderB-->mockBidder, bidderC-->mockBidder
4) tester can then use the existing debugging feature to change bid values

bretg on 3 Dec 2019

Hello @bretg , @snapwich

Sorry for the late response, I was on a holiday trip.

I agree that the debugging related code can be added through the console and it doesn't need page access. Thank you for the sample code.

But I think the bid-modification works only if the bidder responds with a bid as the bid-response-hook is used. https://github.com/prebid/Prebid.js/blob/master/src/debugging.js#L37 Let me know if I have missed something.

Now I think having a common flag will be confusing as all active bidders will submit a tets-bid and it will be difficult to test a bidder specific setup (like line-item).

What do you think about making changes in the existing debugging module to add test-bids (even if a bidder does not respond) through the existing approach? we might need to use bidder-done-hook OR we can add a functionality to modify bidder request params so that test param can be added for required bidder at run time ( in this approach support from each bidder will be required).

pm-harshad-mane on 3 Dec 2019

e.g.

> pbjs.setConfig({
  debugging: {
    enabled: true,
    bidOverride: [{    // processed before the auction rather than after
      bidder: 'bidderA',
      adUnitCode: 'test-div',
      cpm: 1.5,
      adId: '111111',
      ad: '<html><body><img src="https://files.prebid.org/creatives/prebid300x250.png"></body></html>'
     },{
      bidder: 'bidderB',
      adUnitCode: 'test-div',
      cpm: 1.7,
      adId: '22222',
      ad: '<html><body><img src="https://files.prebid.org/creatives/prebid300x250.png"></body></html>'
    }]
  }
});

[Edit] - we would want to avoid firing analytics adapters in this debug scenario.

bretg on 11 Dec 2019

👍2

@bretg it is also a good idea to supply isDebugBid property in bid. So analytic adapters may skip/adjust reporting

GeneGenie on 9 Jan 2020

Hi @all,

What's the status of this feature? will be implement soon?

I'm interested too in have a way to test prebid really.
Will be interesting have a tool or an agile way to request test bids from all my bidders, cause when you refresh the website prebid start to return no bids all time.
And sometimes when I develop an "advance" feature which involve googleadmanager + prebid the only way to know if Prebid is working correctly is show statistics in the next days.

Alejandro-MartinG on 22 Jan 2020

A mock bidder adapter would be a great option to have. The existing debug config to change the bid response is of limited use because it requires a valid bid response in the first place which often won't be available in a testing environment.