UnifiedID is a way of establishing 'universal' user device IDs for web browsers instead of having dozens of exchanges sync IDs with hundreds of demand sources. The basic concept is that the UnifiedID organization stores a set of global IDs at several locations and each publisher using Prebid.js should be able to take advantage of UnifiedID to get better bids from supporting DSPs.
Note that UnifiedID is not a single ID, but potentially a set of IDs. At first just from The TradeDesk, but someday perhaps others.
There are several main steps in the process:
Note that universal user IDs aren't needed in the mobile app world because device ID is available in those ad serving scenarios.
Prebid.org intends to support integration of Unified ID and other 'universal' IDs as a core feature in header bidding products with appropriate publisher-level controls. We will also deprecate the "pubCommonId", folding it's functionality into this more generic module. (The existing module will be left for a few months as there will be page changes to make.)
Prebid support of IDs should include:
gdprConsent settings for the userThe role of Prebid.js is to:
The role of Prebid Server is to:
Then it's up to each adapter to read the ID values from a standard location and forward it through their pipeline.
Examples of the proposed publisher configuration for various scenarios follows:
1) Publisher supports Unified ID and first party domain cookie storage
pbjs.setConfig({
usersync: {
userIds: [{
name: "unifiedId",
params: {
partner: "PARTNER_CODE",
url: "URL_TO_UNIFIED_ID_SERVER"
},
storage: {
type: "cookie",
name: "pbjs-unifiedid", // create a cookie with this name
expires: 60 // cookie can last for 60 days
}
}],
syncDelay: 5000 // 5 seconds after the first bidRequest()
}
});
2) Publisher supports UnifiedID with HTML5 local storage, synchronously with the first PBJS
pbjs.setConfig({
usersync: {
userIds: [{
name: "unifiedId",
params: {
partner: "PARTNER_CODE",
url: "URL_TO_UNIFIED_ID_SERVER"
},
storage: {
type: "html5",
name: "pbjs-unifiedid" // set localstorage with this name
},
maxDelayToAuction: 500 // implies syncDelay of 0
// wait up to 500ms before starting auction
}]
}
});
3) Publisher has integrated with unifiedID on their own and wants to pass the unifiedID directly through to Prebid.js
pbjs.setConfig({
usersync: {
userIds: [{
name: "unifiedId",
value: {"tdid": "D6885E90-2A7A-4E0F-87CB-7734ED1B99A3",
"appnexus_id": "1234"}
}]
}
});
4) Publisher supports PubCommonID and first party domain cookie storage
pbjs.setConfig({
usersync: {
userIds: [{
name: "pubCommonId",
storage: {
type: "cookie",
name: "_pubCommonId", // create a cookie with this name
expires: 1825 // expires in 5 years
}
}]
}
});
5) Publisher supports both unifiedID and PubCommonID and first party domain cookie storage
pbjs.setConfig({
usersync: {
userIds: [{
name: "unifiedId",
params: {
partner: "PARTNER_CODE",
url: "URL_TO_UNIFIED_ID_SERVER"
},
storage: {
type: "cookie",
name: "pbjs-unifiedid" // create a cookie with this name
}
},{
name: "pubCommonId",
storage: {
type: "cookie",
name: "pbjs-pubCommonId" // create a cookie with this name
}
}],
syncDelay: 5000 // 5 seconds after the first bidRequest()
}
});
6) DigiTrust example
pbjs.setConfig({
usersync: {
userIds: [{
name: "digitrust",
params: {
memberId: "123abc"
},
storage: {
type: "cookie",
name: "pbjs-digitrust" // create a cookie with this name
}
}],
syncDelay: 5000 // 5 seconds after the first bidRequest()
}
});
Prebid.js will use the setConfig values configured by the publisher to look for locally stored ID values. If it doesn't find local IDs, it will reach out to the configured URLs, storing any results for the specified number of days as appropriate after checking for GDPR consent.
The prebidServerBidAdapter adds the values to the user.ext.eids section of the OpenRtb2 protocol:
{
"user": {
"ext": {
"eids": [{
"source": "adserver.org",
"uids": [{
"id": "111111111111",
"ext": {
"rtiPartner": "TDID"
}
}]
},
{
"source": "pubcommon",
"uids": [{
"id":"11111111"
}]
}
],
"digitrust": {
"id": "11111111111",
"keyv": 4
}
}
}
}```
Suggested Pseudo-Code
1. The User ID Module (UIM) looks for its config. If no config, exit. Nothing to do.
1. UIM should check the status of the GDPR and exit if the user doesn't consent to local storage of data. Specifically, if the PBJS consentManagement module is present, then the code needs to assume the user does not consent unless it sees a consent string that specifically allows Purpose 1 (store local state). If consent is required but not found, the module should just exit after logging a debug warning for when pbjs_debug=true.
1. For each userId specified in the config, check the appropriate local storage to see if we already have the ID JSON
1. If we don't have local storage for any of the specified userIds, set up a timer to initiate the call the appropriate sub-module 'getId function'. This function has access to the gdprConsent info if available. The responses should cause the UIM to store the JSON return data into the appropriate local storage.
1. If we do have local storage for a sub-module, call its 'decoding function' and build up the object that will be sent to the adapters. e.g. bidRequest.userIds={"tdid": "D6885E90-2A7A-4E0F-87CB-7734ED1B99A3", "appnexus_id": "1234", "pubcid": "c4a4c843-2368-4b5e-b3b1-6ee4702b9ad6", digitrustId: {"id": "skloi903409as9kf8cj", "keyv": 4}}
1. As an example implementation, modify the Rubicon PBJS adapter to read 'bidRequest.userIds.tdid' and pass it through to the exchange.
Each of the "sub-modules" in the User ID Module has these interfaces:
- A get-ID function. This ID-specific function will be called when it's time to retrieve the ID from that service. It will be passed the `params` object from config. The response is assumed to return JSON that will be directly stored by the module in the appropriate local storage with the appropriate expiration date. Note that expiration may be specified in either the `expires` field of the publisher config, OR overridden by the `expires` field on the JSON response from the ID service.
- A JSON decoding function. This ID-specific function will be called when the module is compiling the object that will be passed to all of the adapters. It takes the full data stored in local state for the sub-module and returns the string or object that should be added to the adapter-visible ID object. In most cases,
<a name="spec"></a>
## Spec
### Implementation
module
test if local storage/cookies are enabled
* if neither is enabled, exit
* else add enabled types to enabledStorageTypes
test if any user ids are set in configuration
* if none exist, exit
iterate sub-modules, for each submodule
1. check if configuration exists with matching sub-module config name:
* skip sub-module if none exists
2. validate sub-module config storage props and params
* syncDelay (optional)
* storage (required)
* type (required)
* name (required)
* expires (required)
* params (optional)
* partnerCode
* url
3. if syncDelay exists, use setTimeout with callback wrapping next function, else call immediately
4. use storage key and type to retrieve stored value
* if stored value exists, add value to data array for adding to bid request
* else, call sub-module getId method to retrieve
/**
/**
/**
/**
/**
/**
/**
/**
// if data exists, append to bid objects, which will be incorporated into bid requests
if (Object.keys(data).length) {
const adUnits = config.adUnits || $$PREBID_GLOBAL$$.adUnits
if (adUnits) {
adUnits.forEach((adUnit) => {
adUnit.bids.forEach((bid) => {
Object.assign(bid, data)
})
})
}
}
}
return next.apply(this, arguments)
}
/**
/**
/**
/**
/**
/**
/**
// get expires value from response first if possible, next use a config expires value if defined, lastly use the default
const expires = response.expires || data.storage.expires || submodule.expires
// save response id data to local storage
const savedId = saveLocalStorageValue(response, data.storage.name, data.storage.type, expires)
if (savedId) {
const idData = getLocalStorageValue(data.storage.name)
if (idData) {
// decode is called and it's result added to the array to be added to bid requests
addIdDataToBids(idData, submodule)
// add hook if not added previously
if (extendedBidRequestData.length === 0) {
$$PREBID_GLOBAL$$.requestBids.addHook(requestBidHook)
}
} else {
// log error, invalid id data returned after saving to local storage, skip this id submodule
}
} else {
// log error saving to local storage, skip this id submodule
}
} else {
// log error web request for id failed
}
}
/**
// check if any user id types are set in configuration (must opt-in to enable)
if (!Array.isArray(config.get('usersync.userIds'))) {
// exit if no configurations are set
return
}
[pubCommonId, unifiedId].forEach(function(submodule) {
// try to get config for id submodule
const submoduleConfig = config.get('usersync.userIds').find(userIdConfig => userIdConfig.name === submodule.configKey)
if (!submoduleConfig) {
// log error, config not found for submodule, skip this id submodule
return
}
// get storage key from config if set
const storageName = submoduleConfig.storage.name
if (!storageName) {
// log error, key skip this id submodule
return
}
// get storage type from config if set
const storageType = submoduleConfig.storage.type
if (!storageType) {
// log error, key skip this id submodule
return
}
const url = submoduleConfig.params.url
if (!url) {
// possibly log if url not found in config (though maybe not since pubCommonId does not need this functionality)
}
const syncDelay = submoduleConfig.syncDelay || 0
if (!syncDelay) {
// log sync delay not found in config
}
const storedId = getLocalStorageValue(storageName, submoduleConfig.storage.type)
// If ID from local storage/cookies is valid, make data available to bid requests
if (storedId) {
addIdDataToBids(storedId, submodule)
} else {
// Else, ID does not exist in local storage, call the submodule getId to get a value to save to local storage
// Note: syncDelay implemented here by wrapping with a timer
if (syncDelay) {
setTimeout(function() {
submodule.getId(data, function(response) {
// call webserviceComplete passing storageName, submodule, expires, and the response from the callback
getIdComplete(response, data, submodule)
})
}, syncDelay)
} else {
submodule.getId(data, function(response) {
// call webserviceComplete passing config data, and the response from the callback
getIdComplete(response, data, submodule)
})
}
}
})
// add hook only if data is going to be passed at this point, else add hook in getId complete
if (extendedBidRequestData.length) {
$$PREBID_GLOBAL$$.requestBids.addHook(requestBidHook)
}
}
// call init
initUserId()
```
IdSubmodulePubCommonID Module
IdSubmoduleunifiedID Module
Array.<Object>ID data for appending to bid requests from the requestBidHook
*Decorate ad units with user id properties. This hook function is called before the real pbjs.requestBids is invoked, and can modify its parameter
boolean | *Helper to check if local storage or cookies are enabled Question: Should we add the local storage methods to utils?
booleanHelper to set key va lue to from local storage or cookies as fallback Question: Should we add the local storage methods to utils?
*Helper to set key value in cookies Question: Should we add the cookie methods to utils?
*Helper get value for key from cookies Question: Should we add the cookie methods to utils?
Convert cookie/local storage ID data for bid adapters
ID submodule getId complete handler
init user id module if config values are set correctly
functionfunctionObject | string | numberObjectIdSubmodulePubCommonID Module
IdSubmoduleunifiedID Module
Array.<Object>ID data for appending to bid requests from the requestBidHook
*Decorate ad units with user id properties. This hook function is called before the
real pbjs.requestBids is invoked, and can modify its parameter
Kind: global function
| Param | Type |
| --- | --- |
| config | PrebidConfig |
| next | |
boolean | *Helper to check if local storage or cookies are enabled
Question: Should we add the local storage methods to utils?
booleanHelper to set key va
lue to from local storage or cookies as fallback
Question: Should we add the local storage methods to utils?
Kind: global function
| Param | Type | Description |
| --- | --- | --- |
| data | Object | string | number | |
| name | string | |
| storageType | string | either 'cookie' or 'localStorage' |
| expires | number | |
*Kind: global function
| Param |
| --- |
| name |
| type |
Helper to set key value in cookies
Question: Should we add the cookie methods to utils?
Kind: global function
| Param | Type |
| --- | --- |
| name | string |
| value | string | number |
| expires | number |
*Helper get value for key from cookies
Question: Should we add the cookie methods to utils?
Kind: global function
| Param | Type |
| --- | --- |
| name | string |
Convert cookie/local storage ID data for bid adapters
Kind: global function
| Param | Type |
| --- | --- |
| idData | Object | string | number |
| submodule | IdSubmodule |
ID submodule getId complete handler
Kind: global function
| Param | Type | Description |
| --- | --- | --- |
| response | Object | string | number | |
| data | Object | config data for sub-moduel |
| submodule | IdSubmodule | |
init user id module if config values are set correctly
functionKind: global typedef
| Param | Type | Description |
| --- | --- | --- |
| response | Object | assumed to be a json object |
functionKind: global typedef
Summary: submodule interface for getId function
| Param | Type | Description |
| --- | --- | --- |
| data | Object | |
| callback | webserviceCallback | optional callback to execute on id retrieval |
Object | string | numberKind: global typedef
Summary: submodule interface for decode function
| Param | Type |
| --- | --- |
| idData | Object | string | number |
ObjectKind: global typedef
Properties
| Name | Type |
| --- | --- |
| configKey | string |
| expires | number |
| decode | decode |
| getId | getId |
@bretg What's your thinking about whether the default config is before the bid request or after?
For sync delay for bid request, is there a reason to have separate timer? An option that keeps things simpler would be run the OpenID sync with the rest of the adapter syncs.
Discussed with some members of OpenID consortium and RubiconProject internal. Changed:
delayAuctionStart option as more clearly defined behaviorLooks like a great start - thanks Bret!
delayAuctionStart to something like maxDelayToAuction - indicating that we will trigger the auction ASAP if we get a quick response from openID or until max time is hit. usersync: {
universalIds: [{
name: 'openId',
url: 'URL_TO_OPEN_ID_SERVER',
storage: {
type: 'html5',
name: 'pbjs-openid'
},
maxDelayToAuction: 500
}]
}
Not sure if that's a use case we'd ever need to support though.
This looks similar to the already existing pubCommonId module so we should probably make this generic enough to support multiple implementations.
Right now the pubCommonId uses the requestBids hook to add its extension. Don't know if we want to do something similar here or add a more specific extension point for shared ids.
+1 to Rich's comment. I think we can cover pubCommonId with the proposed structure above.
Edit: We'd probably remove pubCommonId as a separate module and either
1) Put the logic into core generically to support OpenID / pubCommonId / other provider
2) Create a separate module for the same functionality
thanks for the input. Description updated:
@mkendall07 I think it's worth making the mechanism generic enough to support multiple providers given that's far from clear which id providers publishers will want to use, especially with AppNexus leaving the Advertising ID Consortium
@mike-chowla yes Bret updated the proposal to be generic.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Made an update to move the "url" attribute underneath a "params" object and add a "partner" attribute as well.
Looks like the TradeDesk is going to allow us to use their URL as a default for OpenId - will post that once confirming. Once that happens, params.partner attribute would be a way to specify their "partner id" rather than the whole URL.
Looks good. It's great that GDPR for publishers will be taken into account. A few questions from pubCommonId perspective.
PubCommonId doesn't reach out to servers to obtain or sync ID's. Rather, the id is generated locally and stored. I guess the logic flow would be something like this: Universal ID module checks storage for id first. If not found then wait till the sync phase and pass the params object to get ID. This means if the ID was missing or expired, then there is a one-cycle delay. Would it be possible to accommodate submodules that do not require syncs?
Would submodule custom parameters be a part of params section?
More on storage. Should pubCommonId still store its own cookie? If not, then could the storage period be extended beyond 30 days?
Ok - I took a cut at integrating all features of PubCommonID into this proposal.
@mkendall07 - this is now complicated enough to warrant a review. Perhaps we can do that in the next PBJS PMC meeting?
Spoke with DigiTrust - added a couple of minor tweaks to support their involvement in this.
Hello. I've recently come on to supporting the DigiTrust id system via IAB Tech Lab. One of our high priorities is integration with Prebid, so to that end I'll be picking everyone's brains on how to do that correctly.
In reading this thread it isn't clear if a consensus was reached. Is the plan to build on pubCommonId module, or to implement the pseudo code for UniversalId outlined in the spec? Forgive me if some questions appear dense. I'm just starting to dig into this code.
@goosemanjack - we're building this new "universal ID" module that has a a simple requirement for supporting each additional ID system must be implementable using a small amount of code, say ~0.25 KB. The framework does most of the work -- the only code that each ID system provides is:
Isaac has provided much of the general code already in the opening comment - it seems likely that we'll release with OpenID and PubCommon, and when you have DigiTrust ready we'll add it. If you have a webservice all ready to go, then you can pile on to this release.
Great. In most cases the DigiTrust ID will be cryptographically generated on the browser client. Some edge cases with Chrome over non-ssl connections require our web service. We may need to talk more about your file size constraints. Use of DigiTrust IDs is restricted to members and we utilize an iframe to make the ID portable across different member publisher sites without the need for sync calls or server hits. The framework also provides some functionality around GDPR compliance and opt-out that we should insure is otherwise covered in Prebid or integrated from DigiTrust into Prebid.
On a more technical note, it appears prebid is using ES6 syntax for modules. We are currently CommonJS/ES5. It is my understanding that we will need to upgrade to ES6 syntax in order to integrate into your build system. If anyone has insight where that wouldn't be the case, please LMK.
Thanks.
this is LGTM from a spec proposal - I'd have some comments on the implementation but let's do that as a formal review process. @bretg Feel free to open the PR.
They renamed OpenId to UnifiedId. Updated.
Note: added a feature to let the gdprConsent object be available to the getId function. We'll update the unifiedId getId function to use it
@pycnvr noted in https://github.com/prebid/prebid.github.io/pull/1068 that the PubCommonID requires an "optout" cookie flag, which is the per-user ability to avoid setting IDs for sites that don't have a GDPR-compliant CMP. Added this item above:
For sites that don't support GDPR, the module should also support a cookie-based opt-out
like the PubCommonID module does. Specifically, if there exists a first party cookie called
"_pbjs_id_optout" (with any value), then this module becomes inactive for this particular
user just like if there was a CMP without Purpose 1 consent.
Draft documentation at https://staging.prebid.org/dev-docs/modules/universalid.html
@bretg https://staging.prebid.org/dev-docs/modules/universalid.html - 404
when building PB with userID from download page - {"error":"Prebid file not built properly","requestId":"d99fd837-0fa9-41b4-8fa6-98386173c118"}
@bretg: Is there a timeline for release of this module?
Actually, it was released on Weds with PBJS 2.10. Documentation at https://prebid.org/dev-docs/modules/userId.html
Currently only two adapters support Unified ID - would love to see others add support.
Most helpful comment
This looks similar to the already existing pubCommonId module so we should probably make this generic enough to support multiple implementations.
Right now the pubCommonId uses the requestBids hook to add its extension. Don't know if we want to do something similar here or add a more specific extension point for shared ids.