Hosts: whitelist

[patrickdrd]

the following are wrongly in hosts file and should be whitelisted in my opinion:

a.kickass.to
adf.ly
adfoc.us
cdn.onesignal.com
cdn.optimizely.com
cdn3.optimizely.com
cfvod.kaltura.com
com.com
cookie.monster.com
d2c8v52ll5s99u.cloudfront.net
device-metrics-us-2.amazon.com
device-metrics-us.amazon.com
fls-na.amazon.com
g.msn.com
googletagservices.com
h-sdk.online-metrix.net
jwpltx.com
mobile.pipe.aria.microsoft.com
mywot.com
nexusrules.officeapps.live.com
oas.monster.com
om.cbsi.com
piwik.org
pixel.quantserve.com
s.shopify.com
s.youtube.com
s.zkcdn.net
s2.youtube.com
serials.ws
settings-win.data.microsoft.com
softonic.com
traffic.spot.im
unk.vver.kiae.rr
v10.events.data.microsoft.com
v10.vortex-win.data.microsoft.com
viglink.com
www.googletagmanager.com
www.googletagservices.com
www.hotspotshield.com
www.mywot.com
www.serials.ws

[StevenBlack]

Hi Patrick @patrickdrd. We're an aggregator. We find high quality, well-curated hosts lists, merge them, dedupe the resultant list, and try to keep the list frequently updated and reasonably sized.

We also adjudicate and petition, to some degree, with the curators, who may or may not agree with us.

Please take this up with individual curators as I can't see the compelling reasons you may have, so I'm not inclined to petition for any of these.

[dnmTX]

@patrickdrd reading trough your list....led me to this:

(HINT Allow GIF Animation in your browser to see the content)

[anudeepND]

@patrickdrd oh boi! That's lot of ad domains!

[StevenBlack]

[patrickdrd]

well,
these are yours (Steven Black's ad-hoc list):

cfvod.kaltura.com
device-metrics-us-2.amazon.com
device-metrics-us.amazon.com
h-sdk.online-metrix.net
mobile.pipe.aria.microsoft.com
mywot.com
nexusrules.officeapps.live.com
s.zkcdn.net
v10.events.data.microsoft.com
v10.vortex-win.data.microsoft.com
www.googletagmanager.com
www.googletagservices.com
www.mywot.com

[patrickdrd]

@patrickdrd oh boi! That's lot of ad domains!

fyi, they're not

[dnmTX]

OK....i'll try to suppress my emotions....but still:

[anudeepND]

@patrickdrd Unblocking googletagservices.com and googtagmanager.com will enable over 50% tracking on the internet. What makes you think it's safe to whitelist?

From Wikipedia:
Google Tag Manager is a tag management system created by Google to manage JavaScript and HTML tags used for tracking and analytics on websites.

[patrickdrd]

I've posted explanation here:
https://github.com/EnergizedProtection/block/issues/53#issuecomment-447883375

[patrickdrd]

OK....i'll try to suppress my emotions....but still:

f off already man, if you're not willing to help, stop posting bs, enough!

[patrickdrd]

very weird people around here?,
not to say anything worse...

[patrickdrd]

I've got my own whitelist (1800 entries) and the above are in my whitelist for quite some time (and I've seen no ads at all),
I thought about posting here to contribute to the community,
and I didn't expect at all those shitty reactions!
I've regretted it already!

[dnmTX]

I've got my own whitelist (1800 entries) and the above are in my whitelist for quite some time,
I thought about posting here to contribute to the community,
and I didn't expect at all those shitty reactions!

[patrickdrd]

@dnmTX please do me a favor and go somewhere else to masturbate, please buddy!

[anudeepND]

@patrickdrd Posting a bunch of domains and telling to whitelist them is not going to help unless a valid reason is given.

I agree that cdn.optimizlely.com doesn't do tracking or send any analytics data. But v10.vortex-win.data.microsoft.com is used to send diagnostic data to Microsoft.

This is what the documentation states:

The Microsoft Data Management Service routes data back to our secure cloud storage. Only Microsoft personnel with a valid business justification are permitted access.

The following table defines the endpoints for Connected User Experiences and Telemetry component:

Diagnostics data: v10.vortex-win.data.microsoft.com/collect/v1

Windows Advanced Threat Protection is country specific and the prefix changes by country for example: de.vortex-win.data.microsoft.com/collect/v1
settings-win.data.microsoft.com

Read more about it here.

Did you know that softonic.com tried (and failed) to bundle adware with their own installer?
Source 1
Source 2

Did you know that Quantserve, a product of Quantcast is the biggest audience targeting or targeted advertising company?
Read more about it here

This is just a few example of the domains present in your so called "whitelist".

@dnmTX please do me a favor and go somewhere else to masturbate, please buddy!

Provide some information about those domains instead of posting rubbish comments like this.

Have a nice day!

[patrickdrd]

instead of attacking my rubbish comments, you should attack your friend for filling the whole issue/page with his bullshit and which was the originator of all this...
but, I know, we never attack our friends, attacking a stranger is much easier... makes sense...

well, in my research the above domains are needed in various functions,
if I find it I'll post here,
I mean, blocking tracking is one thing, preventing the user from doing his job is another and
in my opinion the second comes first and should always come first..

[patrickdrd]

https://discourse.pi-hole.net/t/commonly-whitelisted-domains/212

[patrickdrd]

you see, clever guy,
the answer is never that simple and one should be very careful when blacklisting important urls!

because github and the internet is full of shitty lists

[patrickdrd]

Windows10 Update

Without the follwing Whitelist Entrys Windows10 update can’t be completed.
Default is blacklisting.

pihole -w settings-win.data.microsoft.com
pihole -w v10.vortex-win.data.microsoft.com

ok, go ahead and block the user's windows update, good job!

[dnmTX]

Windows Update is working just FINE here:

NEXT PLEASE...

[anudeepND]

@patrickdrd Hold your horses,

Windows updates are never delivered via those domains.

Microsoft documentation:

Windows Update

The following endpoint is used for Windows Update downloads of apps and OS updates, including HTTP downloads or HTTP downloads blended with peers. If you turn off traffic for this endpoint, Windows Update downloads will not be managed, as critical metadata that is used to make downloads more resilient is blocked. Downloads may be impacted by corruption (resulting in re-downloads of full files). Additionally, downloads of the same update by multiple devices on the same local network will not use peer devices for bandwidth reduction.

The following endpoints are used to download operating system patches and updates. 

*.prod.do.dsp.mp.microsoft.com
*.windowsupdate.com
fg.download.windowsupdate.com.c.footprint.net
cds.d2s7q6s2.hwcdn.net
*wac.phicdn.net
*wac.edgecastcdn.net
*.tlu.dl.delivery.mp.microsoft.com.c.footprint.net
emdl.ws.microsoft.com
fe2.update.microsoft.com
fe3.delivery.mp.microsoft.com
fe3.delivery.dsp.mp.microsoft.com.nsatc.net
*.dl.delivery.mp.microsoft.com
sls.update.microsoft.com
tsfe.trafficshaping.dsp.mp.microsoft.com
a122.dscd.akamai.net
a1621.g.akamai.net

Link
And this documentation is up to date with 1809 October update.

It is clearly mentioned here that,
v10.vortex-win.data.microsoft.com/collect/v1 and settings-win.data.microsoft.com are used for collecting diagnostic data.

Link

I'm not posting this out of thin air. It's from Microsoft itself.

[dnmTX]

Steve @StevenBlack apologize up front as it's not related to the issue at hand here but it's just....
TOO CUTE:

[patrickdrd]

ok, ok, but the guys at discourse.pi-hole.net posted it, not me, anyway..

[StevenBlack]

Thanks, everyone, for your input on this.

This didn’t go very well so let’s learn from this, and move on.

Closing.

[sido420]

Can we at least put cdn.onesignal.com in whitelist? See the discussion in referenced issue.

[lightswitch05]

I’m not sure which list blocks onesignal- but these sections from their privacy policy
might give you some context as to why it’s blocked:

Before I dig in, it’s important to understand that ‘clients’ in their policy means advertisers:

all of these developers, website operators, partners and advertisers are referred to collectively as our “Clients.”

Ok, let’s start.

When permitted by the operating system, OneSignal may check to see if the device has specific applications installed, based on a limited list, for purposes that include attribution, relevancy of ads, and relevancy of notifications related to those applications.

So on android devices it’s snooping on what apps you have installed and saving that to an online profile about you

Precise Location information, generally an End User’s lat/long data (i.e., GPS-level data) or WiFi information, which we may associate with Mobile IDs, and which may be collected whether or not an app is in use.

Tracking your GPS location if it’s available- if not then it falls back on your WiFi location which is pretty much just as precise since it’s based on your WiFi MAC address

To provide information and analytics to our Clients about the use of these app and website features provided through the SDKs, or to help app developers and website operators create or enhance user profiles.

Ok so obviously it provides the information to the websites and apps that use their services- but the important part of this paragraph is that they are also providing the exact same data to their 3rd party advertisers- remember that’s clients is a grouped term.

To create inferences about End Users categorized into “Data Segments” or to help Clients do so. For instance, if SDK Information indicates that a particular device is frequently seen at restaurants, we might categorize a user for targeting of local restaurant offers. Or, if a user is frequently seen at sports stadiums, we might categorize the user as a “Sports Fan.”

Yup, so they are building profiles on people based on their location. This is a complete violation of privacy. These examples are innocuous enough- but what happens if you regularly attend a doctor? Are you sickly? What if it’s a specialist doctor- say proctologist’s. Now advertisers know that you have specific health issues. Your health information is highly sensitive and doctors have to follow strict guidelines to protect that information. But here is a “free” service that is able to accurately bypass those privacy safeguards- largely without the users knowledge or consent.

Your health information is highly valuable information. Life Insurance and health plan providers could use this information to exclude you from their advertising- both have a financial interest in avoiding customers with health issues. Beyond the advertising issues, they could even use the information to deny you as a potential customer. How would you like being denied life insurance or health coverage because this service decided to put you in a ‘sickly’ category?

Location privacy is a big deal. They could be tracking people that attend Alcoholics Anonymous meetings- oh this person must like alcohol so add that to their online profile so that we can advertise it to them.

The rest of the privacy policy is about how they use the data they have with data other people have to build as an exact profile as possible across services and devices- location, IP address, and email address are all used to help link profiles.

Conclusion: this service is trash and deserves being blocked. The obvious sign that it’s trash is that it’s completely free to use. Websites and apps that include their services are selling their users out, all for some easy push notifications.

[dnmTX]

no one thumbs-upped my Kitty GIF.That's just SAD 😢

[agneevX]

Whitelisting onesignal.com restored macOS Safari push notifications functionality for me.

[Laicure]

This was already closed.

Anyway.
This repo is just an aggregator and it is not this repo's fault why are those blocked. (we just hate tracking, ads and such, right?)
I am using this (from Energized) to unblock possible device/functionality breaking domains.

The only data source here that we can do reporting here is @StevenBlack's own list, that's the one he personally maintains while the others should not be reported here. (that's why the data sources has the update.json file that contains the issue link for their respective sites/repo)

I think @StevenBlack can remove (or move to a separate repo so this repo is not flooded with whitelist/blacklist issues) the generated hosts file altogether so the py tool is only left for usage.