Adguardhome: Assign client tags automatically based on the clients behavior

[ameshkov]

Now that we have ctag modifier support, we should think about how to assign at least some of the tags automatically. At least we could assign os_* and device_* tags.

Here's what I suggest:

1. Create a ctag detector module that checks DNS queries and tries to figure out what the client's OS might be judging by its activity.
2. When we are sure that the client's OS is X.
3. Remember that the client may represent multiple different devices. If the client activity signals that there are different operating systems or devices in use, assign multiple tags.
4. Add a setting to the UI: "Add client tags automatically".

Activity marker

Every time a device connects to the network it connects to a number of home servers. This statement is true for every OS and I believe that this "marker" can help us detect the device type automatically.

Now, to do this we need to figure out what these "activity markers" for each OS/device might be and we need help with that.

How can you help us

Please do the following:

1. Make sure that your device uses AdGuard Home
2. Disconnect it from the network
3. Connect it to the network again
4. Wait for 30 seconds, do nothing while waiting
5. Get the list of domains requested by this device
6. Post the following here or send it to [email protected]: the list of domains requested, your device info (OS type, OS version, device model).

[AnthonyBe]

Amazon Echo Smart Speaker:
ntp-g7g.amazon.com
dp-rsm-prod.amazon.com
spectrum.s3.amazonaws.com
api.amazonalexa.com
device-metrics-us.amazon.com (blocked by AG Simple DNS filter)

There are many others, but the above are the more unique ones and they are polled regularly by my Echo devices.

[AnthonyBe]

Nest Protect (smoke detector)
weave-logsink.nest.com
czfe106.front01.iad01.production.nest.com

[AnthonyBe]

Hisense Smart TV
api-gps-em.hismarttv.com
auth-em.hismarttv.com
msg-em.hismarttv.com
api-launcher-em.hismarttv.com
auth-launcher-em.hismarttv.com
unified-ter-na.hismarttv.com
upgrade-em.hismarttv.com

[AnthonyBe]

SiliconDust HDHomeRun network TV tuner device
tuner-api.hdhomerun.com
location-api.hdhomerun.com

[AnthonyBe]

LIFX Smart light
v2.broker.lifx.co

[AnthonyBe]

Sensibo Smart Air Conditioner controller
new-emq.sensibo.com
new-config.sensibo.com

[AnthonyBe]

Foxtel cable/satellite set top box
_xmpp-client._tcp.managed.xmpp.foxtel.com.au
managed.xmpp.foxtel.com.au
foxtel-prod-events.digitalsmiths.net
e2.resources.foxtel.com.au
a1.resources.foxtel.com.au

[AnthonyBe]

Yamaha AV Receiver
avpro.global.yamaha.com

[AnthonyBe]

Netatmo smart home device
netcom.netatmo.net

[AnthonyBe]

Smart Lock - OEM digital key solution used by Ring.com, Kwikset, Nest and others
tumblergsprod.unikey.com

[AnthonyBe]

Wink Smart home controller
hub-api.wink.com

[AnthonyBe]

SolarEdge Inverter
prod2.solaredge.com

[AnthonyBe]

VELUX ACTIVE indoor climate control device
nv2-nxg.netatmo.net

[AnthonyBe]

Underfloor Heating Thermostat
owd5-r1099-thermostat.ojelectronics.com

OJElectronics is an OEM used by many brands of underfloor heating systems

[DandelionSprout]

Some of those entries posted by AnthonyBe above (which, let's be very honest, could and should have been posted in one single comment instead of 14) seems like a good fit for Perflyst's smart-TV list. I could submit them to it myself.

For the rest of them, some sort of smarthome-specific list would've had to be made, of which none are known to exist at the time of writing.

[ameshkov]

@AnthonyBe thank you so much!

@DandelionSprout well, the goal is not to add them to some blocklist but to use these domains as a "marker" of the device type

[DandelionSprout]

Ah, that explains it.

Also makes me feel slightly embarrassed for having automatically assumed that all of those entries were tracking/ad domains suitable for Perflyst. 😅

At least it's not much difficult to remove entries from his lists that'd be critical for unit functionality, though I only own Yamaha receivers among those units I took care of previously.

[hoshsadiq]

Sony TVs seem to be connecting to these:

||ad8641f3cff742de893d919add74c2bb.ssm1.internet.sony.tv^$important
||ad8641f3cff742de893d919add74c2bb.ssm2.internet.sony.tv^$important
||call.me.sel.sony.com^$important

These two can be generically pointed to TV device:

||reporting-tv1-live.youview.tv^$important
||reporting-tv1.youview.tv^$important

[TXC]

Philips Hue
diagnostics.meethue.com
dcp.dc1.philips.com
www.ecdinterface.philips.com

IKEA Trådfri
webhook.logentries.com

Synology NAS
checkip.synology.com
frlp.lp.cs.quickconnect.to

[DandelionSprout]

• Note to self: The following domains are not supposed to be blocked.
• Note to everyone: * is a wildcard for 2-letter country codes.

PlayStation 3:
a0.ww.np.dl.playstation.net
ena.net.playstation.net
feu01.ps3.update.playstation.net
getprof.*.np.community.playstation.net
iv0001-npxs01001-00.auth.np.ac.playstation.net
native-ps3.np.ac.playstation.net
nsx-e.np.dl.playstation.net
sf.prod.sonyentertainmentnetwork.com
static-resource.np.community.playstation.net
trophy.ww.np.community.playstation.net
trophy01.np.community.playstation.net
ubstorage01.ww.np.community.playstation.net
updptl.*.np.community.playstation.net
us.np.stun.playstation.net
*.np.adproxy.ndmdhs.com

Wii U:
account.nintendo.net
discovery.olv.nintendo.net
ecs.wup.shop.nintendo.net
ias.wup.shop.nintendo.net
mii-secure.account.nintendo.net
nncs1.app.nintendowifi.net
nncs2.app.nintendowifi.net
nppl.app.nintendo.net
npts.app.nintendo.net
nus.wup.shop.nintendo.net
pushmore.wup.shop.nintendo.net
tagaya.wup.shop.nintendo.net
wup-o2fgs.cdn.nintendo.net

Nintendo 3DS (in 3DS mode):
ctr-adqj-live.s3.amazonaws.com
ctr-o2fgs.cdn.nintendo.net
hpp-00051600-l1.n.app.nintendowifi.net
hpp-00178800-l1.n.app.nintendowifi.net
l-npns.app.nintendo.net
nasc.nintendowifi.net
npdl.cdn.nintendowifi.net
nppl.c.app.nintendowifi.net
npul.c.app.nintendowifi.net
nus.c.shop.nintendowifi.net
pubeu-p.est.c.app.nintendowifi.net
tagaya-ctr.cdn.nintendo.net

T-We Boks II (Telenor Norway's main set-top box) (Also applies to T-We Boks 1 and T-We Boks Mini from the 24th of April 2020):
p-sdp-fe.tvs.telenor.net
p-sdp-mqtt.tvs.telenor.net
p-sdp-sso.tvs.telenor.net
tnfba-static.telenorcdn.net
tnfba-static2.telenorcdn.net
tnfba-stream-od.telenorcdn.net
ntp.online.no
d2emmtcqemyhm9.cloudfront.net

LG webOS TVs (Tested with LG 43UH603V):
*.rdx2.lgtvsdp.com
lgtvonline.lge.com
snu.lge.com
*.info.lgsmartad.com
*.lgtvsdp.com
(Various garbled letter combinations of between 8 and 15 letters)

Yamaha RX-V685 (Likely also applies to other receivers in the RX-Vx8x generations, especially RX-Vx85):
3573112786.airable.io
avpro.global.yamaha.com
avs.na.amazonalexa.com
ntp.airable.io
spectrum.s3.amazonaws.com

EPSON Eco-Tank ET-4550:
p01.epsonconnect.com
x01.epsonconnect.com

[ameshkov]

Btw, what about your phones? It's rather easy to detect iOS, but Android may be tricky because of the different manufacturers.

[AnthonyBe]

I've got a Samsung Galaxy S10+ running Android 10.
I rebooted it and checked the AGH logs to see what things stood out that in _combination_ may help identify it:

_points to Android device_
android.clients.google.com
android.googleapis.com
time.android.com

_points to Samsung device_
sspapi-prd.samsungrs.com
api.samsungcloud.com
capi.samsungcloud.com
vas.samsungapps.com
us-auth2.samsungosp.com
acs.samsungmdec.com
nms-m0-ase1.samsungmdec.com
es-central-ase1.samsungmdec.com
acs-central-ase1.samsungmdec.com
pcscf1-c0-ase1.samsungmdec.com
pcscf2-c0-ase1.samsungmdec.com
pcscf3-c0-ase1.samsungmdec.com

_points to Vodafone carrier provided handset_
supl.vodafone.com

there was a lot of other *.googleapis.com stuff but that could be indicative of any Google app on any device.

[DandelionSprout]

Both of my current phones use LineageOS 14, so while my findings wouldn't be able to detect specific manufacturers, I suppose I could look into it nevertheless later today.

[DandelionSprout]

Trying to remember which apps on my Android 7.1.2 main phone that were Android-exclusive and which ones weren't, was rather difficult, but here's my findings for Android:

android.googleapis.com
in.appcenter.ms
firebaseinstallations.googleapis.com
android.clients.google.com
firebaseremoteconfig.googleapis.com
firebase-settings.crashlytics.com
play.googleapis.com
mighty-app.appspot.com
semanticlocation-pa.googleapis.com
googlehomefoyer-pa.googleapis.com
download.lineageos.org
dl.xposed.info
dl-xda.xposed.info
appsitemsuggest-pa.googleapis.com
os-*.storage.googleapis.com

If I find additional domains that seem to be specific to Android, I'll then update this comment.

[devinslick]

Rachio Irrigation Controller
pool.ntp.org
a3bmbcwe3hybwy.iot.us-west-1.amazonaws.com

Wyzecam
api.wyzecam.com
wyze-device-alarm-file.s3.us-west-2.amazonaws.com
gm.iotcplatform.com
cm.iotcplatform.com
time-a.nist.gov
a24rq1e5m4mtei-ats.iot.us-west-2.amazonaws.com

HarmonyHub
home.myharmony.com
sus.dhg.myharmony.com
ps-823.pubnub.com
svcs.myharmony.com
cf-svcs.myharmony.com

Tplink wifi switch
pool.ntp.org
time-a.nist.gov
deventry.tplinkcloud.com
devs.tplinkcloud.com
use1-api.tplinkra.com

Envisalink EyezOn
alerts2.envisacor.com

Some devices, like the the Rachio device listed above, may be difficult to identify based on their limited queries. Would correlating the manufacturer, based on the MAC address be useful, or possible?
Using this information you could see that the Rachio controller (44:91:60:00:00:00) was created by Murata Manufacturing Co., Ltd.
This would obviously require use of AGH as a DHCP server.

[IthildinX]

Xiaomi Air Purifier 2s(小米空氣淨化器2S)
api.miwifi.com
ot.io.mi.com

Mijia Smart AI Alarm Clock(小米小爱智能闹钟)
relay-dcm.ai.xiaomi.com

MiAiSoundbox(小爱触屏音箱)
api.ai.xiaomi.com
app.chat.xiaomi.net
broker.miwifi.com
relay-dcm.ai.xiaomi.com
app.chat.xiaomi.net
resolver.msg.xiaomi.net

zimi-powerstrip(米家智能插线板) & chuangmi-plug-m1/m3(米家智能插座wifi版/增強版)
ott.io.mi.com
ot.io.mi.com

dmaker-fan(米家直流變頻落地扇1X)
ot.io.mi.com
dlg.io.mi.com

[HorayNarea]

I have not a list of domains but more a domain-pattern: *hbbtv* indicates some kind of smart-TV

Edit:
A more specific regex would be ^hbbtv\.* as many stations (at least here in germany) publish their HbbTV content on this subdomain