Matomo: DoNotTrack not recognized by matomo optout-script

[xopez]

Hello everyone,

I noticed that on LineageOS (Android Rom) the default browser (built-in and I think its Jelly Browser) doesn't really recognize the DoNotTrack-function. I set the option in the browser and my page still says that I can Opt-Out. Other Browsers are working fine and I'm told I have the option set. Don't know it's a browser bug or not. Screens with German text, but the position where it stands is marked red.

android package name of the browser: org.lieangeos.jelly
useragent: Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36

OptOut-Iframe:
<iframe style="border: 0; height: 200px; width: 600px;" src="https://analytics.mightful-noobs.de/index.php?module=CoreAdminHome&amp;action=optOut&amp;language=de&amp;backgroundColor=&amp;fontColor=606060&amp;fontSize=100%&amp;fontFamily=Montserrat%20Regular" width="300" height="150"></iframe>

[sgiehl]

Are you able to figure out which headers your browser sends when DNT is enabled?

[xopez]

I can give you this with mod_log_forensic of my apache-server

+Wsdr8QUJi8sAACuXGHkAAABF|GET /piwik.php?ping=1&idsite=1&rec=1&r=618026&h=14&m=45&s=37&url=https%253A%252F%252Fmightful-noobs.de%252Fdatenschutzerklaerung%252F&_id=&_idts=1523018737&_idvc=1&_idn=1&_refts=0&_viewts=1523018737&send_image=1&cookie=1&res=458x813&gt_ms=170&pv_id=BjbJiy HTTP/1.1|Host:analytics.mightful-noobs.de|Connection:keep-alive|User-Agent:Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36|Accept:image/webp,image/apng,image/*,*/*;q=0.8|Referer:https%3a//mightful-noobs.de/datenschutzerklaerung/|Accept-Encoding:gzip, deflate|Accept-Language:de-DE,en-US;q=0.9|X-Requested-With:org.lineageos.jelly
-Wsdr8QUJi8sAACuXGHkAAABF
+Wsdr-QUJi8sAACuWX4MAAAAF|GET /datenschutzerklaerung/ HTTP/1.1|Host:mightful-noobs.de|Connection:keep-alive|Upgrade-Insecure-Requests:1|User-Agent:Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36|Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8|dnt:1|Accept-Encoding:gzip, deflate|Accept-Language:de-DE,en-US;q=0.9|X-Requested-With:org.lineageos.jelly
-Wsdr-QUJi8sAACuWX4MAAAAF
+Wsdr-QUJi8sAACuWX4QAAAAE|GET /wp-content/cache/wpfc-minified/342b2691b69150851c22ca32d736dd98/1523012503index.css HTTP/1.1|Host:mightful-noobs.de|Connection:keep-alive|User-Agent:Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36|Accept:text/css,*/*;q=0.1|Referer:https%3a//mightful-noobs.de/datenschutzerklaerung/|Accept-Encoding:gzip, deflate|Accept-Language:de-DE,en-US;q=0.9|X-Requested-With:org.lineageos.jelly
-Wsdr-QUJi8sAACuWX4QAAAAE
+Wsdr-QUJi8sAACuXGHoAAABJ|GET /wp-content/cache/wpfc-minified/368665ef6540fd2eb25ad7491c2ef378/1523013122index.js HTTP/1.1|Host:mightful-noobs.de|Connection:keep-alive|User-Agent:Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36|Accept:*/*|Referer:https%3a//mightful-noobs.de/datenschutzerklaerung/|Accept-Encoding:gzip, deflate|Accept-Language:de-DE,en-US;q=0.9|X-Requested-With:org.lineageos.jelly
-Wsdr-QUJi8sAACuXGHoAAABJ
+Wsdr-QUJi8sAACuXGHsAAABI|GET /index.php?module=CoreAdminHome&action=optOut&language=de&backgroundColor=&fontColor=606060&fontSize=100%25&fontFamily=Montserrat%2520Regular HTTP/1.1|Host:analytics.mightful-noobs.de|Connection:keep-alive|Upgrade-Insecure-Requests:1|User-Agent:Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36|Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8|Referer:https%3a//mightful-noobs.de/datenschutzerklaerung/|Accept-Encoding:gzip, deflate|Accept-Language:de-DE,en-US;q=0.9|X-Requested-With:org.lineageos.jelly
-Wsdr-QUJi8sAACuXGHsAAABI
+Wsdr-gUJi8sAACuXGHwAAABL|GET /piwik.php?action_name=Datenschutzerkl%25C3%25A4rung%2520%257C%2520Mightful%2520Noobs&idsite=1&rec=1&r=691968&h=14&m=45&s=50&url=https%253A%252F%252Fmightful-noobs.de%252Fdatenschutzerklaerung%252F&_id=&_idts=1523018750&_idvc=1&_idn=1&_refts=0&_viewts=1523018750&send_image=1&cookie=0&res=458x813&gt_ms=84&pv_id=JoDFZe&webgl=1 HTTP/1.1|Host:analytics.mightful-noobs.de|Connection:keep-alive|User-Agent:Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36|Accept:image/webp,image/apng,image/*,*/*;q=0.8|Referer:https%3a//mightful-noobs.de/datenschutzerklaerung/|Accept-Encoding:gzip, deflate|Accept-Language:de-DE,en-US;q=0.9|X-Requested-With:org.lineageos.jelly
-Wsdr-gUJi8sAACuXGHwAAABL
+WsdsAQUJi8sAACuXGH0AAABK|GET /piwik.php?ping=1&idsite=1&rec=1&r=116337&h=14&m=45&s=53&url=https%253A%252F%252Fmightful-noobs.de%252Fdatenschutzerklaerung%252F&_id=&_idts=1523018753&_idvc=1&_idn=1&_refts=0&_viewts=1523018753&send_image=1&cookie=1&res=458x813&gt_ms=170&pv_id=BjbJiy HTTP/1.1|Host:analytics.mightful-noobs.de|Connection:keep-alive|User-Agent:Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36|Accept:image/webp,image/apng,image/*,*/*;q=0.8|Referer:https%3a//mightful-noobs.de/datenschutzerklaerung/|Accept-Encoding:gzip, deflate|Accept-Language:de-DE,en-US;q=0.9|X-Requested-With:org.lineageos.jelly
-WsdsAQUJi8sAACuXGH0AAABK

LogFormat for this line in Apache:
LogFormat "%{forensic-id}n %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

As you can see "dnt:1" at 3rd line is send.

[Findus23]

Hi, I can reproduce it with LineageOS 14.1 and org.lineageos.jelly.
I'll check to find out why.

[Findus23]

I fail to setup SSL with mitmproxy so I can only intercept HTTP requests, but it seems like no matter how one sets the "Aktivitäten nicht verfolgen" setting, it never seems to add an dnt header.

Opening http://request.urih.com/ in the browser shows the same.

[Findus23]

@xopez Which version are you using exactly?

I am having a different user agent then you

yours: Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36
mine:  Mozilla/5.0  (Linux; Android 7.1.2; ONEPLUS A3003 Build/NJH47F) AppleWebKit/537.36  (KHTML, like Gecko) Version/4.0 Chrome/63.0.3239.111 Mobile  Safari/537.36

UPDATE:
That's fascinating: When I switch the WebView-Imprementation in the Android Developer Options from "AOSP WebView" (the open source LineageOS ones) to "Chrome Stable" it starts sending a Dnt:1 header and $_SERVER["HTTP_DNT"]=="1".

But the Opt-Out screen is still shown.

[Findus23]

@sgiehl It seems like the request even gets tracked:
https://gist.github.com/Findus23/0913c69f10c4ff5ce2a0b53e8c98ef3e

[xopez]

I am using

14.1-20180405-NIGHTLY-hiaeuhl

But also noticed it before the version.
But I patch mostly every version and latest gapps-pico package to keep my device up2date.

[xopez]

yes, cause it says it didn't find DNT.
DEBUG PrivacyManager[2018-04-06 13:40:43 UTC] [92aa0] DoNotTrack header not found
https://gist.github.com/Findus23/0913c69f10c4ff5ce2a0b53e8c98ef3e#file-gistfile1-txt-L52

[fdellwing]

As it works fine with Chrome, this is a Jellybug, isn't it?

[Findus23]

@fdellwing At least partly.

But I'm stil unsure why it isn't working when one switches to the Chrome Webview, because it sends dnt:1 and $_SERVER["HTTP_DNT"] is correctly set to "1"

[fdellwing]

I don't have Jelly installed, only Chrome, so I cannot help figuring this out.

[Findus23]

Isn't it installed by default (simply called "Browser")?

[xopez]

yes. The jelly-browser is called "Browser" in the app launcher in LineageOS. And it's a system app, so can't remove it so easily.

[fdellwing]

I think I choose to get rid of it for Chrome while setting up the phone. At least there is no default Browser app on my phone anymore.

[Findus23]

After a lot of debugging I have now finally found (at least part of) the solution:

I have created an HTML page with an opt-out-iframe of an non-https matomo instance (so I can proxy the request)

It turns out that the DNT-header isn't sent to pages in iFrames.

@xopez So if you could try out to directly surf to the https://yourmatomo.example/index.php?module=CoreAdminHome&action=optOut&idsite=14&language=de URL, it should correctly show the Opt-Out.

[fdellwing]

It turns out that the DNT-header isn't sent to pages in iFrames.

So, a problem with the implementation of iframe in jelly?

[Findus23]

And to fully solve the mystery and show that this has nothing to do with Matomo:

I added JS to the page to make a AJAX request, and it turns out that it also doesn't get a DNT header:

Therefore Matomo has no chance to know that the user has enabled DNT and therefore tracks the user.

I'll create a bugreport to LineageOS as it seems the DNT-feature is completely useless.

[fdellwing]

@Findus23 I dont see a thread in jira, can you provide a link?

[Findus23]

They only open their bug tracker from saturday to Sunday.

[fdellwing]

Ok, if you are able to please send it to me via forum :)

[xopez]

Just post it here, so we can follow it.

[Findus23]

@fdellwing, @xopez
The bug has been reported here: https://jira.lineageos.org/browse/BUGBASH-1552
Also a bit unfortunately that a great OS encouraging people not to depend on Google are using Google Analytics on their website.

[Findus23]

Semi-related:
I have now started a discussion about using Google Analytics:
https://www.reddit.com/r/LineageOS/comments/8aowso/please_dont_use_google_analytics/