Matomo: New system check to warn that Piwik is not compatible with mod_security

[mattab]

Reported in: #2997, some work was done in the early days in #1460

• Hostgator users need to contact their host to disable mod_security. Hundreds of piwik users have had to contact their hosts to disable mod_security.
• There are several known issues with Piwik and mod_security!
  
  
  
  • this thread and here
  
  

I have found that almost all rules in modsecurity_crs_41_sql_injection_attacks.conf need 

!REQUEST_COOKIES:/^_pk_ref.*/|!REQUEST_COOKIES:/^__utmz$/|!ARGS:gclid  
for google adwords, google analytics  and piwik to work ok with mod_security.

• this one

Pattern match "\\%(?![0-9a-fA-F]{2}|u[0-9a-fA-F]{4})" at ARGS:pattern. [file "/usr/local/apache/conf/modsec2.user.conf"] [line "20"] [id "1234123440"] [msg "URL Encoding Abuse Attack Attempt"] [severity "WARNING"]
  U4n-qUIt@YIADWbyzkUAAAB1 86.112.15.155 35095 66.45.249.132 80
--74545369-B-- 

As a proposed solution to inform users of potential issues early:

• Write a system check entry to check for mod_security and issue Warning if it is detected.
  
  
  
  • Suggest to user that it is OK to disable mod_security for Piwik app.
  
  
• Maybe in this system check message we could also link to FAQ and this FAQ could list the rules to disable in the mod_security config. if some users reading here may contribute them?

See similar #5081

[mattab]

It's never going to happen I think!

[mattab]

Also reported here: http://forum.piwik.org/read.php?2,88617,page=1#msg-98619

In the custom rule file, add following lines:

# Allow Piwik queries
  SecRule REQUEST_FILENAME "^/path_to_your_piwik_dir/piwik\.php$" id:99998,log,phase:2,chain,allow
  SecRule ARGS_NAMES "^action_name$"

  SecRule REQUEST_FILENAME "^/path_to_your_piwik_dir/index\.php$" id:99999,log,phase:2,chain,allow
  SecRule ARGS_NAMES "^module$"

[napa-web-designer]

Is there any update to this? I have my hosting with HostGator shared hosting and they are not willing to disable mod_sec for the whole server. The one tech I talked with said he could disable some rules for my domain as long as they are not flagged as required for them - but he would need to know which rules to disable. Is there any progress in knowing what rules to disable? This could be a path to allowing many more users to install Piwik. God Willing. Thanks.

[elsonsolano]

Same problem here with napa-web-designer. HostGator told us that they can't disable mod_sec for the whole server because i'm on a shared hosting. They just need to know which rules to disable. I don't know what rule it is to tell them.

[UVLabs]

Same, i'm on hostgator

[ordex]

This problem exists also on my onlydomains.com host. Any chance we can get the right custom rules that we can suggest to the hoster ?

[mattab]

Hello guys, at this point we do not have enough information to know how to make progress re: this issue.

The short answer: your web host should ideally not enable software that break stuff. mod_security rules are really breaking Piwik and it's not Piwik's fault (unfortunately, because that also means we can't easily fix it).

Maybe you have some details which mod security rules trigger the warnings/errors? Maybe we could contact Hostgator to get them to disable such rules for Piwik users...

[ordex]

I do agree with you: it's the way they have configured mod_sec that is breaking non-malicious applications and should be their duty to fix it.
However, I also do understand their position of not being willing to modify the mod_sec configuration to pleasure one app only (they may not know what else it will break b ychanging the config)...
Honestly, I don't know what to suggest :) But at least I got a log from my hoster OnlyDomains about my IP being blocked. Posting it here just in case it can be useful in a way or another:

 ~/scripts/modgrep.pl -s x.x.x.x -t $(date --date="Thu Jul  7 03:16:38 2016" +"%s") -r 300 -n 1234123440
Last result:

--f9ceae7f-A-- [07/Jul/2016:03:16:32 --0500] V34P4HdRQuoAAronW7MAAAAS
x.x.x.x 48624 x.x.x.x 80 --f9ceae7f-B-- POST
/piwik/index.php?date=today&format=JSON2&idSite=1&limit=15&method=SitesManager.getPatternMatchSites&module=API&pattern=%25&period=day
HTTP/1.1 Host: [1]www.yyyyyy.com User-Agent: Mozilla/5.0 (X11;
Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0 Accept: application/json,
text/plain, */* Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate
DNT: 1 Content-Type: application/x-www-form-urlencoded Cache-Control: no-cache
Referer:
[2]http://www.yyyyyy.com/piwik/index.php?module=UsersManager&action=userSettings&idSite=1&period=day&date=today&ignoreSalt=XXXXXXXXXXXXXXXXXXXXX
Content-Length: 43 Cookie:
piwik_auth=login%3DXXXXXXXXXXXXXX%3Atoken_auth%3DXXXXXXXXXXXXXXXXXXXXXXXXXXX%3D%3D%3A_%3DXXXXXXXXXXXXXXXXXXXXXx;
piwik_ignore=ignore%3DXXXXX%3D%3A_%3DXXXXXXXXXXXXXXXXXXXXXXx;
PIWIK_SESSID=XXXXXXXXXXXXXXXX Via: 1.1 x.x.x.x Connection:
keep-alive --f9ceae7f-F-- HTTP/1.1 406 Not Acceptable Content-Length: 532
Keep-Alive: timeout=5, max=91 Connection: Keep-Alive Content-Type: text/html;
charset=iso-8859-1 --f9ceae7f-H-- Message: Access denied with code 406 (phase
2). Pattern match "\\%(?![0-9a-fA-F]{2}|u[0-9a-fA-F]{4})" at ARGS:pattern.
[file "/usr/local/apache/conf/modsec2.user.conf"] [line "50"] [id
"1234123440"] [msg "URL Encoding Abuse Attack Attempt"] [severity "WARNING"]
Apache-Error: [file "core.c"] [line 3722] [level 3] File does not exist:
/home/wzoq9i8f/public_html/406.shtml, referer:
[3]http://www.yyyyyy.com/piwik/index.php?module=UsersManager&action=userSettings&idSite=1&period=day&date=today&ignoreSalt=XXXXXXXXXXXXXXXXXXXX
Action: Intercepted (phase 2) Stopwatch: Thu Jul 7 03:16:32 2016 985 (- - -)
Stopwatch2: Thu Jul 7 03:16:32 2016 985; combined=114, p1=67, p2=45, p3=0,
p4=0, p5=2, sr=31, sw=0, l=0, gc=0 Producer: ModSecurity for Apache/2.9.0
(http://www.modsecurity.org/). Server: Apache/2.2.31 (Unix) mod_ssl/2.2.31
OpenSSL/1.0.1e-fips mod_bwlimited/1.4 Engine-Mode: "ENABLED" --f9ceae7f

does it ring any bell ? something that can be changed in piwik to workaround this ? I know this has already be answered...but just in case this log has something different..

Thanks!

[mattab]

Hi @ordex thanks for the log line. In this case I believe the issue is that the Piwik URL index.php?date=today&format=JSON2&idSite=1&limit=15&method=SitesManager.getPatternMatchSites&module=API&pattern=%25&period=day matches the rule %(?![0-9a-fA-F]{2}|u[0-9a-fA-F]{4}) via the %25 in pattern=%25... it is the URL encoded of %, likely someone searches for % in the Search bar in the Site selector...

If anyone has some other log lines matching some other mod security rules, feel free to post them here.

[dune73]

Hi there, I do not know much about PiWik but quite a bit about ModSecurity. The log reported by @ordex points to rule id 1234123440 which is a custom rule outside of the official rule id ranges. So this is a local rule designed by the hoster. They should know how to disable it.

Otherwise, we are looking into publishing a brief guide on how to run and secure PiWik in combination with ModSecurity.

[JonTheWong]

Hey guys;

I just wanted to follow up on this. Here is a list of rules for piwik to serve its files properly

950120
950901
981257
981245
981240
981246
981243
950109
981248

The only issue i've been seeing is; that end-users who have piwik tracking enabled are also seeing issues, and disabling all of the above is not idea.

ps: i also found this

http://www.ashworthconsulting.com/scripts-code/using-piwik-when-mod_security-is-enabled-on-your-site.html

But haven't tested it.

[dune73]

ModSecurity Core Rule Set 3.0 has been released in the meantime. Last week actually. The new release brings a huge reduction in false positives. Most likely all or almost all of the ones mentioned by @zmjwong.

https://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/2016-November/002265.html

Please upgrade.

In the meantime I have been continuing on my ModSec/Piwik integration. Complete blogpost coming soon now.

[falzard]

http://www.ashworthconsulting.com/scripts-code/using-piwik-when-mod_security-is-enabled-on-your-site.html

Is it enough? It seems quite simple

[dune73]

They configure ModSecurity to not look at POST requests. It depends on the setup, but generally this does not sound like a safe practice.

[falzard]

Sorry if I misunderstand but they don't configure ModSecurity, they configure Piwik, no? or atleast they configure ModSecurity to not look at POST requests but for Piwik? like a whitelist?

piwikTracker.setRequestMethod('POST');

I didn't enable mod_security yet but I intend to and I just installed Piwik.

[dune73]

Sorry for not making myself clear.

Yes, that's what I meant. They configure it in a way that piwik requests coming in as POST requests bypass ModSecurity.

[Findus23]

Inspired by https://twitter.com/julianguttzeit/status/1107794236701925377
I want to dig out this old Matomo bug (feel free to move it back a few releases if they are already full) because mod_security related issues make up quite a fraction of all support requests and broken Matomo instances.

I think it would be worth it to find out common mod_security rules that break Matomo and make a system check that makes requests to URLs matching these patterns and checks if it returns 403.
If so a short text could be shown to users explaining what's the issue.

[tsteur]

I'll move it to the priority backlog. 3.10, 3.11, and pretty much 3.12 are already quite full and I think it doesn't have too much of a priority compared to all the other things.

[Findus23]

@tsteur Okay, I added the Help wanted label as implementing this shouldn't require much Matomo knowledge. (And more mod_security knowledge)

[tsteur]

👍 be for sure good to have