Owasp-modsecurity-crs: Content-Type: ... charset=ibm037 and friends allow for a bypass

[dune73]

Payload: post_param1=<script>alert(000)</script>&post_param2=<script>alert(111)</script>
Encoded: %97%96%A2%A3m%97%81%99%81%94%F1%7EL%A2%83%99%89%97%A3n%81%93%85%99%A3M%F0%F0%F0%5DLa%A2%83%99%89%97%A3nP%97%96%A2%A3m%97%81%99%81%94%F2%7EL%A2%83%99%89%97%A3n%81%93%85%99%A3M%F1%F1%F1%5DLa%A2%83%99%89%97%A3n

Attack:

POST /index.html?a=1 HTTP/1.1
...
Content-Type: application/x-www-form-urlencoded;charset=ibm037
...

%97%96%A2%A3m%97%81%99%81%94%F1%7EL%A2%83%99%89%97%A3n%81%93%85%99%A3M%F0%F0%F0%5DLa%A2%83%99%89%97%A3nP%97%96%A2%A3m%97%81%99%81%94%F2%7EL%A2%83%99%89%97%A3n%81%93%85%99%A3M%F1%F1%F1%5DLa%A2%83%99%89%97%A3n

This brings the following results (CRS 3.0.2 on ModSec 2.9.2)

PL1: 0
PL2: 3
PL3: 16
PL4: 29

Extracted from https://www.slideshare.net/SoroushDalili/waf-bypass-techniques-using-http-standard-and-web-servers-behaviour slide 32.

The presentation also looks into multipart transfer with special encodings. That might be even harder to catch.

I take it, we need to create a whitelist for these encodings. UTF-8 seems like a minimal entry, but what other encodings do we need by default?

Also: This is quite a thing and I think we need to create something for 3.1.

[lifeforms]

I had searched a few days of logs for this header while working on my Content-Type rule, and I only ever saw UTF-8 there. But my installs are not so enterprisey. Do others have more log data to search?

If we whitelist this, I reckon we should make it a configurable setting, like we are doing with MIME types, file extensions, etc. Would you agree? If so, there would be lower pressure to create an exhaustive list, as we'd have a solid workaround for those with more exotic requirements.

[dune73]

Spot on. I'm currently in touch with @irsdl, who published this, and he agrees on the UTF-8 and suggests to include ISO-8859-1, possibly also other UTF variants.

I agree on the sane default in a configurable setting and then people can add additional charsets at their discretion. Maybe we need to point out in the configuration that more exotic charsets will allow for bypasses, namely in PL1 while higher PLs will lead to lots of FPs with exotic charsets.

[csanders-git]

v3.2 candidate or v3.1.1?

[dune73]

No new rules for v3.1.1 please, so it's a question if this is urgent enough to squeeze it into v3.1.0 or postpone it to v3.2.

Slide 25 of the presentation linked above lists the backends that comply with the charsets to allow for the bypass and the exposure looks really big. So personally, I would opt for an urgent fix for 3.1.0 but I will support your call.

Also, it seems this is something where we can set ourselves apart from the commercial vendors that are not based on CRS.

@irsdl: Do you want to chime in?

[dune73]

More private conversation with @irsdl: UTF-8 should be sufficient.

[lifeforms]

I vote for 3.1.

[dune73]

I have a PR mostly ready, but my rule can be bypassed via charset=utf-8,ibm037. Apache eats this just fine and I have no idea what a backend will make of it. I get the feeling we should do a characterset whitelisting on the charset parameter and rule out commas. But ideally in @lifeforms 920470 (see PR #1103), or in an additional rule.

Here is what I have so far:

SecRule REQUEST_HEADERS:Content-Type "@rx charset\s*=\s*([^;\s]+)" \
    "id:920480,\
    phase:1,\
    block,\
    t:none,t:lowercase,\
    msg:'Illegal Content-Type charset',\
    logdata:'%{matched_var}',\
    tag:'application-multi',\
    tag:'language-multi',\
    tag:'platform-multi',\
    tag:'attack-protocol',\
    tag:'OWASP_CRS/PROTOCOL_VIOLATION/CONTENT_TYPE_CHARSET',\
    tag:'WASCTC/WASC-20',\
    tag:'OWASP_TOP_10/A1',\
    tag:'OWASP_AppSensor/EE2',\
    tag:'PCI/12.1',\
    rev:1,\
    ver:'OWASP_CRS/3.0.0',\
    severity:'CRITICAL',\
    capture,\
    chain"
    SecRule TX:1 "!@rx ^%{tx.allowed_request_content_type_charset}$" \
        "t:none,\
        ctl:forceRequestBodyVariable=On,\
        setvar:'tx.msg=%{rule.msg}',\
        setvar:'tx.anomaly_score=+%{tx.critical_anomaly_score}',\
        setvar:'tx.%{rule.id}-OWASP_CRS/PROTOCOL_VIOLATION/CONTENT_TYPE_CHARSET-%{matched_var_name}=%{matched_var}'"

I think I need to depend on the capture group here. Alternative ideas that do not involve an additional SecRule are welcome.

[spartantri]

Hi @dune73 ,

Have you tried that rule with multiple headers?
First good then bad:
Content-Type: charset=good;
Content-Type: charset=bad;

And bad then good:
Content-Type: charset=bad;
Content-Type: charset=good;

We fixed a few instances of that behavior a few months back. There should be only one C-T (multipart is a different story) or kill otherwise.

Multipart improvements are missing in ModSec there is no way to pipe the parts to different body processors yet may be implemented as lua script or transform but painful from rules.

I prefer white listing more popular charsets/CT that modsec can transform/decode and warn the user on the settings to add but from black list point of view this is a hard battle if the application is badly coded however this is a very hard task when we are aiming for a generic implementation, maybe some of the positive checks can live inside the exceptions so that we do positive and negative checks on most popular CMS/apps.

[dune73]

Sweet. Going to take this into consideration. It's true that we do not have anything in CRS stopping a client from submitting multiple CT headers.

As for whitelist / blacklist: Yes, we need to do WL on CT header. No way being liberal here.

[dune73]

I checked: Double header is being concatenated to application/x-www-form-urlencoded; charset = utf-8, application/x-www-form-urlencoded; charset = ibm037, which results in TX:1 being utf-8,. Not good.

[spartantri]

Well an additional &REQUEST_HEADERS:Content-Type !@eq 1 block rule in protocol enforcement should fix that, but this may happen with some other headers, maybe during the next monthly meeting we can make a list of those we would like to forbid any duplicates.

[dune73]

You wish...

Apache concatenates double headers with comma+space as separator. When counting the header collection and the header names collection, the 2nd instance of the header is already missing.

I second your proposal to add very strict rules on what we accept in some of the key headers. Like whitelisting the header format at PL3 and using the whitelist pattern to make sure no is duplicate headers are submitted.

[spartantri]

oops :( wasn't that done after phase:1? then some regex to do the trick REQUEST_HEADERS:Content-Type @rx charset.+charset I wonder if we have other issues such as this somewhere else?

[csanders-git]

We can sneak it in if it's done soon (in the next 5 days).... But we really need to merge hybrid paranoia first... Can you take care of conflicts there

[dune73]

1076 / Hybrid Paranoia Mode is ready to be merged without any remaining conflicts.

[dune73]

OK. Here is what we do:

Step 1

I'll write the PR to cover this as laid out above and try to merge this into 3.1/dev as it seems @csanders-git allows us to do that.

Step 2

We forget the duplicate headers for the time being and I open an issue to add multiple rules to whitelist the format of the most important request headers for 3.2 (maybe at PL3 or so).

[dune73]

OK, I checked RFC 7231 for the the format of charset and it seems we do not need to worry about utf-8,ibm037. This is illegal. So we can postpone this to v3.2 too and I will now base my PR on the proposal made above.

[dune73]

PR in #1143.

[emphazer]

@dune73 you should definetly add ISO-8859-1 3-5% will need it. especially for older pages

[dune73]

Thank you @emphazer.

[dune73]

Closing in favor of #1258.