Nixpkgs: Broken PCRE JIT in PHP SAPI module for Apache

[aszlig]

TL;DR: When using PHP 7.1 or lower in conjunction with the Apache SAPI, the preg_* functions no longer work as expected.

I stumbled on this while investigating a test failure for nixcloud and I bisected this to the update of the PCRE library to version 8.41 in 119ebee6a4f66d4e623e058e897bdd5928ba478b (#30963). After a bit of debugging and another bisect against the PCRE library I found the commit which is causing this:

https://vcs.pcre.org/pcre?view=revision&revision=1690

With the CLI version of PHP this problem isn't showing up at all, which I found a bit odd.

So I dug a bit deeper and found that PHP bundles PCRE 8.40, which of course doesn't include the PCRE commit mentioned above. Now PHP itself doesn't have pcre in its buildInputs but Apache uses it.

My suspicion is that some parts of the shared library from Apache are re-used in PHP, but for now I don't have time to dig further into that.

Here a small test which should help to reproduce the problem:

{ jit ? true, ... }@args:

import <nixpkgs/nixos/tests/make-test.nix> {
  name = "apache-php-sapi-pcre-issue";

  machine = { lib, pkgs, ... }: {
    services.httpd.enable = true;
    services.httpd.adminAddr = "[email protected]";
    services.httpd.enablePHP = true;
    services.httpd.phpOptions = lib.optionalString (!jit) "pcre.jit=0";
    services.httpd.documentRoot = pkgs.writeTextFile {
      name = "docroot";
      destination = "/index.php";
      text = ''
        <?php
        if (preg_match('/^(?:[0-9]+|[a-z])$/', $_GET['input']))
          echo "YAY\n";
      '';
    };
  };

  testScript = ''
    sub testInput {
        return 'curl "http://localhost/index.php?input='.$_[0].'"'
             . ' | grep -qF YAY';
    }

    $machine->waitForUnit('multi-user.target');
    $machine->succeed(testInput 10);
    $machine->fail(testInput 'foo');
    $machine->succeed(testInput 'x');
  '';
} args

Whenever running with --arg jit false or the commit before 119ebee6a4f66d4e623e058e897bdd5928ba478b, the test runs fine.

With the preg_* functions not working as expected (due to the top-down stack in the JIT), I'd consider this quite risky for people running various PHP web applications that rely on regular expressions to do their input validation.

Maybe it makes sense to disable the PCRE JIT for PHP versions <= 7.1 until we have found out what's wrong?

PHP version 7.2 bundles PCRE 8.41 and doesn't have this problem.

Cc: @qknight, @fpletz, @globin, @grahamc

[vcunat]

It's theoretically possible they manage to mix up the symbols from the two versions. (E.g. dynamic lib pulled by some dependency.)

[aszlig]

@vcunat: Mhm, that was my suspicion as well.

[vcunat]

BTW, if they bundle pcre-8.40, they might be affected by that vulnerability...

[aszlig]

They should have patched it already, see the changelog at http://www.php.net/ChangeLog-7.php#7.1.11:

• PCRE:
  
  
  
  • Fixed bug #75207 (applied upstream patch for CVE-2016-1283).
  
  

[pbogdan]

As mentioned in https://github.com/NixOS/nixpkgs/issues/31451 building PHP with system pcre library would seem to fix this though I don't understand why (or what other implications it might have).

[dezgeg]

Symbol mixup definitely seems plausible. They do apparently attempt to mangle the symbols of the bundled libpcre (by adding a php_ prefix) but miss some global symbols... like pcre_jit_exec:

$ readelf -s result/modules/libphp7.so | grep pcre_ 170: 00000000001c6720 303 FUNC GLOBAL DEFAULT 11 php_pcre_get_stringnumber 305: 00000000001c6d40 103 FUNC GLOBAL DEFAULT 11 php_pcre_get_named_substr 351: 00000000001b1a10 5514 FUNC GLOBAL DEFAULT 11 php_pcre_compile2 409: 00000000001c6dc0 806 FUNC GLOBAL DEFAULT 11 php_pcre_maketables 489: 00000000001f0870 36 FUNC GLOBAL DEFAULT 11 pcre_jit_stack_free 618: 00000000001f56a0 2348 FUNC GLOBAL DEFAULT 11 php_pcre_split_impl 707: 00000000001c7660 96 FUNC GLOBAL DEFAULT 11 php_pcre_refcount 713: 00000000001c6ca0 9 FUNC GLOBAL DEFAULT 11 php_pcre_free_substring_l 826: 00000000001c6bb0 233 FUNC GLOBAL DEFAULT 11 php_pcre_get_substring_li 1375: 00000000001f12d0 2639 FUNC GLOBAL DEFAULT 11 pcre_get_compiled_regex_c 1615: 00000000001c8ab0 939 FUNC GLOBAL DEFAULT 11 php_pcre_study 1731: 00000000001c6ad0 103 FUNC GLOBAL DEFAULT 11 php_pcre_copy_substring 1946: 00000000001f0520 421 FUNC GLOBAL DEFAULT 11 pcre_jit_exec 2343: 00000000001f1df0 4633 FUNC GLOBAL DEFAULT 11 php_pcre_match_impl 2431: 00000000001f08c0 137 FUNC GLOBAL DEFAULT 11 pcre_jit_free_unused_memo 2465: 00000000001f1d20 92 FUNC GLOBAL DEFAULT 11 pcre_get_compiled_regex 2506: 0000000001068bb8 8 OBJECT GLOBAL DEFAULT 26 pcre_stack_guard 2650: 00000000010488e0 8 OBJECT GLOBAL DEFAULT 25 php_pcre_malloc 2675: 00000000001f1d80 108 FUNC GLOBAL DEFAULT 11 pcre_get_compiled_regex_e 2792: 00000000010845ec 4 OBJECT GLOBAL DEFAULT 26 pcre_globals_id 3020: 00000000001f6260 1004 FUNC GLOBAL DEFAULT 11 php_pcre_grep_impl 3057: 00000000010488c8 8 OBJECT GLOBAL DEFAULT 25 php_pcre_stack_free 3257: 00000000001c6b40 109 FUNC GLOBAL DEFAULT 11 php_pcre_copy_named_subst 3289: 0000000001068bc0 8 OBJECT GLOBAL DEFAULT 26 php_pcre_callout 3349: 00000000001c6cb0 135 FUNC GLOBAL DEFAULT 11 php_pcre_get_substring 3430: 00000000001c4f50 5442 FUNC GLOBAL DEFAULT 11 php_pcre_exec 3464: 00000000001c6850 468 FUNC GLOBAL DEFAULT 11 php_pcre_get_stringtable_ 3488: 00000000001c9210 8 FUNC GLOBAL DEFAULT 11 php_pcre_version 3527: 00000000001f0750 283 FUNC GLOBAL DEFAULT 11 pcre_jit_stack_alloc 3606: 00000000010488d8 8 OBJECT GLOBAL DEFAULT 25 php_pcre_free 3631: 00000000010488d0 8 OBJECT GLOBAL DEFAULT 25 php_pcre_stack_malloc 3815: 00000000001c8a70 50 FUNC GLOBAL DEFAULT 11 pcre_free_study 3834: 00000000001b2fb0 177 FUNC GLOBAL DEFAULT 11 php_pcre_config 3840: 00000000001f4430 108 FUNC GLOBAL DEFAULT 11 php_pcre_replace 3968: 00000000001c64a0 633 FUNC GLOBAL DEFAULT 11 php_pcre_fullinfo 4273: 00000000001f08a0 29 FUNC GLOBAL DEFAULT 11 pcre_assign_jit_stack 4417: 00000000001b2fa0 16 FUNC GLOBAL DEFAULT 11 php_pcre_compile 4859: 00000000001c6db0 9 FUNC GLOBAL DEFAULT 11 php_pcre_free_substring 4884: 00000000001f3360 4297 FUNC GLOBAL DEFAULT 11 php_pcre_replace_impl 4624: 00000000001f0950 36 FUNC LOCAL DEFAULT 11 pcre_clean_cache 4627: 00000000001f0b60 51 FUNC LOCAL DEFAULT 11 zm_globals_dtor_pcre 4629: 00000000001f0ba0 48 FUNC LOCAL DEFAULT 11 zm_globals_ctor_pcre 4630: 00000000001f0bd0 66 FUNC LOCAL DEFAULT 11 php_free_pcre_cache 4631: 00000000001f0c20 225 FUNC LOCAL DEFAULT 11 zm_info_pcre 4632: 00000000001f0d10 18 FUNC LOCAL DEFAULT 11 zm_shutdown_pcre 4633: 00000000001f0d30 469 FUNC LOCAL DEFAULT 11 zm_startup_pcre 4637: 00000000001f1260 104 FUNC LOCAL DEFAULT 11 zm_activate_pcre 4639: 00000000001f3010 814 FUNC LOCAL DEFAULT 11 php_do_pcre_match 4650: 0000000000fa11e0 352 OBJECT LOCAL DEFAULT 21 pcre_functions 8545: 0000000000fd8920 48 OBJECT LOCAL DEFAULT 21 arginfo_phpcredits 12505: 00000000007d3680 24 OBJECT LOCAL DEFAULT 13 php__pcre_utf8_table1 12620: 00000000001c9220 2257 FUNC LOCAL DEFAULT 11 php__pcre_xclass 12661: 00000000007d3500 56 OBJECT LOCAL DEFAULT 13 _pcre_ucp_typerange 12678: 00000000001f0350 454 FUNC LOCAL DEFAULT 11 _pcre_jit_exec 12830: 00000000007cf460 316 OBJECT LOCAL DEFAULT 13 _pcre_ucd_caseless_sets 12849: 00000000007d3678 4 OBJECT LOCAL DEFAULT 13 php__pcre_utf8_table1_siz 13495: 00000000001aba20 324 FUNC LOCAL DEFAULT 11 _pcre_find_bracket 13596: 00000000001f0730 13 FUNC LOCAL DEFAULT 11 _pcre_jit_get_size 13615: 00000000007d36c0 80 OBJECT LOCAL DEFAULT 13 _pcre_hspace_list 14148: 00000000007d2fe0 1287 OBJECT LOCAL DEFAULT 13 php__pcre_utt_names 14156: 00000000007d3600 64 OBJECT LOCAL DEFAULT 13 php__pcre_utf8_table4 14194: 00000000001ec5c0 15753 FUNC LOCAL DEFAULT 11 _pcre_jit_compile 14448: 00000000001f0740 8 FUNC LOCAL DEFAULT 11 _pcre_jit_get_target 14756: 00000000001c7350 631 FUNC LOCAL DEFAULT 11 php__pcre_was_newline 14816: 00000000001c75d0 140 FUNC LOCAL DEFAULT 11 _pcre_ord2utf 14828: 00000000007d3660 24 OBJECT LOCAL DEFAULT 13 php__pcre_utf8_table2 15177: 00000000007d3540 52 OBJECT LOCAL DEFAULT 13 _pcre_ucp_gbtable 15495: 000000000050e240 100 FUNC LOCAL DEFAULT 11 zif_phpcredits 15572: 00000000007cdde0 5760 OBJECT LOCAL DEFAULT 13 php__pcre_ucd_records 15665: 00000000007cbbe0 8704 OBJECT LOCAL DEFAULT 13 php__pcre_ucd_stage1 16005: 00000000007bd8e0 58112 OBJECT LOCAL DEFAULT 13 php__pcre_ucd_stage2 16032: 00000000007d3720 162 OBJECT LOCAL DEFAULT 13 php__pcre_OP_lengths 16141: 0000000000562400 849 FUNC LOCAL DEFAULT 11 php_zip_pcre 16335: 00000000007d2be0 1014 OBJECT LOCAL DEFAULT 13 php__pcre_utt 16395: 00000000001c8e60 936 FUNC LOCAL DEFAULT 11 _pcre_valid_utf 16487: 00000000001f06d0 96 FUNC LOCAL DEFAULT 11 _pcre_jit_free 16535: 00000000007d36a0 32 OBJECT LOCAL DEFAULT 13 _pcre_vspace_list 16536: 00000000007d2bc0 4 OBJECT LOCAL DEFAULT 13 php__pcre_utt_size 16542: 0000000001048900 168 OBJECT LOCAL DEFAULT 25 pcre_module_entry 16605: 00000000001c70f0 597 FUNC LOCAL DEFAULT 11 php__pcre_is_newline 16719: 00000000007bd4a0 1088 OBJECT LOCAL DEFAULT 13 php__pcre_default_tables 17145: 00000000007d3580 120 OBJECT LOCAL DEFAULT 13 php__pcre_ucp_gentype 17273: 00000000007d3640 24 OBJECT LOCAL DEFAULT 13 php__pcre_utf8_table3 18185: 00000000001c6720 303 FUNC GLOBAL DEFAULT 11 php_pcre_get_stringnumber 18320: 00000000001c6d40 103 FUNC GLOBAL DEFAULT 11 php_pcre_get_named_substr 18366: 00000000001b1a10 5514 FUNC GLOBAL DEFAULT 11 php_pcre_compile2 18424: 00000000001c6dc0 806 FUNC GLOBAL DEFAULT 11 php_pcre_maketables 18504: 00000000001f0870 36 FUNC GLOBAL DEFAULT 11 pcre_jit_stack_free 18633: 00000000001f56a0 2348 FUNC GLOBAL DEFAULT 11 php_pcre_split_impl 18722: 00000000001c7660 96 FUNC GLOBAL DEFAULT 11 php_pcre_refcount 18728: 00000000001c6ca0 9 FUNC GLOBAL DEFAULT 11 php_pcre_free_substring_l 18841: 00000000001c6bb0 233 FUNC GLOBAL DEFAULT 11 php_pcre_get_substring_li 19390: 00000000001f12d0 2639 FUNC GLOBAL DEFAULT 11 pcre_get_compiled_regex_c 19630: 00000000001c8ab0 939 FUNC GLOBAL DEFAULT 11 php_pcre_study 19746: 00000000001c6ad0 103 FUNC GLOBAL DEFAULT 11 php_pcre_copy_substring 19961: 00000000001f0520 421 FUNC GLOBAL DEFAULT 11 pcre_jit_exec 20358: 00000000001f1df0 4633 FUNC GLOBAL DEFAULT 11 php_pcre_match_impl 20446: 00000000001f08c0 137 FUNC GLOBAL DEFAULT 11 pcre_jit_free_unused_memo 20480: 00000000001f1d20 92 FUNC GLOBAL DEFAULT 11 pcre_get_compiled_regex 20521: 0000000001068bb8 8 OBJECT GLOBAL DEFAULT 26 pcre_stack_guard 20665: 00000000010488e0 8 OBJECT GLOBAL DEFAULT 25 php_pcre_malloc 20690: 00000000001f1d80 108 FUNC GLOBAL DEFAULT 11 pcre_get_compiled_regex_e 20807: 00000000010845ec 4 OBJECT GLOBAL DEFAULT 26 pcre_globals_id 21035: 00000000001f6260 1004 FUNC GLOBAL DEFAULT 11 php_pcre_grep_impl 21072: 00000000010488c8 8 OBJECT GLOBAL DEFAULT 25 php_pcre_stack_free 21272: 00000000001c6b40 109 FUNC GLOBAL DEFAULT 11 php_pcre_copy_named_subst 21304: 0000000001068bc0 8 OBJECT GLOBAL DEFAULT 26 php_pcre_callout 21364: 00000000001c6cb0 135 FUNC GLOBAL DEFAULT 11 php_pcre_get_substring 21445: 00000000001c4f50 5442 FUNC GLOBAL DEFAULT 11 php_pcre_exec 21479: 00000000001c6850 468 FUNC GLOBAL DEFAULT 11 php_pcre_get_stringtable_ 21503: 00000000001c9210 8 FUNC GLOBAL DEFAULT 11 php_pcre_version 21542: 00000000001f0750 283 FUNC GLOBAL DEFAULT 11 pcre_jit_stack_alloc 21621: 00000000010488d8 8 OBJECT GLOBAL DEFAULT 25 php_pcre_free 21646: 00000000010488d0 8 OBJECT GLOBAL DEFAULT 25 php_pcre_stack_malloc 21830: 00000000001c8a70 50 FUNC GLOBAL DEFAULT 11 pcre_free_study 21849: 00000000001b2fb0 177 FUNC GLOBAL DEFAULT 11 php_pcre_config 21855: 00000000001f4430 108 FUNC GLOBAL DEFAULT 11 php_pcre_replace 21983: 00000000001c64a0 633 FUNC GLOBAL DEFAULT 11 php_pcre_fullinfo 22287: 00000000001f08a0 29 FUNC GLOBAL DEFAULT 11 pcre_assign_jit_stack 22431: 00000000001b2fa0 16 FUNC GLOBAL DEFAULT 11 php_pcre_compile 22873: 00000000001c6db0 9 FUNC GLOBAL DEFAULT 11 php_pcre_free_substring 22898: 00000000001f3360 4297 FUNC GLOBAL DEFAULT 11 php_pcre_replace_impl

[dezgeg]

If someone wants to try that theory out, apply a patch to main/php_compat.h to add more of those
````

define pcre_compile php_pcre_compile

crud for all the rest of the buggy PCRE symbols:
489: 00000000001f0870 36 FUNC GLOBAL DEFAULT 11 pcre_jit_stack_free
1375: 00000000001f12d0 2639 FUNC GLOBAL DEFAULT 11 pcre_get_compiled_regex_c
1946: 00000000001f0520 421 FUNC GLOBAL DEFAULT 11 pcre_jit_exec
2431: 00000000001f08c0 137 FUNC GLOBAL DEFAULT 11 pcre_jit_free_unused_memo
2465: 00000000001f1d20 92 FUNC GLOBAL DEFAULT 11 pcre_get_compiled_regex
2506: 0000000001068bb8 8 OBJECT GLOBAL DEFAULT 26 pcre_stack_guard
2675: 00000000001f1d80 108 FUNC GLOBAL DEFAULT 11 pcre_get_compiled_regex_e
2792: 00000000010845ec 4 OBJECT GLOBAL DEFAULT 26 pcre_globals_id
3527: 00000000001f0750 283 FUNC GLOBAL DEFAULT 11 pcre_jit_stack_alloc
3815: 00000000001c8a70 50 FUNC GLOBAL DEFAULT 11 pcre_free_study
4273: 00000000001f08a0 29 FUNC GLOBAL DEFAULT 11 pcre_assign_jit_stack
18504: 00000000001f0870 36 FUNC GLOBAL DEFAULT 11 pcre_jit_stack_free
19390: 00000000001f12d0 2639 FUNC GLOBAL DEFAULT 11 pcre_get_compiled_regex_c
19961: 00000000001f0520 421 FUNC GLOBAL DEFAULT 11 pcre_jit_exec
20446: 00000000001f08c0 137 FUNC GLOBAL DEFAULT 11 pcre_jit_free_unused_memo
20480: 00000000001f1d20 92 FUNC GLOBAL DEFAULT 11 pcre_get_compiled_regex
20521: 0000000001068bb8 8 OBJECT GLOBAL DEFAULT 26 pcre_stack_guard
20690: 00000000001f1d80 108 FUNC GLOBAL DEFAULT 11 pcre_get_compiled_regex_e
20807: 00000000010845ec 4 OBJECT GLOBAL DEFAULT 26 pcre_globals_id
21542: 00000000001f0750 283 FUNC GLOBAL DEFAULT 11 pcre_jit_stack_alloc
21830: 00000000001c8a70 50 FUNC GLOBAL DEFAULT 11 pcre_free_study
22287: 00000000001f08a0 29 FUNC GLOBAL DEFAULT 11 pcre_assign_jit_stack
````

[srhb]

I'd love to hear arguments for each of these fixes. I think it's a bit annoying to do something very differently from the rest of the php community (that is, the "use nixpkgs pcre" solution) but on the other hand I think the bundle solution is pretty bad in general.

[vcunat]

I generally prefer to avoid bundling.

[vcunat]

Has some of you checked if they even noticed this problem upstream? I suppose not many will use bundling and at the same time have an incompatible version of libpcre available.

[srhb]

No, I just assumed they had since it's patched in php 7.2

[vcunat]

I suspect 7.2 only works because updating their libpcre to the same version as we have now.

[dezgeg]

https://github.com/NixOS/nixpkgs/issues/31451#issuecomment-343674821 indicates that indeed upstream has noticed.

[dezgeg]

But, I would also be surprised if other distributions like Fedora or Debian used the bundled pcre.