Nokogiri: Upgrade packaged libxml2 to 2.9.12

[flavorjones]

libxml 2.9.12

libxml 2.9.12 was released today (2021-05-13). Let's plan on upgrading to it for the next release.

• [x] update dependencies.yml
• [x] remove patches that are no longer needed, ensure remaining patches apply cleanly
• [x] get all the tests green, particularly under valgrind
• [x] look deeply at what CVEs are patched in this release, and whether they are valid within Nokogiri

(note that 2.9.11 was also released on 2021-05-13, but was superseded by 2.9.12 shortly thereafter)

What's in it?

From the release announcement:

This release incoporate [sic] the various security and quadatic [sic] patches that Nick has found
and fixed, and CVE-2021-3541. There is also a very large list of bug fixes and improvement[s] ...

Yeah, but what _else_ is in it?

See the full security analysis in the comments, below. Here's the summary ...

The following CVEs are patched in Nokogiri by the upgrade to libxml 2.9.12:

CVE-2019-20388

• Severity: Medium
• Type: Denial of service
• Description: A memory leak was found in the xmlSchemaValidateStream function of libxml2. Applications that use this library may be vulnerable to memory not being freed leading to a denial of service.

CVE-2020-24977

• Severity: Medium
• Type: Information disclosure
• Description: GNOME project libxml2 <= 2.9.10 has a global buffer over-read vulnerability in xmlEncodeEntitiesInternal at libxml2/entities.c.

CVE-2021-3517

• Severity: Medium
• Type: Arbitrary code execution
• Description: A heap-based buffer overflow was found in libxml2 before version 2.9.11 when processing truncated UTF-8 input.

CVE-2021-3518

• Severity: Medium
• Type: Arbitrary code execution
• Description: A use-after-free security issue was found in libxml2 before version 2.9.11 in xmlXIncludeDoProcess() in xinclude.c when processing crafted files.

CVE-2021-3537

• Severity: Low
• Type: Denial of service
• Description: It was found that libxml2 before version 2.9.11 did not propagate errors while parsing XML mixed content, causing a NULL dereference. If an untrusted XML document was parsed in recovery mode and post-validated, the flaw could be used to crash the application.

CVE-2021-3541

• Severity: Low
• Type: Denial of service
• Description: A security issue was found in libxml2 before version 2.9.11. Exponential entity expansion attack its possible bypassing all existing protection mechanisms and leading to denial of service.

Note that Nokogiri's default parse options prevent this attack from succeeding; but applications using DTDLOAD to load external DTDs may be vulnerable.

Other notes

Note that two additional CVEs were addressed upstream but are not relevant to this release. CVE-2021-3516 via xmllint is not present in Nokogiri, and CVE-2020-7595 has been patched in Nokogiri since v1.10.8 (see #1992).

[flavorjones]

Because a CVE is patched, I've tagged this as topic/security and we'll target a v1.11.x patch release.

[flavorjones]

Description updated with what I think is a canonical list of CVEs fixed in 2.9.11/2.9.12. Deeper analysis following shortly.

[flavorjones]

All information below is sourced from security.archlinux.org, which appears to have the most up-to-date information as of this analysis.

Please do not ask me why the libxml2 maintainers don't provide this information for downstream consumers, because I am a little cranky about it at the moment.

CVE-2019-20388

• Severity: Medium
• Type: Denial of service
• Description: A memory leak was found in the xmlSchemaValidateStream function of libxml2. Applications that use this library may be vulnerable to memory not being freed leading to a denial of service.
• Fixed: https://gitlab.gnome.org/GNOME/libxml2/commit/7ffcd44d7e6c46704f8af0321d9314cd26e0e18a

Verified that the fix commit first appears in v2.9.11. It seems possible that this issue would be present in programs using Nokogiri < v1.11.4.

CVE-2020-7595

• Severity: Medium
• Type: Denial of service
• Description: xmlStringLenDecodeEntities in parser.c in libxml2 2.9.10 has an infinite loop in a certain end-of-file situation.
• Fixed: https://gitlab.gnome.org/GNOME/libxml2/commit/0e1a49c8907645d2e155f0d89d4d9895ac5112b5

This has been patched in Nokogiri since v1.10.8 (see #1992).

CVE-2020-24977

• Severity: Medium
• Type: Information disclosure
• Description: GNOME project libxml2 <= 2.9.10 has a global buffer over-read vulnerability in xmlEncodeEntitiesInternal at libxml2/entities.c.
• Fixed: https://gitlab.gnome.org/GNOME/libxml2/commit/50f06b3efb638efb0abd95dc62dca05ae67882c2

Verified that the fix commit first appears in v2.9.11. It seems possible that this issue would be present in programs using Nokogiri < v1.11.4.

CVE-2021-3516

• Severity: Medium
• Type: Arbitrary code execution (no remote vector)
• Description: A use-after-free security issue was found libxml2 before version 2.9.11 when "xmllint --html --push" is used to process crafted files.
• Issue: https://gitlab.gnome.org/GNOME/libxml2/-/issues/230
• Fixed: https://gitlab.gnome.org/GNOME/libxml2/-/commit/1358d157d0bd83be1dfe356a69213df9fac0b539

Verified that the fix commit first appears in v2.9.11. This vector does not exist within Nokogiri, which does not ship xmllint.

CVE-2021-3517

• Severity: Medium
• Type: Arbitrary code execution
• Description: A heap-based buffer overflow was found in libxml2 before version 2.9.11 when processing truncated UTF-8 input.
• Issue: https://gitlab.gnome.org/GNOME/libxml2/-/issues/235
• Fixed: https://gitlab.gnome.org/GNOME/libxml2/-/commit/bf22713507fe1fc3a2c4b525cf0a88c2dc87a3a2

Verified that the fix commit first appears in v2.9.11. It seems possible that this issue would be present in programs using Nokogiri < v1.11.4.

CVE-2021-3518

• Severity: Medium
• Type: Arbitrary code execution
• Description: A use-after-free security issue was found in libxml2 before version 2.9.11 in xmlXIncludeDoProcess() in xinclude.c when processing crafted files.
• Issue: https://gitlab.gnome.org/GNOME/libxml2/-/issues/237
• Fixed: https://gitlab.gnome.org/GNOME/libxml2/-/commit/1098c30a040e72a4654968547f415be4e4c40fe7

Verified that the fix commit first appears in v2.9.11. It seems possible that this issue would be present in programs using Nokogiri < v1.11.4.

CVE-2021-3537

• Severity: Low
• Type: Denial of service
• Description: It was found that libxml2 before version 2.9.11 did not propagate errors while parsing XML mixed content, causing a NULL dereference. If an untrusted XML document was parsed in recovery mode and post-validated, the flaw could be used to crash the application.
• Issue: https://gitlab.gnome.org/GNOME/libxml2/-/issues/243
• Fixed: https://gitlab.gnome.org/GNOME/libxml2/-/commit/babe75030c7f64a37826bb3342317134568bef61

Verified that the fix commit first appears in v2.9.11. It seems possible that this issue would be present in programs using Nokogiri < v1.11.4.

CVE-2021-3541

• Severity: Low
• Type: Denial of service
• Description: A security issue was found in libxml2 before version 2.9.11. Exponential entity expansion attack its possible bypassing all existing protection mechanisms and leading to denial of service.
• Fixed: https://gitlab.gnome.org/GNOME/libxml2/-/commit/8598060bacada41a0eb09d95c97744ff4e428f8e

Verified that the fix commit first appears in v2.9.11. It seems possible that this issue would be present in programs using Nokogiri < v1.11.4, however Nokogiri's default parse options prevent the attack from succeeding (it is necessary to opt into DTDLOAD which is off by default).

Attack reproduction and validation of this assertion.

#! /usr/bin/env ruby

require 'nokogiri'

xml = <<EOF
<?xml version = "1.0" encoding = "UTF-8"?>
<!DOCTYPE address SYSTEM "foo.dtd" [
<!ENTITY % pe_1 "foo">
<!ENTITY % pe_a "&#37;pe_1;">
<!ENTITY % pe_2 "&#37;pe_1;,&#37;pe_1;,&#37;pe_1;,&#37;pe_1;,&#37;pe_1;,&#37;pe_1;,&#37;pe_1;,&#37;pe_1;,&#37;pe_1;,&#37;pe_1;,&#37;pe_1;">
<!ENTITY % pe_3 "&#37;pe_2;,&#37;pe_2;,&#37;pe_2;,&#37;pe_2;,&#37;pe_2;,&#37;pe_2;,&#37;pe_2;,&#37;pe_2;,&#37;pe_2;,&#37;pe_2;,&#37;pe_2;">
<!ENTITY % pe_4 "&#37;pe_3;,&#37;pe_3;,&#37;pe_3;,&#37;pe_3;,&#37;pe_3;,&#37;pe_3;,&#37;pe_3;,&#37;pe_3;,&#37;pe_3;,&#37;pe_3;,&#37;pe_3;">
<!ENTITY % pe_5 "&#37;pe_4;,&#37;pe_4;,&#37;pe_4;,&#37;pe_4;,&#37;pe_4;,&#37;pe_4;,&#37;pe_4;,&#37;pe_4;,&#37;pe_4;,&#37;pe_4;,&#37;pe_4;">
<!ENTITY % pe_6 "&#37;pe_5;,&#37;pe_5;,&#37;pe_5;,&#37;pe_5;,&#37;pe_5;,&#37;pe_5;,&#37;pe_5;,&#37;pe_5;,&#37;pe_5;,&#37;pe_5;,&#37;pe_5;">
<!ENTITY % pe_7 "&#37;pe_6;,&#37;pe_6;,&#37;pe_6;,&#37;pe_6;,&#37;pe_6;,&#37;pe_6;,&#37;pe_6;,&#37;pe_6;,&#37;pe_6;,&#37;pe_6;,&#37;pe_6;">
<!ENTITY % pe_8 "&#37;pe_7;,&#37;pe_7;,&#37;pe_7;,&#37;pe_7;,&#37;pe_7;,&#37;pe_7;,&#37;pe_7;,&#37;pe_7;,&#37;pe_7;,&#37;pe_7;,&#37;pe_7;">
<!ENTITY % pe_9 "&#37;pe_8;,&#37;pe_8;,&#37;pe_8;,&#37;pe_8;,&#37;pe_8;,&#37;pe_8;,&#37;pe_8;,&#37;pe_8;,&#37;pe_8;,&#37;pe_8;,&#37;pe_8;">
<!ENTITY % pe_10 "&#37;pe_9;,&#37;pe_9;,&#37;pe_9;,&#37;pe_9;,&#37;pe_9;,&#37;pe_9;,&#37;pe_9;,&#37;pe_9;,&#37;pe_9;,&#37;pe_9;,&#37;pe_9;">
]>
<address>
  <residence><foo></foo></residence>
</address>
EOF

Nokogiri::XML::Document.parse(xml) # default parse options DO NOT allow the attack to succeed
Nokogiri::XML::Document.parse(xml) { |c| c.dtdload } # this enables the attack

<!-- foo.dtd -->
<!ELEMENT address (residence)>
<!ELEMENT residence (%pe_10;)>
<!ELEMENT foo (#PCDATA)>

[flavorjones]

Summary of security analysis:

The following CVEs are patched in Nokogiri by the upgrade to libxml 2.9.12:

• CVE-2019-20388
• CVE-2020-24977
• CVE-2021-3517
• CVE-2021-3518
• CVE-2021-3537
• CVE-2021-3541

Note that two additional CVEs were addressed upstream but are not relevant to this release. CVE-2021-3516 via xmllint is not present in Nokogiri, and CVE-2020-7595 has been patched in Nokogiri since v1.10.8 (see #1992).

[flavorjones]

v1.11.4 has been released with libxml2 2.9.12. See https://github.com/sparklemotion/nokogiri/security/advisories/GHSA-7rrm-v45f-jp64 for more details on the security advisory created for it.

[Esaron]

The advisory field used by bundler-audit is coming across as blank and is preventing temporary ignores.

[flavorjones]

@Esaron I'm not sure what you're referring to; since https://github.com/rubysec/ruby-advisory-db/commit/febf9e8f404b4c8ad2860a33ca0f11e2c38e8c58 I am able to run bundler-audit 0.8.0 and get notified; and run bundle audit check --ignore GHSA-7rrm-v45f-jp64 to ignore it.

If you're having problems, please take it to the bundler-audit project and/or the ruby-advisory-db project.

[kashyapSheladiya]

Hi @flavorjones,
The advisory field doesn't shows CVE identifier number. So, how to ignore vulnerability using CVE. Of course bundle audit check --ignore GHSA-7rrm-v45f-jp64 this works. But, is there any possibilities to get CVE Identifier number for this vulnerability?

[Esaron]

I had forgotten I commented here, but the recent comment reminded me. In case it helps anyone, our problem was that we had a typo in our Gemfile and were referencing https://rubygems.org/gems/bundle-audit, which when I tried to update, seemed to not use any version of the real dependency greater than 0.6.X, which only had support for CVE ids. As for why that was happening, I have no clue. It was probably my fault. 0.8.0 has support for GHSA ids, and works fine.

[flavorjones]

@kashyapSheladiya There is no single identifier for this because there are several being addressed by libxml2; and I am not going to generate a CVE for Nokogiri to wrap other CVEs. The proper thing to do in this case is to publish an advisory with a unique identifier, which I have done.

[kashyapSheladiya]

@flavorjones Thanks for comment.