Homebrew-core: Running msgfmt to build Java2 resources crashes with specific string lengths of msgctxt and msgid

[p3k]

Please note we will close your issue without comment if you delete, do not read or do not fill out the issue checklist below and provide ALL the requested information. If you repeatedly fail to use the issue template, we will block you from ever submitting issues to Homebrew again.

• [x] are reporting a bug others will be able to reproduce and not asking a question or requesting software. If you're not sure or want to ask a question do so on our Discourse: https://discourse.brew.sh. To get software added or changed in Homebrew please file a Pull Request
• [ ] have a problem with brew install (or upgrade, reinstall) a single, Homebrew/homebrew-core formula (not cask) on macOS? If it's a general brew problem please file this issue at Homebrew/brew: https://github.com/Homebrew/brew/issues/new/choose. If it's a Linux problem please file this issue at https://github.com/Homebrew/linuxbrew-core/issues/new/choose. If it's a brew cask problem please file this issue at https://github.com/Homebrew/homebrew-cask/issues/new/choose. If it's a tap (e.g. Homebrew/homebrew-php) problem please file this issue at the tap.
• [x] ran brew update and can still reproduce the problem?
• [x] ran brew doctor, fixed all issues and can still reproduce the problem?
• [ ] ran brew gist-logs <formula> (where <formula> is the name of the formula that failed) and included the output link?
• [ ] if brew gist-logs didn't work: ran brew config and brew doctor and included their output with your issue?

To help us debug your issue please explain:

• What you were trying to do (and why)
  Using msgfmt to create Java message resource from the following de.po file to provide localized message strings to Java application.

  # SOME DESCRIPTIVE TITLE.
  # Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
  # This file is distributed under the same license as the PACKAGE package.
  # FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
  #
  msgid ""
  msgstr ""
  "MIME-Version: 1.0\n"
  "Content-Type: text/plain; charset=UTF-8\n"
  "Content-Transfer-Encoding: 8bit\n"
  "Plural-Forms: nplurals=2; plural=(n != 1);\n"
  
  msgctxt "123456789"
  msgid "abcdef"
  msgstr "whatever"
  

• What happened (include command output)
  msgfmt crashed.

  # msgfmt --java2 --source -d/tmp --resource=foo.bar.Messages --locale=de de.po
  Abort trap: 6
  

• What you expected to happen
  Creation of the file /tmp/foo/bar/Messages_de.java with the following contents:

  /* Automatically generated by GNU msgfmt.  Do not modify!  */
  package foo.bar;
  public class Messages_de extends java.util.ResourceBundle {
  private static final java.lang.String[] table;
  static {
    java.lang.String[] t = new java.lang.String[4];
    t[0] = "";
    t[1] = "MIME-Version: 1.0\nContent-Type: text/plain; charset=UTF-8\nContent-Transfer-Encoding: 8bit\nPlural-Forms: nplurals=2; plural=(n != 1);\n";
    t[2] = "1a\u000423456789012345";
    t[3] = "whatever";
    table = t;
  }
  public java.lang.Object handleGetObject (java.lang.String msgid) throws java.util.MissingResourceException {
    int hash_val = msgid.hashCode() & 0x7fffffff;
    int idx = (hash_val % 2) << 1;
    java.lang.Object found = table[idx];
    if (found != null && msgid.equals(found))
      return table[idx + 1];
    return null;
  }
  public java.util.Enumeration getKeys () {
    return
      new java.util.Enumeration() {
        private int idx = 0;
        { while (idx < 4 && table[idx] == null) idx += 2; }
        public boolean hasMoreElements () {
          return (idx < 4);
        }
        public java.lang.Object nextElement () {
          java.lang.Object key = table[idx];
          do idx += 2; while (idx < 4 && table[idx] == null);
          return key;
        }
      };
  }
  public java.util.ResourceBundle getParent () {
    return parent;
  }
  }
  

The problem seems to be related to the combined string lenghts of the msgctxt and msgid values.

For example, the following values crash msgfmt, too (msgstr can be arbitrary):

msgctxt "four"
msgid "sixplusfive"

msgctxt "context"
msgid "can hurt"

It looks like if the sum of both values’ lengths equals 15, msgfmt is going to crash – slightly modified values prevent the crash and create the Java message file accordingly:

msgctxt "123456789x"
msgid "abcdef"

msgctxt "123456789"
msgid "abcdef_"

msgctxt "four"
msgid "six-plus-five"

msgctxt "context"
msgid "is hard"

What could cause this behaviour? And how could it sneak into the macos binary only – not in the Linux one (assuming the same source code is used for both platforms)?

---

The issue could be verified on two different macos installations, one of them with the following specs:

# sw_vers
ProductName:    Mac OS X
ProductVersion: 10.14.3
BuildVersion:   18D109

# msgfmt -V
msgfmt (GNU gettext-tools) 0.19.8.1
Copyright (C) 1995-1998, 2000-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Ulrich Drepper.

# brew -v
Homebrew 2.1.0
Homebrew/homebrew-core (git revision 4ab838; last commit 2019-04-09)
Homebrew/homebrew-cask (git revision 5371d; last commit 2019-04-10)

Please let me know if additional information is required.

[p3k]

Also see ticket for MacPorts binary.

[ryandesign]

I filed a bug report with the gettext developers.

[fxcoudert]

Thanks @p3k for the report and @ryandesign for reporting it upstream, so they can fix it and all newer versions benefit from the fix.

[p3k]

@fxcoudert when will the upgraded msgfmt binary be available via brew update? does it need a PR for the formula?

[equal-l2]

@p3k The upstream seems to be preparing for a new release, which contains the fix.
Once released, the new release will be available on Homebrew soon.