Homebrew-core: gnu-units: UTF-8 warnings on invocation

Created on 5 Dec 2017 · 3Comments · Source: Homebrew/homebrew-core

The gunits command, from gnu-units, throws a bunch of UTF-8 errors when invoked. The relevant configuration lines appear to be currency units in the units definition file, /usr/local/Cellar/gnu-units/2.16/share/units/definitions.units specifically:

sed -ne '6401p; 6403p; 6404p; 6415,6421p;' /usr/local/Cellar/gnu-units/2.16/share/units/definitions.units
₺                       turkeylira
₹                       indiarupee      # official rupee sign
؋                       afghanafghani
₲                       paraguayguarani
₴                       ukrainehryvnia
₵                       ghanacedi
₸                       kazakhstantenge
₼                       azerbaijanmanat
₽                       russiaruble
₾                       georgialari

I'm suspecting either bad defs or missing UTF- 8 support somewhere here (or both, interacting).

I've commented the Unicode currency symbols definitions (following submission of this bug) which avoids the warnings. That's lines 6394 - 6427 or so of the file.

Please note we will close your issue without comment if you delete, do not read or do not fill out the issue checklist below and provide ALL the requested information. If you repeatedly fail to use the issue template, we will block you from ever submitting issues to Homebrew again.

Please always follow these steps:

[x] Confirmed this is a problem with brew installing one, specific Homebrew/homebrew-core formula (not cask or tap) and not every time you run brew? If it's a general brew problem please file this issue at https://github.com/Homebrew/brew/issues/new. If it's a brew cask problem please file this issue at https://github.com/caskroom/homebrew-cask/issues/new. If it's a tap (e.g. Homebrew/homebrew-php) problem please file this issue at the tap.

The problem exists only for gnu-units that I've been able to determine.

[x] Ran brew update and retried your prior step?

Yep.

[x] Ran brew doctor, fixed all issues and retried your prior step?

To the best of my ability. Note that Xcode requires an OS upgrade to update, so I've not upgraded Xcode.

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry and just ignore them. Thanks!

Warning: Unbrewed dylibs were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected dylibs:
  /usr/local/lib/libtcl8.6.dylib
  /usr/local/lib/libtk8.6.dylib

Warning: Unbrewed header files were found in /usr/local/include.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected header files:
  /usr/local/include/fakemysql.h
  /usr/local/include/fakepq.h
  /usr/local/include/fakesql.h
  /usr/local/include/itcl.h
  /usr/local/include/itcl2TclOO.h
  /usr/local/include/itclDecls.h
  /usr/local/include/itclInt.h
  /usr/local/include/itclIntDecls.h
  /usr/local/include/itclMigrate2TclCore.h
  /usr/local/include/itclTclIntStubsFcn.h
  /usr/local/include/mysqlStubs.h
  /usr/local/include/odbcStubs.h
  /usr/local/include/pqStubs.h
  /usr/local/include/tcl.h
  /usr/local/include/tclDecls.h
  /usr/local/include/tclOO.h
  /usr/local/include/tclOODecls.h
  /usr/local/include/tclPlatDecls.h
  /usr/local/include/tclThread.h
  /usr/local/include/tclTomMath.h
  /usr/local/include/tclTomMathDecls.h
  /usr/local/include/tdbc.h
  /usr/local/include/tdbcDecls.h
  /usr/local/include/tdbcInt.h
  /usr/local/include/tk.h
  /usr/local/include/tkDecls.h
  /usr/local/include/tkPlatDecls.h

Warning: Unbrewed .pc files were found in /usr/local/lib/pkgconfig.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected .pc files:
  /usr/local/lib/pkgconfig/tcl.pc
  /usr/local/lib/pkgconfig/tk.pc

Warning: Unbrewed static libraries were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected static libraries:
  /usr/local/lib/libtclstub8.6.a
  /usr/local/lib/libtkstub8.6.a

Warning: Homebrew's sbin was not found in your PATH but you have installed
formulae that put executables in /usr/local/sbin.
Consider setting the PATH for example like so
  echo 'export PATH="/usr/local/sbin:$PATH"' >> ~/.bash_profile

Warning: Your Xcode (8.0) is outdated.
Please update to Xcode 8.2.1 (or delete it).
Xcode can be updated from the App Store.


Warning: Broken symlinks were found. Remove them with `brew prune`:
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/HiraKakuPro-W3
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/HiraKakuPro-W6
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/HiraKakuProN-W3
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/HiraKakuProN-W6
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/HiraKakuStd-W8
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/HiraKakuStdN-W8
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/HiraMaruPro-W4
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/HiraMaruProN-W4
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/HiraMinPro-W3
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/HiraMinPro-W6
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/HiraMinProN-W3
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/HiraMinProN-W6
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/HiraginoSansGB-W3
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/HiraginoSansGB-W6
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/YuMin-Demibold
  /usr/local/share/ghostscript/9.16/Resource/CIDFont/YuMin-Medium

[x] Ran brew gist-logs <formula> (where <formula> is the name of the formula that failed) and included the output link?

$ brew gist-logs gnu-units
Error: No logs.

[x] If brew gist-logs didn't work: ran brew config and brew doctor and included their output with your issue?

Doctor output above.

$ brew config
HOMEBREW_VERSION: 1.3.9
ORIGIN: https://github.com/Homebrew/brew.git
HEAD: 4f5e938a5dbe049e05dea19495b6547acc5c5b5b
Last commit: 27 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 9f0ac390e2af1603c970feca42acecddade7f8cf
Core tap last commit: 3 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_NO_ANALYTICS: 1
HOMEBREW_VISUAL: vim
CPU: quad-core 64-bit skylake
Homebrew Ruby: 2.3.3 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3/bin/ruby
Clang: 8.0 build 800
Git: 2.15.1 => /usr/local/bin/git
Curl: 7.43.0 => /usr/bin/curl
Perl: /usr/bin/perl
Python: /usr/local/opt/python/libexec/bin/python => /usr/local/Cellar/python/2.7.14/Frameworks/Python.framework/Versions/2.7/bin/python2.7
Ruby: /usr/bin/ruby => /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby
Java: N/A
macOS: 10.11.6-x86_64
Xcode: 8.0
CLT: 8.2.0.0.1.1480973914
X11: 2.7.11 => /opt/X11
$

To help us debug your issue please explain:

What you were trying to do (and why)

Running gnu-units (gunits)

What happened (include command output)

$ gunits
Currency exchange rates from finance.yahoo.com on 2017-10-31
gunits: invalid/nonprinting UTF-8 on line 6401 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6403 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6404 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6415 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6416 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6417 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6418 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6419 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6420 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6421 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
3035 units, 109 prefixes, 109 nonlinear units

You have:

What you expected to happen

Approximately zero "invalid/nonprinting UTF-8" messages.

Step-by-step reproduction instructions (by running brew install commands)

$ brew update; brew update; brew reinstall gnu-units; echo '' | gunits
Already up-to-date.
Already up-to-date.
==> Reinstalling gnu-units
==> Downloading https://homebrew.bintray.com/bottles/gnu-units-2.16.el_capitan.b
Already downloaded: /Users/karsten/Library/Caches/Homebrew/gnu-units-2.16.el_capitan.bottle.tar.gz
==> Pouring gnu-units-2.16.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/gnu-units/2.16: 13 files, 720.4KB
Currency exchange rates from finance.yahoo.com on 2017-10-31
gunits: invalid/nonprinting UTF-8 on line 6401 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6403 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6404 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6415 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6416 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6417 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6418 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6419 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6420 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
gunits: invalid/nonprinting UTF-8 on line 6421 of '/usr/local/Cellar/gnu-units/2.16/share/units/definitions.units'
3035 units, 109 prefixes, 109 nonlinear units

You have: You have:
$

Formula additions or changes

To get formulae added or changed in Homebrew please file a Pull Request
We will close issues requesting formulae changes.

Source

dredmorbius

Most helpful comment

After chasing it, it turns out to be an Apple bug, which I reported as radar 36044615:

$ cat a.c
#include <wchar.h>
#include <locale.h>
#include <stdio.h>

int main(void) {
  setlocale(LC_CTYPE, "en_GB.UTF-8");
  wchar_t s1[3] = { 0x20ba, 0x20, 0 };
  wchar_t s2[3] = { 0x20ac, 0x20, 0 };
  printf("%d %ls\n", wcswidth(s1, 2), s1);
  printf("%d %ls\n", wcswidth(s2, 2), s2);

  printf("%d\n", wcwidth(s1[0]));
  printf("%d\n", wcwidth(s2[0]));
}
$ clang a.c && ./a.out
-1 ₺ 
2 € 
-1
1

fxcoudert on 14 Dec 2017

👍2

All 3 comments

I can confirm that this occurs on macOS 10.13 with the following locale:

LANG="fr_FR.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL="en_US.UTF-8"

The bug comes from upstream. The file in question (definitions.units) appears to me to be valid UTF-8, and the characters that cause trouble (TURKISH LIRA SIGN, INDIA RUPEE SIGN, AFGHANI SIGN, GUARANI SIGN, HRYVNIA SIGN, CEDI SIGN, TENGE SIGN, MANAT SIGN, RUBLE SIGN, and LARI SIGN) are definitely not nonprinting.

I've sent this report by email to the maintainer ([email protected]), as they do not have an open bug-reporting system.

fxcoudert on 13 Dec 2017

👍1

After chasing it, it turns out to be an Apple bug, which I reported as radar 36044615:

$ cat a.c
#include <wchar.h>
#include <locale.h>
#include <stdio.h>

int main(void) {
  setlocale(LC_CTYPE, "en_GB.UTF-8");
  wchar_t s1[3] = { 0x20ba, 0x20, 0 };
  wchar_t s2[3] = { 0x20ac, 0x20, 0 };
  printf("%d %ls\n", wcswidth(s1, 2), s1);
  printf("%d %ls\n", wcswidth(s2, 2), s2);

  printf("%d\n", wcwidth(s1[0]));
  printf("%d\n", wcwidth(s2[0]));
}
$ clang a.c && ./a.out
-1 ₺ 
2 € 
-1
1

fxcoudert on 14 Dec 2017

👍2

@fxcoudert Thanks.

In the meantime, would commenting that block be an appropriate workaround, or are we just waiting on Apple?

dredmorbius on 21 Dec 2017

Was this page helpful?

0 / 5 - 0 ratings