Linguist: Installing dependencies (and other tasks) not working on macOS Mojave

[nwhetsell]

I can’t seem to get the steps for contributing to Linguist to work on macOS Mojave 10.14.3.

First, I have CMake, ICU4C, and pkg-config installed through Homebrew.

Running script/bootstrap results in

script/bootstrap: line 7: bundle: command not found

I then ran sudo gem install bundler successfully, but running script/bootstrap then resulted in:

You are replacing the current local value of path, which is currently nil
Fetching gem metadata from https://rubygems.org/.........
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies...
Bundler could not find compatible versions for gem "bundler":
  In Gemfile:
    bundler (~> 1.10)

  Current Bundler version:
    bundler (2.0.1)
This Gemfile requires a different version of Bundler.
Perhaps you need to update Bundler by running `gem install bundler`?

Could not find gem 'bundler (~> 1.10)' in any of the relevant sources:
  the local ruby installation

I then ran:

sudo gem uninstall bundler
sudo gem install bundler -v 1.10

But running script/bootstrap then resulted in:

Fetching gem metadata from https://rubygems.org/..........
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
Resolving dependencies...
Installing rake 12.3.2
Installing public_suffix 3.0.3
Installing addressable 2.5.2
Using bundler 1.10.0
with native extensions Installing byebug 11.0.0
with native extensions 
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /Users/nathanwhetsell/Documents/linguist/vendor/gems/ruby/2.3.0/gems/charlock_holmes-0.7.6/ext/charlock_holmes
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/bin/ruby -r ./siteconf20190306-87297-m6l1ib.rb extconf.rb
checking for main() in -licui18n... no
checking for main() in -licui18n... yes
checking for unicode/ucnv.h... yes
checking for main() in -lz... yes
checking for main() in -licuuc... yes
checking for main() in -licudata... yes
creating Makefile

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /Users/nathanwhetsell/Documents/linguist/vendor/gems/ruby/2.3.0/extensions/universal-darwin-18/2.3.0/charlock_holmes-0.7.6/mkmf.log

current directory: /Users/nathanwhetsell/Documents/linguist/vendor/gems/ruby/2.3.0/gems/charlock_holmes-0.7.6/ext/charlock_holmes
make "DESTDIR=" clean

current directory: /Users/nathanwhetsell/Documents/linguist/vendor/gems/ruby/2.3.0/gems/charlock_holmes-0.7.6/ext/charlock_holmes
make "DESTDIR="
compiling converter.c
compiling encoding_detector.c
compiling ext.c
compiling transliterator.cpp
In file included from transliterator.cpp:1:
In file included from ./common.h:9:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/include/ruby-2.3.0/ruby.h:33:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/include/ruby-2.3.0/ruby/ruby.h:24:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/include/ruby-2.3.0/universal-darwin18/ruby/config.h:365:31: error: invalid suffix on literal; C++11 requires a space between literal and identifier [-Wreserved-user-defined-literal]
#define RUBY_ARCH "universal-"RUBY_PLATFORM_OS
                              ^

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/include/ruby-2.3.0/universal-darwin18/ruby/config.h:366:35: error: invalid suffix on literal; C++11 requires a space between literal and identifier [-Wreserved-user-defined-literal]
#define RUBY_PLATFORM "universal."RUBY_PLATFORM_CPU"-"RUBY_PLATFORM_OS
                                  ^

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/include/ruby-2.3.0/universal-darwin18/ruby/config.h:366:55: error: invalid suffix on literal; C++11 requires a space between literal and identifier [-Wreserved-user-defined-literal]
#define RUBY_PLATFORM "universal."RUBY_PLATFORM_CPU"-"RUBY_PLATFORM_OS
                                                      ^

In file included from transliterator.cpp:1:
In file included from ./common.h:9:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/include/ruby-2.3.0/ruby.h:33:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/include/ruby-2.3.0/ruby/ruby.h:1988:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/include/ruby-2.3.0/ruby/intern.h:47:19: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
void rb_mem_clear(register VALUE*, register long);
                  ^~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/include/ruby-2.3.0/ruby/intern.h:47:36: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
void rb_mem_clear(register VALUE*, register long);
                                   ^~~~~~~~~
2 warnings and 3 errors generated.
make: *** [transliterator.o] Error 1

make failed, exit code 2

Gem files will remain installed in /Users/nathanwhetsell/Documents/linguist/vendor/gems/ruby/2.3.0/gems/charlock_holmes-0.7.6 for inspection.
Results logged to /Users/nathanwhetsell/Documents/linguist/vendor/gems/ruby/2.3.0/extensions/universal-darwin-18/2.3.0/charlock_holmes-0.7.6/gem_make.out
An error occurred while installing charlock_holmes (0.7.6), and Bundler cannot
continue.
Make sure that `gem install charlock_holmes -v '0.7.6'` succeeds before
bundling.

At this point I decided to give up on script/bootstrap and see if I could add a grammar anyway.

I ran brew install docker to install Docker. Running script/add-grammar resulted in:

Checking docker is installed and running
$ docker ps
  > Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Command failed. Aborting.

I eventually stumbled on https://stackoverflow.com/questions/40523307/brew-install-docker-does-not-include-docker-engine and tried:

brew uninstall docker
brew cask install docker

After starting the Docker app (by double-clicking it), script/add-grammar eventually resulted in errors from script/licensed (certainly because of missing dependencies):

  > /Library/Ruby/Gems/2.3.0/gems/bundler-1.10.0/lib/bundler/resolver.rb:341:in `block in verify_gemfile_dependencies_are_found!': Could not find gem 'minitest (>= 5.0) ruby' in any of the gem sources listed in your Gemfile or available on this machine. (Bundler::GemNotFound)
  >     from /Library/Ruby/Gems/2.3.0/gems/bundler-1.10.0/lib/bundler/resolver.rb:319:in `each'
  >     from /Library/Ruby/Gems/2.3.0/gems/bundler-1.10.0/lib/bundler/resolver.rb:319:in `verify_gemfile_dependencies_are_found!'
  >     from /Library/Ruby/Gems/2.3.0/gems/bundler-1.10.0/lib/bundler/resolver.rb:198:in `start'
  >     from /Library/Ruby/Gems/2.3.0/gems/bundler-1.10.0/lib/bundler/resolver.rb:182:in `resolve'
  >     from /Library/Ruby/Gems/2.3.0/gems/bundler-1.10.0/lib/bundler/definition.rb:199:in `resolve'
  >     from /Library/Ruby/Gems/2.3.0/gems/bundler-1.10.0/lib/bundler/definition.rb:139:in `specs'
  >     from /Library/Ruby/Gems/2.3.0/gems/bundler-1.10.0/lib/bundler/definition.rb:184:in `specs_for'
  >     from /Library/Ruby/Gems/2.3.0/gems/bundler-1.10.0/lib/bundler/definition.rb:173:in `requested_specs'
  >     from /Library/Ruby/Gems/2.3.0/gems/bundler-1.10.0/lib/bundler/environment.rb:18:in `requested_specs'
  >     from /Library/Ruby/Gems/2.3.0/gems/bundler-1.10.0/lib/bundler/runtime.rb:13:in `setup'
  >     from /Library/Ruby/Gems/2.3.0/gems/bundler-1.10.0/lib/bundler.rb:126:in `setup'
  >     from /Library/Ruby/Gems/2.3.0/gems/bundler-1.10.0/lib/bundler/setup.rb:18:in `<top (required)>'
  >     from /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:127:in `require'
  >     from /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:127:in `rescue in require'
  >     from /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:39:in `require'
  >     from script/licensed:3:in `<main>'
Command failed. Aborting.

[lildude]

Whoops. Looks like we need to document that bundler, and specifically v1.x is required.

As for the currently failure, ultimately, this looks to be a problem with charlock_holmes which is outside of the control of Linguist 😞 . I wish I could say there's an easy solution to this, but I've not found it yet. I normally end up trawling through the issues at https://github.com/brianmario/charlock_holmes looking for similar errors and trying the suggestions there.

Looking at my bundler config, the last time I installed charlock_holmes I ended up running

bundle config build.charlock_holmes --with-icu-dir /usr/local/opt/icu4c --with-cxxflags=-std=c++11

... first and then installed charlock_holmes using gem install charlock_holmes -v '0.7.6'. That might do the trick here.

[nwhetsell]

Thanks for the tips. Alas, running

bundle config build.charlock_holmes --with-icu-dir /usr/local/opt/icu4c --with-cxxflags=-std=c++11
sudo gem install charlock_holmes -v '0.7.6'

fails with the same error.

[nwhetsell]

I found the magic words:

sudo gem install charlock_holmes --version 0.7.6 -- --with-icu-dir=/usr/local/opt/icu4c --with-cxxflags=\'-Wno-reserved-user-defined-literal -std=c++11\'

[lildude]

Another set of magic flags to remember for one gem 😞 . Good work.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in a long time. If this issue is still relevant and should remain open, please reply with a short explanation (e.g. "I have checked the code and this issue is still relevant because ___."). Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had activity in a long time. Please feel free to reopen it or create a new issue.

[kunalbhatia18]

I found the magic words:

sudo gem install charlock_holmes --version 0.7.6 -- --with-icu-dir=/usr/local/opt/icu4c --with-cxxflags=\'-Wno-reserved-user-defined-literal -std=c++11\'

Thanks man this really helped me out