Linuxbrew-core: libprelude: ruby not found

[carlorosati]

Last 15 lines from /home/carlorosati/.cache/Homebrew/Logs/libprelude/01.configure:
    class RegexFlag(enum.IntFlag):
AttributeError: module 'enum' has no attribute 'IntFlag'
checking Python.h usability... no
checking Python.h presence... no
checking for Python.h... no
checking for ruby... /usr/bin/ruby
checking for ruby version... 1.8.7
checking for ruby platform... x86_64-linux
checking for ruby script directory... $(prefix)/lib64/ruby/1.8/x86_64-linux
checking for ruby extension module directory... $(prefix)/lib64/ruby/site_ruby/1.8/x86_64-linux
checking for RUBY_PKG_CONFIG... no
checking ruby.h usability... no
checking ruby.h presence... no
checking for ruby.h... no
configure: error: Could not find ruby

[carlorosati]

There is a line depends_on :ruby => "1.8" in the libprelude.rb . I think this has come up before... Why isn't the dependency being installed?

[sjackman]

Your host likely has ruby installed but not the development package. Try brew install libruby-dev
or alternatively use the brewed ruby with brew install ruby libprelude
The :ruby requirement probably checks for the Ruby executable, but should also check for the headers and shared library.

[carlorosati]

How could we make it so this doesn't error on hosts without the development package installed?

[sjackman]

Does the workaround brew install ruby libprelude work for you?

[carlorosati]

Oh yeah, I already knew that. I just opened this issue, because I thought it was intended for ruby to be installed if libprelude needed it.

[sjackman]

The :ruby requirement needs a fix for that.
See https://github.com/Linuxbrew/brew/blob/master/Library/Homebrew/requirements/ruby_requirement.rb#L11
Alternatively we could make :ruby a hard dependency on Linuxbrew on the brewed "ruby". Only 14 formulae have a dependency on :ruby.

[carlorosati]

Do we want a separate development ruby formula and development requirement?

[sjackman]

In Linuxbrew we'll continue to do what the upstream Homebrew do, which is a single requirement. I prefer not to diverge too far from Homebrew unless there's a strong argument for it.

[carlorosati]

I'm not sure I understand.
So would the options be one of the following?

• require brewed ruby -- meaning installed from source
• require system ruby to be devel -- requiring modification of ruby_requirement.rb

[sjackman]

Yes, the three possible options are:

• Modify libprelude.rb to add depends_on "ruby" unless OS.mac? (required brewed Ruby)
• Modify ruby_requirement.rb to require system ruby to be devel or else install brewed Ruby
• Modify ruby_requirement.rb to always require a brewed Ruby on Linuxbrew (don't use the host Ruby)

[carlorosati]

I would prefer to

Modify ruby_requirement.rb to require system ruby to be devel or else install brewed Ruby

My plan is the following.
if pkg-config installed, check ruby --cpaths
else check /usr/include/{ruby-version}/

If ruby.h not found, install brewed ruby.

The issues are that

• Mac buries ruby.h inside of XCode.app
• Each *nix distro installs ruby somewhere different.
  e.g.
  Centos 6:/usr/local/x64/ruby-1.8.7-p334/lib/ruby/1.8/x86_64-linux/ruby.h
  Debian Wheezy:/usr/lib/ruby/1.8/x86_64-linux/ruby.h

@sjackman Suggestions?

[sjackman]

Try

❯❯❯ /usr/bin/ruby -e 'puts RbConfig::CONFIG["rubyhdrdir"]'
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/include/ruby-2.0.0
❯❯❯ /usr/local/bin/ruby -e 'puts RbConfig::CONFIG["rubyhdrdir"]'
/usr/local/Cellar/ruby/2.4.0/include/ruby-2.4.0

[randy3k]

what is the status of the issue? It seems that in rstudio-server formula https://github.com/Homebrew/homebrew-science/pull/5442

it failed while installing libprelude.

checking for Python.h... yes
checking for ruby... /usr/bin/ruby
checking for ruby version... 1.9.3
checking for ruby platform... x86_64-linux
checking for ruby script directory... $(prefix)/lib/ruby/1.9.1/x86_64-linux
checking for ruby extension module directory... $(prefix)/local/lib/site_ruby/1.9.1/x86_64-linux
checking for RUBY_PKG_CONFIG... no
checking ruby.h usability... no
checking ruby.h presence... no
checking for ruby.h... no
configure: error: Could not find ruby

[carlorosati]

Can you provide brew config and brew list. This should have been fixed.

[randy3k]

The log was copied from the Travis build for the formula, not from my local machine.

[carlorosati]

Can you provide the config.log for the libprelude on travis?

[randy3k]

I can actually reproduce the same error on my local machine.

Before installing libprelude

HOMEBREW_VERSION: 1.1.11-7-g6db39d7
ORIGIN: https://github.com/Linuxbrew/brew
HEAD: 6db39d7080dfb5dd5449f39f254d7972099bf9f7
Last commit: 11 hours ago
Core tap ORIGIN: https://github.com/Linuxbrew/homebrew-core
Core tap HEAD: 2db0e34bb146e0d5a655f9f44cecc6bfff1f5fd5
Core tap last commit: 56 minutes ago
HOMEBREW_PREFIX: /home/randy/.linuxbrew
HOMEBREW_REPOSITORY: /home/randy/.linuxbrew
HOMEBREW_CELLAR: /home/randy/.linuxbrew/Cellar
HOMEBREW_BOTTLE_DOMAIN: https://linuxbrew.bintray.com
CPU: single-core 64-bit ivybridge
Homebrew Ruby: 2.3.1 => /usr/bin/ruby2.3
Clang: N/A
Git: 2.7.4 => /usr/bin/git
Perl: /usr/bin/perl
Python: /home/randy/miniconda3/bin/python => /home/randy/miniconda3/bin/python3.5
Ruby: /usr/bin/ruby => /usr/bin/ruby2.3
Java: 1.8.0_112
Kernel: Linux 4.4.0-64-generic x86_64 GNU/Linux
OS: Ubuntu 16.04 LTS
Codename: xenial
OS glibc: 2.23
OS gcc: 5.4.0
Linuxbrew glibc: N/A
Linuxbrew gcc: 5.3.0
Linuxbrew xorg: 20170115
HOMEBREW_VERSION: 1.1.11-7-g6db39d7
ORIGIN: https://github.com/Linuxbrew/brew
HEAD: 6db39d7080dfb5dd5449f39f254d7972099bf9f7
Last commit: 11 hours ago
Core tap ORIGIN: https://github.com/Linuxbrew/homebrew-core
Core tap HEAD: 2db0e34bb146e0d5a655f9f44cecc6bfff1f5fd5
Core tap last commit: 56 minutes ago
HOMEBREW_PREFIX: /home/randy/.linuxbrew
HOMEBREW_REPOSITORY: /home/randy/.linuxbrew
HOMEBREW_CELLAR: /home/randy/.linuxbrew/Cellar
HOMEBREW_BOTTLE_DOMAIN: https://linuxbrew.bintray.com
CPU: single-core 64-bit ivybridge
Homebrew Ruby: 2.3.1 => /usr/bin/ruby2.3
Clang: N/A
Git: 2.7.4 => /usr/bin/git
Perl: /usr/bin/perl
Python: /home/randy/miniconda3/bin/python => /home/randy/miniconda3/bin/python3.5
Ruby: /usr/bin/ruby => /usr/bin/ruby2.3
Java: 1.8.0_112
Kernel: Linux 4.4.0-64-generic x86_64 GNU/Linux
OS: Ubuntu 16.04 LTS
Codename: xenial
OS glibc: 2.23
OS gcc: 5.4.0
Linuxbrew glibc: N/A
Linuxbrew gcc: 5.3.0
Linuxbrew xorg: 20170115

(randyvb)-~$ /usr/bin/ruby2.3 -e 'puts RbConfig::CONFIG["rubyhdrdir"]'
/usr/include/ruby-2.3.0
(randyvb)-~$ ls /usr/include/ruby-2.3.0
ruby  ruby.h
(randyvb)-~$ brew install libprelude
==> Installing dependencies for libprelude: ruby
==> Installing libprelude dependency: ruby
....
....

Although ruby.h exists, linuxbrew is still trying to get a brewed ruby?

Alright, let's proceed with the brewed ruby. Somehow, linuxbrew built the ruby from source rather than downloaded a bottled ruby. And after the successful installation of ruby, the same old error

checking for ruby... /usr/bin/ruby
checking for ruby version... 2.3.1
checking for ruby platform... x86_64-linux-gnu
checking for ruby script directory... $(prefix)/lib/x86_64-linux-gnu/ruby/2.3.0
checking for ruby extension module directory... $(prefix)/local/lib/x86_64-linux-gnu/site_ruby
checking for RUBY_PKG_CONFIG... no
checking ruby.h usability... no
checking ruby.h presence... no
checking for ruby.h... no
configure: error: Could not find ruby

It seems it didn't pick up the newly installed ruby.

The complete log: https://gist.github.com/c53204ba38435f04b973583e1227a027

[randy3k]

I guess there are two issues here.

1. https://github.com/Linuxbrew/brew/pull/269 failed to detect the system ruby is devel.
2. the formula of libprelude did not pick up the the correct version of ruby.

[sjackman]

Although ruby.h exists, linuxbrew is still trying to get a brewed ruby?

When installing a formula from a bottle, Homebrew also installs all of its requirements from a bottle, even if the requirement is satisfied by a host dependency. Linuxbrew inherits that behaviour from Homebrew.

Alright, let's proceed with the brewed ruby. Somehow, linuxbrew built the ruby from source rather than downloaded a bottled ruby. And after the successful installation of ruby, the same old error

The precompiled binary bottles of non-relocatable bottles can only be used if you install in /home/linuxbrew/.linuxbrew, otherwise they have to be built from source. See the documentation below, but wherever it reads /usr/local, which is for macOS, read instead /home/linuxbrew/.linuxbrew.

• http://docs.brew.sh/Installation.html#alternative-installs
• http://docs.brew.sh/FAQ.html#why-do-you-compile-everything
• http://docs.brew.sh/Bottles.html#cellar-cellar

Consider installing Linuxbrew in /home/linuxbrew/.linuxbrew/ if possible so that you can use precompiled binary packages (known as bottles) for non-relocatable formula like ruby.

It seems it didn't pick up the newly installed ruby.

It's possible that's a bug related to brew install --env=super, which is now the default. Try…

brew install --env=std libprelude

[sjackman]

I'm attempting to reproduce this build error at https://github.com/Linuxbrew/homebrew-core/pull/2396
I don't know why libprelude has no bottles for macOS.

[randy3k]

When installing a formula from a bottle, Homebrew also installs all of its requirements from a bottle, even if the requirement is satisfied by a host dependency. Linuxbrew inherits that behaviour from Homebrew.

I was installing libprelude from source. If I recall correctly, for the case of Homebrew,

  depends_on :python

will only install python if it is not found on the host.

[randy3k]

By the way, brew install --env=std libprelude works!

[sjackman]

I was installing libprelude from source.

Then it should use your host ruby if the requirement is satisfied. It's a bug if not.

If I recall correctly, for the case of Homebrew,
depends_on :python
will only install python if it is not found on the host.

My understanding is that if you're installing foo that depends on :python from source, then it'll use the host Python, and if you're installing a bottle of foo, then it will not use the host Python.

[randy3k]

Then it should use your host ruby if the requirement is satisfied. It's a bug if not.

Then it must be a bug.

(randyvb)-~$ type -a ruby
ruby is /home/randy/.linuxbrew/bin/ruby
ruby is /usr/bin/ruby
(randyvb)-~$ /usr/bin/ruby --version
ruby 2.3.1p112 (2016-04-26) [x86_64-linux-gnu]

[sjackman]

Okay. Thanks for the bug report, Randy. I won't have the time myself to look into the issue. A PR is welcome if you have the time and inclination to look into it yourself.

[carlorosati]

The issue was not that it wasn't finding ruby.h but that it wasn't finding config.h

-I/usr/lib/x86_64-linux-gnu/ruby/2.3.0 -I/usr/include/ruby-2.3.0 -I/usr/include/ruby-2.3.0/x86_64-linux-gnu conftest.c >&5
In file included from /usr/include/ruby-2.3.0/ruby.h:33:0,
                 from conftest.c:99:
/usr/include/ruby-2.3.0/ruby/ruby.h:24:25: fatal error: ruby/config.h: No such file or directory
compilation terminated.
configure:21984: $? = 1

[randy3k]

It is because linuxbrew picked up the system ruby rather than the brewed ruby. It should not use the system ruby in the first place. @sjackman's suggestion

brew install --env=std libprelude

does fix this issue.

[sjackman]

Both of the files /usr/include/ruby-2.3.0/ruby.h and /usr/include/x86_64-linux-gnu/ruby-2.3.0/ruby/config.h are provided on Ubuntu 16 Xenial by the package ruby2.3-dev. It's perhaps relevant that they appear to be in different directories.

I was able to successfully build a bottle for libprelude using brew install --env=super libprelude on CircleCI at #2396.

[randy3k]

Actually what are the difference between superenv and stdenv? I want to understand why it happened.

EDIT: https://github.com/Homebrew/brew/blob/081461a268f02aafa0753b6391e5129c929259c9/docs/Formula-Cookbook.md#superenv-notes

[randy3k]

After I have removed everything and started a fresh new linuxbrew installation, brew install libprelude without --env=std works now.

[sjackman]

I don't unfortunately see any user-oriented documentation describing --env=super. The best I found is this: http://www.rubydoc.info/github/Homebrew/brew/Superenv

The gist of it is that it insert a shell script named cc into the PATH to intercept calls to the compiler. It then inserts and removes compiler options as necessary.

Why superenv?

1. Only specify the environment we need (NO LDFLAGS for cmake)
2. Only apply compiler specific options when we are calling that compiler
3. Force all incpaths and libpaths into the cc instantiation (less bugs)
4. Cater toolchain usage to specific Xcode versions
5. Remove flags that we don't want or that will break builds
6. Simpler code
7. Simpler formula that just work
8. Build-system agnostic configuration of the tool-chain