Linuxbrew-core: rubberband: error: Accelerate/Accelerate.h: No such file or directory

[binarykitchen]

Hello folks

Seeing compiling errors here when installing this: brew install ffmpeg --with-rubberband on Ubuntu 16.04:

$ brew install ffmpeg --with-rubberband                                                            
==> Installing dependencies for ffmpeg: rubberband
==> Installing ffmpeg dependency: rubberband
==> Downloading http://code.breakfastquay.com/attachments/download/34/rubberband-1.8.1.tar.bz2
Already downloaded: /home/michael-heuberger/.cache/Homebrew/rubberband-1.8.1.tar.bz2
==> Downloading https://raw.githubusercontent.com/Homebrew/formula-patches/1fd51a983/rubberband/rubberband-1.8.1-yo
Already downloaded: /home/michael-heuberger/.cache/Homebrew/rubberband--patch-7686dd9d05fddbcbdf4015071676ac37ecad5c7594cc06470440a18da17c71cd.diff
==> Patching
==> Applying rubberband-1.8.1-yosemite.diff
patching file src/dsp/FFT.cpp
patching file src/system/VectorOps.h
patching file src/system/sysutils.cpp
==> make -f Makefile.osx
Last 15 lines from /home/michael-heuberger/.cache/Homebrew/Logs/rubberband/01.make:
                 from src/StretcherImpl.h:29,
                 from src/StretcherProcess.cpp:24:
src/system/VectorOps.h:36:35: fatal error: Accelerate/Accelerate.h: No such file or directory
compilation terminated.
<builtin>: recipe for target 'src/audiocurves/SpectralDifferenceAudioCurve.o' failed
make: *** [src/audiocurves/SpectralDifferenceAudioCurve.o] Error 1
<builtin>: recipe for target 'src/StretcherProcess.o' failed
make: *** [src/StretcherProcess.o] Error 1
In file included from src/dsp/Window.h:32:0,
                 from src/StretcherImpl.h:29,
                 from src/RubberBandStretcher.cpp:24:
src/system/VectorOps.h:36:35: fatal error: Accelerate/Accelerate.h: No such file or directory
compilation terminated.
<builtin>: recipe for target 'src/RubberBandStretcher.o' failed
make: *** [src/RubberBandStretcher.o] Error 1

READ THIS: https://github.com/Linuxbrew/brew/blob/master/docs/Troubleshooting.md#troubleshooting
If reporting this issue please do so at (not Homebrew/brew):
  https://github.com/Linuxbrew/homebrew-core/issues

Also ran brew update and brew doctor. Doctor spit few warnings, probably irrelevant:

$ brew doctor

Warning: "config" scripts exist outside your system or Homebrew directories.
`./configure` scripts often look for *-config scripts to determine if
software packages are installed, and what additional flags to use when
compiling and linking.

Having additional scripts in your path can confuse software installed via
Homebrew if the config script overrides a system or Homebrew provided
script of the same name. We found the following "config" scripts:
    /usr/local/bin/powerline-config

Warning: You have a curlrc file
If you have trouble downloading packages with Homebrew, then maybe this
is the problem? If the following command doesn't work, then try removing
your curlrc:
  curl https://github.com

[maxim-belkin]

The error message is pretty clear:

fatal error: Accelerate/Accelerate.h: No such file or directory

Given that Accelerate.h is provided by OS X's Accelerate framework -- we have to investigate how to solve this problem on Linux.

[binarykitchen]

@maxim-belkin thanks, please do investigate. will probably be tricky i guess ...

[sjackman]

PR #1051 may fix your issue.

[binarykitchen]

@sjackman thanks - go go go, merge and release :)

[sjackman]

Hi, Michael. I resolved a number of issue, but I'm stuck at error: ladspa.h: No such file or directory. rubberband seems to depend on ladspa, which is not packaged for Linuxbrew. If you'd like to help troubleshoot, please comment at #1051. Is it possible to disable the ladspa dependency of rubberband?

[binarykitchen]

thanks for that but i am afraid i have no idea what ladspa is and can't answer that question

[binarykitchen]

@sjackman had a quick look here https://github.com/swh/ladspa and i think it should be easy to complile that on linux, see their readme

[sjackman]

Would you like to create a formula for LADSPA?
See https://github.com/Homebrew/brew/blob/master/docs/Formula-Cookbook.md
and start with brew create https://github.com/swh/ladspa/archive/v0.4.16.tar.gz

[binarykitchen]

hmmm i ll try over the weekend

[binarykitchen]

hmmm, i was wrong and it's not that easy. encountering compilation issues with ladspa and have reported issues there:
https://github.com/swh/ladspa/issues/36
https://github.com/swh/ladspa/issues/37
https://github.com/swh/ladspa/issues/38

[sjackman]

Can rubberband be configured to _not_ use LADSPA?

[binarykitchen]

not sure @sjackman

i think it's probably wiser if we ask the makers of rubberband for advice here:
@cannam would you mind to help us here for a minute?

• is ladspa a required dependency for compiling rubberband?
• do you think the latest brew formula for rubberband is looking good? is is anything missing? https://github.com/sjackman/homebrew-core/blob/7b94d4fc3a64f896e3d318a832fb97aa780a74fc/Formula/rubberband.rb

thanks :)

[cannam]

I think the basic problem is using the default Make target, which builds several different things:

• static libraries
• dynamic libraries
• command-line tool
• Vamp plugin
• LADSPA plugin

But other targets are available! (see section 4c in the Rubber Band README file)

The Homebrew formula that you linked appears to be installing the dynamic libraries and the command-line tool but nothing else, which means it doesn't need to build the rest. There isn't a Make target that builds exactly those, but it could run make twice (I guess? I know very little about Homebrew) for each of the "dynamic" and "program" targets. That would immediately eliminate the dependencies on LADSPA and Vamp, which are only needed for their respective plugins.

(The Rubber Band LADSPA plugin is an audio effects plugin that pitch-shifts audio using the Rubber Band library. It's quite useful to a certain niche. The Rubber Band Vamp plugin is an audio analysis plugin that simply reports what features the library finds in your audio. That one is of very little use to most people.)

But I think you have been misled about the nature of LADSPA, possibly by searching for it on Github and making (reasonable) deductions from that. LADSPA is a plugin format that defines a C API for audio effects plugins compiled into shared libraries, and plugin code only has to include a header file, not compile against any sort of SDK. The LADSPA dependency is literally a single header file, ladspa.h. While there's no harm in losing the Rubber Band LADSPA plugin if you think it's not relevant, it shouldn't need any other code to be compiled to support it. Steve Harris's LADSPA repository that you have been trying to build contains a set of LADSPA-compatible plugins, it isn't the definition of the LADSPA format itself. No other LADSPA code depends on Steve's plugins.

I notice the formula you linked appears to be using the OSX-specific Makefile and installing the .dylib file even if one isn't using OSX, which seems a bit odd to me. Rubber Band's "home" OS is Linux and the provided configure script is expected to work there, and of course Linux shared libraries are .so files rather than .dylib. But perhaps Linuxbrew is doing something I don't understand - I guess it's not supposed to interoperate with package-manager-installed packages or something? (To be honest it's a puzzle to me why Linuxbrew exists, when I thought Linux already had package managers that were nicer to use than Homebrew, so I should probably go and educate myself on that before asking about it.)

Anyway, hope this is helpful. Sorry it's so long, there turned out to be rather a lot of separate questions to answer there.

[cannam]

(I wrote "it's a puzzle to me why Linuxbrew exists... I should probably go and educate myself on that". I see at http://linuxbrew.sh/ "Installing a modern version of glibc and gcc in your home directory on an old distribution of Linux takes five minutes". That makes sense. I use Arch Linux on my development box, but I do have problems with new custom software running on old servers and I appreciate that. Installing a .dylib file still seems funny though?)

[sjackman]

The style of the formula is confusing. I apologize for that.
Lines 38 through 46 are for Linux
Lines 48 through 55 are for Mac

[sjackman]

To be honest it's a puzzle to me why Linuxbrew exists, when I thought Linux already had package managers that were nicer to use than Homebrew, so I should probably go and educate myself on that before asking about it.

See http://linuxbrew.sh Many users of high performance computing systems and shared servers do not have root access, and so cannot use the native package manager. Linuxbrew packages are often much more up to date than the versions available to the native package manager.

It can be installed in your home directory and does not require root access. The same package manager can be used on both your Linux server and your Mac laptop. Installing a modern version of glibc and gcc in your home directory on an old distribution of Linux takes five minutes.

[sjackman]

The LADSPA dependency is literally a single header file, ladspa.h

Would it be reasonable for rubberband to include ladspa.h in its tarball?

@binarykitchen If @cannam 's answer is no, then you need to add ladspa.h as a resource to the rubberband formula. See for example
https://github.com/Linuxbrew/homebrew-core/blob/master/Formula/openssl.rb#L19

[cannam]

Lines 38 through 46 are for Linux

Oh I quite missed the fact that there was a separate block with a return at the end. Sorry. (It is confusing though!)

[cannam]

Would it be reasonable for rubberband to include that file.

Not sure what you mean. The Rubber Band configure script just checks for the existence of the header, and ladspa/RubberBandPitchShifter.h includes it. What are you asking for?

[cannam]

Oh wait I see, you're asking for the ladspa.h header to be included in the package rather than #included in the source. Sorry, bit slow tonight.

(edit: and then you ninja edited anyway)

I just got told off in another issue for including third-party code in Rubber Band, so I'm reluctant to add more. And ladspa.h is pretty universal. Why not make a LADSPA formula that just has that one file in it? If you're packaging eccentric stuff like Rubber Band, you might as well get common stuff like ladspa.h first.

[sjackman]

Why not make a LADSPA formula that just has that one file in it? If you're packaging eccentric stuff like Rubber Band, you might as well get common stuff like ladspa.h first.

:+1: Sounds good to me. @binarykitchen Would you like to create a ladspa formula that does simply

def install
    include.install "ladspa.h"
end

@cannam What's the canonical versioned URL for ladspa.h?

[cannam]

What's the canonical versioned URL for ladspa.h?

I don't think there is one, though I might have missed a new development. I don't think there is even a canonical site for it since ladspa.org disappeared. Distro packages of course just contain this file directly, they don't need to get it from anywhere (presumably originally via hearsay or whatever).

[sjackman]

The distro packages have to get it from somewhere. They can't manifest it out of thin air. 😉

I'd recommend using this Debian cached version of the ladspa-sdk tarball, if we can't find a more appropriate source for the tarball.
http://http.debian.net/debian/pool/main/l/ladspa-sdk/ladspa-sdk_1.13.orig.tar.gz

[cannam]

The distro packages have to get it from somewhere. They can't manifest it out of thin air. :wink:

But it only needs to exist when the package is built, right? Not when it's installed. LADSPA hasn't changed in a decade and I guess nobody much has bothered to rebuild in that time either. (Don't listen to me though, I haven't been a distro packager in a very long time.) The fact that it hasn't changed in (practically) ever also means it might not matter much where you get your upstream from, so long as it's a trustworthy source.

[binarykitchen]

wow, lots of responses and dont understand everything. since you two know more about this matter, i think it would probably be wiser if one of you write the formula instead of me? otherwise i'd end up asking too many questions here ...

[cannam]

I think you should ask the questions, whatever they are, and go ahead.

[binarykitchen]

alright, first stumbling block here. we have a new ladspa.rb in the /Formula folder with following content:

class Ladspa < Formula
  desc "The SWH Plugins package for the LADSPA plugin system"
  homepage "http://plugin.org.uk/"
  url "https://github.com/swh/ladspa/archive/v0.4.16.tar.gz"
  version "0.4.16"
  sha256 "8accd2fcc932c9d35822d42e579308f7c19096af537729dc804b7bdc92de8c6b"

  depends_on "fftw"

  def install
    include.install "ladspa.h"
  end
end

then ran brew install -i ladspa for interactive mode and have following output:

$ brew install -i ladspa                                                                           
==> Downloading https://github.com/swh/ladspa/archive/v0.4.16.tar.gz
Already downloaded: /home/michael-heuberger/.cache/Homebrew/ladspa-0.4.16.tar.gz
==> Entering interactive mode
Type `exit' to return and finalize the installation
Install to this prefix: /home/michael-heuberger/.linuxbrew/Cellar/ladspa/0.4.16

what should i enter here @sjackman @cannam ?

when not interactive, it's just this:

$ brew install ladspa                                                                 
==> Downloading https://github.com/swh/ladspa/archive/v0.4.16.tar.gz
Already downloaded: /home/michael-heuberger/.cache/Homebrew/ladspa-0.4.16.tar.gz
==> ./configure
Last 15 lines from /home/michael-heuberger/.cache/Homebrew/Logs/ladspa/01.configure:
2016-10-17 20:02:53 +1300

./configure


READ THIS: https://github.com/Linuxbrew/brew/blob/master/docs/Troubleshooting.md#troubleshooting
If reporting this issue please do so at (not Homebrew/brew):
  https://github.com/Linuxbrew/homebrew-core/issues

[maxim-belkin]

https://github.com/drfill/homebrew-liquidsoap/blob/master/Formula/ladspa_header.rb

[maxim-belkin]

Formula for https://github.com/swh/ladspa should be probably called swh-ladspa

[maxim-belkin]

... or ladspa-swh... :)
And here is a good starting point for ladspa-swh. Just write a reasonable test block and create a PR to Homebrew-core.

class LadspaSwh < Formula
  desc "LADSPA-SWH plugins"
  homepage "http://plugin.org.uk"
  url "https://github.com/swh/ladspa/archive/v0.4.16.tar.gz"
  sha256 "8accd2fcc932c9d35822d42e579308f7c19096af537729dc804b7bdc92de8c6b"

  option "with-sse", "Uses SSE instructions" => :recommended
  option "with-rpath", "Hardcode runtime library paths" => :optional

  depends_on "autoconf" => :build
  depends_on "automake" => :build
  depends_on "libtool" => :build
  depends_on "gettext" => :build
  depends_on "pkg-config" => :build
  depends_on "fftw"
  depends_on "XML::Parser" => :perl
  depends_on "gcc" => :build if build.with? "sse"

  def install
    system "autoreconf", "-i"

    args = [
      "--prefix=#{prefix}",
      "--disable-dependency-tracking",
      "--disable-silent-rules",
      "--enable-darwin",
      "--enable-3dnow",
    ]
    args << "--enable-sse" if build.with? "sse"
    args << "--disable-rpath" if build.without? "rpath"

    system "./configure", *args

    system "make"
    system "make", "install"
  end

  test do
    # `test do` will create, run in and delete a temporary directory.
    #
    # This test will fail and we won't accept that! It's enough to just replace
    # "false" with the main program this formula installs, but it'd be nice if you
    # were more thorough. Run the test with `brew test ladspa`. Options passed
    # to `brew install` such as `--HEAD` also need to be provided to `brew test`.
    #
    # The installed folder is not in the path, so use the entire path to any
    # executables being tested: `system "#{bin}/program", "do", "something"`.
    system "false"
  end
end

[maxim-belkin]

Also, for a header file, it might worth saving it to a gist.

[binarykitchen]

thanks guys, gave the above a try but there is this error when running make:

libtool: link: /usr/bin/gcc-5 -shared  -fPIC -DPIC  .libs/hermes_filter_1200.o   -L/home/michael-heuberger/.linuxbrew/lib -Lutil -lblo -lrt -lm  -Os -march=native -Wl,--dynamic-linker=/home/michael-heuberger/.linuxbrew/lib/ld.so -Wl,-rpath -Wl,/home/michael-heuberger/.linuxbrew/lib   -Wl,-soname -Wl,hermes_filter_1200.so -o .libs/hermes_filter_1200.so
/usr/bin/ld: util/libblo.a(libblo_a-blo.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
util/libblo.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
Makefile:1061: recipe for target 'hermes_filter_1200.la' failed
make[2]: *** [hermes_filter_1200.la] Error 1
make[2]: *** Waiting for unfinished jobs....
...
libtool: link: ( cd ".libs" && rm -f "multivoice_chorus_1201.la" && ln -s "../multivoice_chorus_1201.la" "multivoice_chorus_1201.la" )
libtool: link: /usr/bin/gcc-5 -shared  -fPIC -DPIC  .libs/flanger_1191.o   -L/home/michael-heuberger/.linuxbrew/lib -lrt -lm  -Os -march=native -Wl,--dynamic-linker=/home/michael-heuberger/.linuxbrew/lib/ld.so -Wl,-rpath -Wl,/home/michael-heuberger/.linuxbrew/lib   -Wl,-soname -Wl,flanger_1191.so -o .libs/flanger_1191.so
libtool: link: ( cd ".libs" && rm -f "flanger_1191.la" && ln -s "../flanger_1191.la" "flanger_1191.la" )
libtool: link: /usr/bin/gcc-5 -shared  -fPIC -DPIC  .libs/decimator_1202.o   -L/home/michael-heuberger/.linuxbrew/lib -lrt -lm  -Os -march=native -Wl,--dynamic-linker=/home/michael-heuberger/.linuxbrew/lib/ld.so -Wl,-rpath -Wl,/home/michael-heuberger/.linuxbrew/lib   -Wl,-soname -Wl,decimator_1202.so -o .libs/decimator_1202.so
libtool: link: /usr/bin/gcc-5 -shared  -fPIC -DPIC  .libs/single_para_1203.o   -L/home/michael-heuberger/.linuxbrew/lib -lrt -lm  -Os -march=native -Wl,--dynamic-linker=/home/michael-heuberger/.linuxbrew/lib/ld.so -Wl,-rpath -Wl,/home/michael-heuberger/.linuxbrew/lib   -Wl,-soname -Wl,single_para_1203.so -o .libs/single_para_1203.so
libtool: link: /usr/bin/gcc-5 -shared  -fPIC -DPIC  .libs/triple_para_1204.o   -L/home/michael-heuberger/.linuxbrew/lib -lrt -lm  -Os -march=native -Wl,--dynamic-linker=/home/michael-heuberger/.linuxbrew/lib/ld.so -Wl,-rpath -Wl,/home/michael-heuberger/.linuxbrew/lib   -Wl,-soname -Wl,triple_para_1204.so -o .libs/triple_para_1204.so
libtool: link: ( cd ".libs" && rm -f "decimator_1202.la" && ln -s "../decimator_1202.la" "decimator_1202.la" )
libtool: link: ( cd ".libs" && rm -f "single_para_1203.la" && ln -s "../single_para_1203.la" "single_para_1203.la" )
libtool: link: ( cd ".libs" && rm -f "triple_para_1204.la" && ln -s "../triple_para_1204.la" "triple_para_1204.la" )
make[2]: Leaving directory '/tmp/ladspa-swh-20161018-21277-1r6szco/ladspa-0.4.16'
Makefile:1390: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/tmp/ladspa-swh-20161018-21277-1r6szco/ladspa-0.4.16'
Makefile:870: recipe for target 'all' failed
make: *** [all] Error 2

doesnt say much. any clues?

[binarykitchen]

also i see some warnings when compiling manually

autoheader: WARNING: Using auxiliary files such as `acconfig.h', `config.h.bot'
autoheader: WARNING: and `config.h.top', to define templates for `config.h.in'
autoheader: WARNING: is deprecated and discouraged.
autoheader: 
autoheader: WARNING: Using the third argument of `AC_DEFINE' and
autoheader: WARNING: `AC_DEFINE_UNQUOTED' allows one to define a template without
autoheader: WARNING: `acconfig.h':
autoheader: 
autoheader: WARNING:   AC_DEFINE([NEED_FUNC_MAIN], 1,
autoheader:         [Define if a function `main' is needed.])
autoheader: 
autoheader: WARNING: More sophisticated templates can also be produced, see the
autoheader: WARNING: documentation.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/\${ <-- HERE ([^ \t=:+{}]+)}/ at /home/michael-heuberger/.linuxbrew/bin/automake line 3936.
configure.ac:3: installing './config.guess'
configure.ac:3: installing './config.sub'
configure.ac:6: installing './missing'
Makefile.am:94: warning: '%'-style pattern rules are a GNU make extension

[binarykitchen]

never mind, solved this with ENV.append_to_cflags "-fPIC" unless OS.mac? ... stay tuned

[binarykitchen]

compilation worked. but now i dont know what to write for tests. any clues?

[maxim-belkin]

You can do something like:

assert File.exist?("some_file.ext"), "Does not exist!"
assert_match "/some/library.so", shell_output("ldd tested_library").chomp
assert_equal "Hello World!", shell_output("./executable-if-any").chomp

The last line is just for reference as I don't remember seeing executables in ladspa-swh when I installed it with the above formula. Also not sure why yours needed -fPIC.

[maxim-belkin]

Also, I'd suggest submitting a PR to Homebrew first. Once that one goes through -- Shaun will merge it and make necessary changes for Linuxbrew.

[sjackman]

Does LADSPA build on Mac?

LADSPA is an acronym for Linux Audio Developer's Simple Plugin API

[binarykitchen]

@maxim-belkin thanks for that - will give this a try later on this week (pretty busy right now)

@sjackman can't answer that, don't have a Mac. Ony Linux here.

[cannam]

@sjackman yes - at least there is nothing actually Linux-specific about the spec and header. Individual plugins may vary, but mostly I think they're portable to Mac too. That's a point though, surely LADSPA stuff would be in Homebrew already?

[maxim-belkin]

I created a gist for ladspa.h.
So, now we can create ladspa resource.

[maxim-belkin]

If @sjackman or @rwhogg approve merging of #1111, one could then do depends_on "ladspa-header".

[sjackman]

I'm fine with either a LADSPA resource in rubberband or a formula. I'd like to know how many formula depend on this LADSPA header. If it's only rubberband, then a resource is the way to go.

[binarykitchen]

i am a bit lost here - what should i do with the ladspa header formula mentioned in https://github.com/Linuxbrew/homebrew-core/pull/1111 ?

[retokromer]

By chance, do you know when approximately the Accelerate framework has been added for macOS? With an older version of rubberband this worked fine on Ubuntu 14.04 LTS, yet now on 16.04 LTS I got the same error.

[maxim-belkin]

@binarykitchen : you should use url in that formula for a resource:

resource "ladspa-header" do
   url "http://..."
   sha256 ""
end

and then in the main formula something like: include.install resource("ladspa-header")

[binarykitchen]

@maxim-belkin huh, i dont understand. already have added an url, see my PR at https://github.com/Linuxbrew/homebrew-core/pull/1171/files#diff-f715dd30ac1534d023d8a0d30ae57951R4

can you comment on that PR and suggest changes there directly? ta

[binarykitchen]

ping - can anyone help me here? really need the ability to install ffmpeg with rubberband on linux ...

[sjackman]

Add to rubberband the following unttested code. It may need work.

resource "ladspa" do
    url "https://gist.githubusercontent.com/maxim-belkin/721fafe404e78e52cf3bf29a259ea9d2/raw/c50f363ec478e23e0bd5b28ea705b5ae52523abb/ladspa.h"
    sha256 "0c73624df742727076c65a9fec4c2806d37bcb7245f62e973d070048b6d333af"
end

def install
    include.install resource("ladspa")
    # more code goes here
end

[binarykitchen]

Sorry but I'm stopping here, found another way around to install ffmpeg with rubberband.

[sjackman]

No worries. Glad to hear that you found a workaround.
@maxim-belkin Do you want to pick up this issue?

[maxim-belkin]

ok, will do