Homebrew-core: Formulae with opportunistic linkage issues

[ilovezfs]

During the jpeg 9 PR, these formulae ended up opportunistically linked:

(The pull requests indicate a fix, and the commits indicate just that a new bottle was pulled as a workaround.)

• [x] argyll-cms (libpng) (#23381)
• [x] fdroidserver (libyaml) (https://github.com/Homebrew/homebrew-core/commit/ade3afb42c480df19aaed5bdd18ec9cba8d255c5) (won't fix)
• [x] fizmo (freetype) (#23383)
• [x] fox (fontconfig) (#23382)
• [x] freeswitch (postgresql and xz) (https://github.com/Homebrew/homebrew-core/commit/dfa3b91435d7273042dc8fae3ddaa7410ea33c5b)
• [x] gegl (suite-sparse) (#23437)
• [x] ghostscript (fontconfig, libidn) (#16497)
• [x] graphicsmagick (libwmf) (#23930)
• [x] gst-plugins-bad (faad2) (https://github.com/Homebrew/homebrew-core/commit/261104401193e0eaa5c386be6ebd70cbc66d0488)
• [x] gst-plugins-ugly (lame) (https://github.com/Homebrew/homebrew-core/commit/f2075b3ce5fe7ffb0cb9761eba1a15fc67f87d47)
• [x] imagemagick (libwmf) (#16492)
• [x] imagemagick@6 (libwmf) (#23937)
• [x] imlib2 (libid3tag) (#23931)
• [x] io (theora) (https://github.com/Homebrew/homebrew-core/commit/19c588b0b32150fb9aee43915a139d33fada5d36)
• [ ] jbig2enc (giflib) (d659fbdfc81968fee97ed9716f8fa33d3f950436)
• [x] mldonkey (libmagic) (#23933)
• [x] nagios (libtool) (#23934)
• [x] open-scene-graph (openexr, ilmbase, liblas, sdl) (#23991)
• [x] podofo (libidn) (#23995)
• [ ] rlvm (guichan) (d7cf6b27f806d1bec2bbe7b15520d81156bfaed8)
• [x] sane-backends (libpng) (564f1542b23ddc87834942a0ef79a44eaea74224)
• [x] vice (mpg123) (3fdc8a47587f54ea3e686dd8f4f15736ed0f2726)
• [ ] volatility (gmp) (a49dc6cbc54249601ccf2c0f09b6483550afb04d)
• [x] webp (giflib) (#16501)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[JCount]

Go away @stale[bot], this is most definitely still an issue.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[DomT4]

Relax bot. Take another few weeks off.

[mwunsch]

Hello. I came across this issue from a failed attempt to install gst-plugins-bad. Is there a place to better understand what this issue is and what it means for something to be "opportunistically linked". Would love to better understand why these builds are failing.

[vinyldarkscratch]

@ilovezfs I'd be happy to help with this, but can you possibly explain what the term "opportunistically linked" means? (Thank you!)

[ilovezfs]

It means that one package (e.g. argyll-cms) ends up linked to the libraries of another package (e.g. libpng) even though there's no direct or recursive dependency of the former on the latter.

iMac-TMP:~ joe$ brew deps argyll-cms
jpeg
libtiff
iMac-TMP:~ joe$ brew install libpng
==> Downloading https://homebrew.bintray.com/bottles/libpng-1.6.34.el_capitan.bottle.tar.gz
Already downloaded: /Users/joe/Library/Caches/Homebrew/libpng-1.6.34.el_capitan.bottle.tar.gz
==> Pouring libpng-1.6.34.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/libpng/1.6.34: 26 files, 1.2MB
iMac-TMP:~ joe$ brew install -s argyll-cms
==> Installing dependencies for argyll-cms: jam, jpeg, libtiff
==> Installing argyll-cms dependency: jam
==> Downloading https://homebrew.bintray.com/bottles/jam-2.6.el_capitan.bottle.tar.gz
######################################################################## 100.0%
==> Pouring jam-2.6.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/jam/2.6: 4 files, 103.6KB
==> Installing argyll-cms dependency: jpeg
==> Downloading https://homebrew.bintray.com/bottles/jpeg-9b.el_capitan.bottle.tar.gz
Already downloaded: /Users/joe/Library/Caches/Homebrew/jpeg-9b.el_capitan.bottle.tar.gz
==> Pouring jpeg-9b.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/jpeg/9b: 20 files, 725.1KB
==> Installing argyll-cms dependency: libtiff
==> Downloading https://homebrew.bintray.com/bottles/libtiff-4.0.9_1.el_capitan.bottle.tar.gz
Already downloaded: /Users/joe/Library/Caches/Homebrew/libtiff-4.0.9_1.el_capitan.bottle.tar.gz
==> Pouring libtiff-4.0.9_1.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/libtiff/4.0.9_1: 246 files, 3.5MB
==> Installing argyll-cms
==> Downloading https://www.argyllcms.com/Argyll_V2.0.0_src.zip
######################################################################## 100.0%
==> sh makeall.sh
==> ./makeinstall.sh
🍺  /usr/local/Cellar/argyll-cms/2.0.0: 328 files, 115MB, built in 1 minute 3 seconds
iMac-TMP:~ joe$ brew linkage argyll-cms
System libraries:
  /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
  /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
  /System/Library/Frameworks/AudioToolbox.framework/Versions/A/AudioToolbox
  /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon
  /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa
  /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
  /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
  /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
  /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
  /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
  /usr/lib/libSystem.B.dylib
  /usr/lib/libobjc.A.dylib
Homebrew libraries:
  /usr/local/opt/jpeg/lib/libjpeg.9.dylib (jpeg)
  /usr/local/opt/libpng/lib/libpng16.16.dylib (libpng)
  /usr/local/opt/libtiff/lib/libtiff.5.dylib (libtiff)
Undeclared dependencies with linkage:
  libpng
iMac-TMP:~ joe$ 

[vinyldarkscratch]

Ah, I see — that’s what I thought, but I wasn’t 100% sure. I have some ideas on how to fix these, will be making this my day’s project. 😉

[ilovezfs]

Usually it just needs an explicit --without-foo passed to configure.

[vinyldarkscratch]

I'll be updating this comment as my own little to-do list, so that way everything can be kept track of!
(Legend: 💤 = not started, 🚫 = no solution found, ❌ = could not replicate, 🕐 = working on a solution, ✅ = solution finished and in a PR.)

✅ argyll-cms loads it's own version of libpng if it cannot find a pre-installed version. However, this seems to be the case with all of its dependencies (jpeg, libtiff, and even zlib). Unfortunately, there's no way to specify --without-libpng, which is the final piece of evidence that points towards it being required. #23381

🚫 fdroidserver also seems to be linking to little-cms2.

✅ fizmo actually requires freetype, but when it can't find fontconfig, it loads an old one bundled with X11. As @ilovezfs says, it's better to just require fontconfig. #23383

✅ fox has the same issue as fizmo. #23382

✅ freeswitch also seems to be linking to unixodbc. Both it and postgresql seem to be installing anyways, however. As such, I've decided to simply add these two as dependencies. (I didn't have the issue with xz, possibly patched already?) #23384

✅ gegl also seems to be linking to cairo and jasper. All of them have been added as optional dependencies, and will not link unless requested. #23437

:zzz: rlvm
:zzz: volatility

:zzz: gst-plugins-bad (doesn't have --without-XXX options)
:zzz: gst-plugins-ugly (doesn't have --without-XXX options)
:zzz: jbig2enc (doesn't have --without-XXX options)
:zzz: vice (doesn't have --without-XXX options)

[ilovezfs]

@vinyldarkscratch unfortunately the problem is primarily about preventing these from becoming corrupted during CI jobs that happen to involve the other formula, not about giving users an opt-out.

[vinyldarkscratch]

That's true -- I'm working on argyll-cms (and learning about what a Jamfile is in the process) at the moment, but I'll get back to fox and see if I can find a good solution for it?

[ilovezfs]

Actually, I'd just require fontconfig because when our fontconfig isn't installed, it just tries to use the random old fontconfig that is bundled with X11.

[vinyldarkscratch]

@ilovezfs Ah, that's definitely a good reason -- PR incoming. :wink:

[apjanke]

Ping: fizmo and gegl have been fixed; can be checked off.

[apjanke]

gst-plugins-bad picks up several formulae opportunistically, and I don't see configure controls for them. That'll be a larger job to fix.

$ brew install -s gst-plugins-bad
...
$ brew linkage gst-plugins-bad
...
Undeclared dependencies with linkage:
  cairo
  faad2
  glib
  gstreamer
  libpng
  pango

Similar with gst-plugins-ugly.

$ brew linkage gst-plugins-ugly
...
Undeclared dependencies with linkage:
  glib
  gstreamer
  lame

[ilovezfs]

I think we should ignore whatever doesn't have a --without-foo

[apjanke]

Fair enough. I'll skip over those.

[apjanke]

Remaining formulae:

gst-plugins-bad: has no --without-LIB option
gst-plugins-ugly: has no --without-LIB option
io: cmake
jbig2enc: has no --without-* switches for its lib dependencies.
open-scene-graph: cmake
podofo: cmake
rlvm: uses SCons; gonna have to research that.
vice: has no --without-mpg123 option
volatility: Python, has several linkages; postponing that one till I hear about fdroidserver

The CMake-based formulae require hacking the CMakelists file for each lib, the way open-scene-graph does for ffmpeg:

    # Turning off FFMPEG takes this change or a dozen "-DFFMPEG_" variables
    if build.without? "ffmpeg"
      inreplace "CMakeLists.txt", "FIND_PACKAGE(FFmpeg)", "#FIND_PACKAGE(FFmpeg)"
    end

Do you want fixes for those?

[ilovezfs]

The CMake-based formulae require hacking the CMakelists file

These can almost always be addressed with a CMake arg.

[apjanke]

You're right. -DCMAKE_DISABLE_FIND_PACKAGE_<packagename> does it; it was just a bit hard to find that in docs. Added PRs for io and open-scene-graph.

[ilovezfs]

Usually you can run cmake -LAH or cmake -LAH .. just after the cmake step in a formula to get a dump of most of the available variables, including undocumented ones.

[MikeMcQuaid]

Closing as @ilovezfs isn't around to maintain the list any more.