Vcpkg: Error bootstrapping vcpkg using gcc 8 from PPA on ubuntu trusty: undefined reference to iostream_category

[mcraveiro]

Describe the bug

Bootstrapping vcpkg results in a link error. I am unable to build vcpkg on with GCC 8 from the toolchain PPA [1].

Environment

• OS: Linux Ubuntu Trusty 14.04.6 (Image used by Travis)
• Compiler: GCC 8.1.0

To Reproduce

Steps to reproduce the behavior:

1. Run OS image in Virtual Box
2. Add PPA and Install GCC 8
3. Bootstrap vcpkg.

Expected behavior

vcpkg to build correctly

Failure logs

Could not find a way to attach the build log. The key error is:

[0/2] Re-checking globbed directories...
[65/65] Linking CXX executable vcpkg
FAILED: vcpkg 
: && /usr/bin/g++-9  -O3 -DNDEBUG   CMakeFiles/vcpkglib.dir/src/vcpkg/archives.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/base/checks.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/base/chrono.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/base/cofffilereader.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/base/downloads.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/base/enums.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/base/files.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/base/hash.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/base/machinetype.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/base/strings.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/base/stringview.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/base/system.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/base/system.print.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/binaryparagraph.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/build.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.autocomplete.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.buildexternal.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.cache.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.ci.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.contact.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.create.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.dependinfo.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.edit.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.env.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.exportifw.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.import.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.integrate.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.list.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.owns.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.portsdiff.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.search.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.upgrade.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.version.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/commands.xvsinstances.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/dependencies.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/export.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/globalstate.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/help.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/input.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/install.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/logicexpression.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/metrics.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/packagespec.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/packagespecparseresult.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/paragraphparseresult.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/paragraphs.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/parse.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/postbuildlint.buildtype.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/postbuildlint.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/remove.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/sourceparagraph.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/statusparagraph.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/statusparagraphs.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/tools.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/triplet.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/update.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/userconfig.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/vcpkgcmdarguments.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/vcpkglib.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/vcpkgpaths.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/versiont.cpp.o CMakeFiles/vcpkglib.dir/src/vcpkg/visualstudio.cpp.o CMakeFiles/vcpkg.dir/src/vcpkg.cpp.o  -o vcpkg  -lstdc++fs -pthread && :
CMakeFiles/vcpkglib.dir/src/vcpkg/base/hash.cpp.o: In function `vcpkg::Hash::get_file_hash(vcpkg::Files::Filesystem const&, std::experimental::filesystem::v1::path const&, vcpkg::Hash::Algorithm, std::error_code&)':
hash.cpp:(.text+0xb9d9): undefined reference to `std::iostream_category()'
hash.cpp:(.text+0xba11): undefined reference to `std::iostream_category()'
hash.cpp:(.text+0xba51): undefined reference to `std::iostream_category()'

Additional context

I have updated vcpkg to latest on my personal fork [0], and built it locally using Debian testing. It all worked just fine. I then went to my Ubuntu Trusty VM to update the exports for my travis build. It uses the PPA [1] to get the latest GCCs. When trying to build with both GCC 8 and 9 I got the above error. As per linking error, the dependency against iostream_category is pulled in here [2].

After some investigation it appears that this error is to do with the way libstdc++ is built in the PPA. On my local PC I get:

$ cd /usr/lib/gcc/x86_64-linux-gnu/8
$ grep iostream_category libstdc++*
Binary file libstdc++.a matches
Binary file libstdc++.so matches

However, on the Ubuntu Trusty machine I get:

$ cd /usr/lib/gcc/x86_64-linux-gnu/8
$ grep iostream_category libstdc++*
$

It appears that for some reason the libstdc++ bundled with the PPA is not including iostream_category. I had a look at the file where this is implemented and - to the untrained eye - found no evidence of conditional compilation [3].

My questions are as follows:

• google does not locate any questions related to undefined references to iostream_category, even though the PPA is commonly used by Travis users to provide toolchains. Is vcpkg doing something different in its usage of iostreams that could explain this?
• is there anyway in which I can bypass this problem?

Many thanks for your time.

[0] https://github.com/MASD-Project/vcpkg
[1] https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test
[2] https://github.com/microsoft/vcpkg/blob/master/toolsrc/src/vcpkg/base/hash.cpp#L736
[3] https://github.com/gcc-mirror/gcc/blob/f2b443b321f8a6d4a9dc82d81761046e5fe19870/libstdc%2B%2B-v3/src/c%2B%2B11/cxx11-ios_failure.cc#L84

[mcraveiro]

I just noticed that the owner of the toolchain PPA is on GitHub (Matthias Klose). @doko42, apologies for the spam but if you could shed some light on this (or point to the right forum) it would be greatly appreciated. Many thanks.

[mcraveiro]

The plot thickens: I manually downloaded the debs from the PPA [1] on my main machine and now I can see the missing symbols. This is really strange. I think my Ubuntu VM has gone crazy somehow. I will uninstall and reinstall GCC from the PPA and see if it solves the problem...

[1] https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test/+packages

[mcraveiro]

The previous comment was erroneous: I had extracted a symlink from the package.

So, after some more investigation, I am now fully doubting my sanity. At any rate, I believe the following to be true:

1. The libstdc++ binary is provided by a regular Ubuntu Trusty package. It creates a SO located under /usr/lib/x86_64-linux-gnu/libstdc++.so.6.
2. The PPA provides packages for GCC and the development versions of libstdc++ - e.g. libstdc++-8-dev. These packages install binaries for libstdc++ under /usr/lib/gcc/x86_64-linux-gnu/8. However, the file libstdc++.so is a symlink back up to /usr/lib/x86_64-linux-gnu.

This leads me to the rather weird conclusion that the PPA is shipping headers and static libraries, but relying on the distro for the libstdc++ binary. This seems _rather_ strange, because one would assume that the distro is compiling using the standard compiler at the time (say GCC 4.8 for Trusty). At this point I am thoroughly confused and not sure how anything is working at all at present :-) I'm sure I must be missing something rather obvious, so I will resume this investigation later with a fresh pair of eyes. Meanwhile, if you have any ideas, they would be most appreciated.

[mcraveiro]

Right, I still haven't got to the bottom of how SO's are handled in the PPA, but meanwhile found another useful test. I downloaded both the 14.04 package [1] and the 16.04 [2] package and checked the static libraries supplied (libstdc++.a). For 14.04 we have nothing:

$ strings libstdc++.a | grep iostream_category
$ 

Whereas 16.04 looks much healthier:

$ strings libstdc++.a | grep iostream_category
_ZSt17iostream_categoryv
_ZSt17iostream_categoryv
.rela.text._ZSt17iostream_categoryv

Very puzzling.

[1] https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test/+build/17225595/+files/libstdc++-8-dev_8.3.0-16ubuntu3~14.04.2_amd64.deb
[2] https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test/+build/17222316/+files/libstdc++-8-dev_8.3.0-16ubuntu3~16.04_amd64.deb

[mcraveiro]

I have looked at the build logs of both 16.04 [1] and 14.04 [2] builds of gcc in the PPA, and could not find anything obvious. I can see the ios_failure file being compiled in both cases. However, to be fair, there is a lot of output in these logs and I may have missed something obvious.

[1] https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test/+build/17222316/+files/buildlog_ubuntu-xenial-amd64.gcc-8_8.3.0-16ubuntu3~16.04_BUILDING.txt.gz
[2] https://launchpadlibrarian.net/431903365/buildlog_ubuntu-trusty-amd64.gcc-8_8.3.0-16ubuntu3~14.04.2_BUILDING.txt.gz

[mcraveiro]

Actually, this looks like a really tricky problem so I'm going to chicken out and take the easy way out: I'm going to upgrade my Travis builds to use 16.04 (Xenial) instead of 14.04 (Trusty), as it seems its now available on Travis [1]. I'll close this issue once I successfully built vcpkg on 16.04.

[1] https://docs.travis-ci.com/user/reference/xenial/

[mcraveiro]

I have successfully build vcpkg using the Ubuntu Toolchain PPA for Xenial (16.04). To summarise this issue for anyone that bumps into this problem: it seems you cannot build vcpkg using the Ubuntu Toolchain PPA for 14.04 (Trusty) and GCC 8 or 9. You will likely bump into the link error above. However, the compilation works fine for Ubuntu Toolchain PPA for Xenial (16.04). If, like me, you were using 14.04 due to Travis, you can uppgrade to 16.04 [1].

[1] https://travis-ci.community/t/ubuntu-18-04-1-lts-bionic-beaver/1270

[amaiorano]

Thank you for this thread, @mcraveiro. I switched to Xenial as well, and everything works again :)

[nickbopp]

@mcraveiro @amaiorano In both these cases, I think the problem was that you (or Travis) are not doing an 'apt upgrade' after adding the PPA for the newer gcc compilers. There is a single shared package for libstdc++6 (the runtime .so) that's already installed as the xenial version. When you add the repository and 'apt update', it sees that there is a newer version of that package provided by the PPA (which will match libstdc++-9-dev headers/static lib). But you need to actually force an update with the package manager to pull down that newer SO.

[zivmc]

Here is a work around to compile vcpkg on Linux Ubuntu Trusty 14.04.

The code cuased “undefined reference to std::iostream_category()” locates in /usr/include/c++/7.5.0/bits/ios_base.h, similar file path for gcc-8 and gcc-9.

Precisely, code around line 200 of bits/ios_base.h caused the link error:

#if __cplusplus >= 201103L
  /// I/O error code
  enum class io_errc { stream = 1 };

  template <> struct is_error_code_enum<io_errc> : public true_type { };

  const error_category& iostream_category() noexcept;

  inline error_code
  make_error_code(io_errc __e) noexcept
  { return error_code(static_cast<int>(__e), iostream_category()); }

  inline error_condition
  make_error_condition(io_errc __e) noexcept
  { return error_condition(static_cast<int>(__e), iostream_category()); }
#endif

Inline function make_error_code and make_error_condition are used by vcpkg, and iostream_category() are used in these two functions.

As discussed above, there is no implementation of iostream_category() in Ubuntu Trusty's gcc 7, 8 or 9.

To avoid the link error, we can modify code here to use non-params constructor of error_code and error_condition:

  // const error_category& iostream_category() noexcept;

  inline error_code
  make_error_code(io_errc __e) noexcept
  // { return error_code(static_cast<int>(__e), iostream_category()); }
  { return error_code(); }

  inline error_condition
  make_error_condition(io_errc __e) noexcept
  // { return error_condition(static_cast<int>(__e), iostream_category()); }
  { return error_condition(); }

With these modifications, vcpkg can be compiled on Ubuntu Trusty. And the side effects, some behavior related to error info reporting of vcpkg may not work properly, which I think are tolerable.

After vcpkg compiled, don't forget to revert modifications in /usr/include/c++/7.5.0/bits/ios_base.h.