Misp: GLIBCXX 3.4.20 not found

[nrgopi786]

Hi

am facing issue that system not finding the installed lief component. When manually trying to import lief in python module, throwing error as below

import lief
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python2.7/site-packages/lief/__init__.py", line 4, in
import _pylief
ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /usr/lib64/python2.7/site-packages/_pylief.so)

If you would like to report a bug, please fill the template bellow

Work environment

| Questions | Answers
|---------------------------|--------------------
| Type of issue | Bug
| OS version (server) | CentOS 7.4
| OS version (client) | XP, Seven, 10, Ubuntu, ...
| PHP version | 5.4, 5.5, 5.6, 7.0, 7.1...
| MISP version / git hash | 2.4.XX, hash of the commit
| Browser | If applicable

Expected behavior

LIEF Installed but diagnostics page showing that it's not installed

Actual behavior

Steps to reproduce the behavior

building MISP platform on CentOs 7.4

Logs, screenshots, configuration dump, ...

[Rafiot]

Did you try something like that?
https://askubuntu.com/questions/575505/glibcxx-3-4-20-not-found-how-to-fix-this-error

modulo how to install libstdc++6 on centos.

[nrgopi786]

I tried those options already but still no hope

[Rafiot]

Which packages did you install? Do you get any other error messages?

Because this one is relatively clear.

[nrgopi786]

the link which you provided is for ubuntu
am trying installation in centos 7.4

[nrgopi786]

python __init__.py.in
Traceback (most recent call last):
File "__init__.py.in", line 4, in
import _pylief
ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /usr/lib64/python2.7/site-packages/_pylief.so)
[root@baby python]# ll
total 92
drwxr-xr-x. 3 root root 89 Feb 28 19:10 Abstract
-rw-r--r--. 1 root root 6327 Feb 28 19:10 CMakeLists.txt
drwxr-xr-x. 3 root root 4096 Feb 28 19:10 ELF
-rw-r--r--. 1 root root 2603 Feb 28 19:10 encoding.cpp
-rw-r--r--. 1 root root 827 Feb 28 19:10 encoding.hpp
-rw-r--r--. 1 root root 334 Feb 28 19:10 __init__.py.in
drwxr-xr-x. 3 root root 4096 Feb 28 19:10 MachO
-rw-r--r--. 1 root root 25 Feb 28 19:10 MANIFEST.in
drwxr-xr-x. 3 root root 4096 Feb 28 19:10 PE
-rw-r--r--. 1 root root 1989 Feb 28 19:10 pyExceptions.cpp
-rw-r--r--. 1 root root 2917 Feb 28 19:10 pyIterators.cpp
-rw-r--r--. 1 root root 1727 Feb 28 19:10 pyIterators.hpp
-rw-r--r--. 1 root root 7810 Feb 28 19:10 pyJson.cpp
-rw-r--r--. 1 root root 1485 Feb 28 19:10 pyLIEF.cpp
-rw-r--r--. 1 root root 1299 Feb 28 19:10 pyLIEF.hpp
-rw-r--r--. 1 root root 1834 Feb 28 19:10 pyLogger.cpp
-rw-r--r--. 1 root root 1935 Feb 28 19:10 pyUtils.cpp
-rw-r--r--. 1 root root 514 Feb 28 19:10 README
-rw-r--r--. 1 root root 916 Feb 28 19:10 setup.cfg
-rwxr-xr-x. 1 root root 9218 Feb 28 19:10 setup.py.in

[nrgopi786]

[root@baby python]# ll /lib64/libstdc++.so.6
lrwxrwxrwx. 1 root root 19 Feb 27 12:45 /lib64/libstdc++.so.6 -> libstdc++.so.6.0.19
[root@baby python]# strings /lib64/libstdc++.so.6 | grep GLIBCXX
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_DEBUG_MESSAGE_LENGTH
[root@baby python]#

[nrgopi786]

[root@baby python]# yum list libstdc*
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile

• base: centos.mirror.net.in
• epel: mirror.nes.co.id
• extras: centos.excellmedia.net
• ius: hkg.mirror.rackspace.com
• updates: centos.excellmedia.net
  Installed Packages
  libstdc++.i686 4.8.5-16.el7_4.1 @updates
  libstdc++.x86_64 4.8.5-16.el7_4.1 @updates
  libstdc++-devel.i686 4.8.5-16.el7_4.1 @updates
  libstdc++-devel.x86_64 4.8.5-16.el7_4.1 @updates
  libstdc++-docs.x86_64 4.8.5-16.el7_4.1 @updates
  libstdc++-static.i686 4.8.5-16.el7_4.1 @updates
  libstdc++-static.x86_64 4.8.5-16.el7_4.1 @updates
  [root@baby python]#

[nrgopi786]

[root@baby python]# python
Python 2.7.5 (default, Aug 4 2017, 00:39:18)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import lief
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python2.7/site-packages/lief/__init__.py", line 4, in
import _pylief
ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /usr/lib64/python2.7/site-packages/_pylief.so)

[5]+ Stopped python
[root@baby python]# yum list glibc*
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile

• base: centos.mirror.net.in
• epel: mirror.nes.co.id
• extras: centos.excellmedia.net
• ius: hkg.mirror.rackspace.com
• updates: centos.excellmedia.net
  Installed Packages
  glibc.i686 2.17-196.el7_4.2 @updates
  glibc.x86_64 2.17-196.el7_4.2 @updates
  glibc-common.x86_64 2.17-196.el7_4.2 @updates
  glibc-devel.i686 2.17-196.el7_4.2 @updates
  glibc-devel.x86_64 2.17-196.el7_4.2 @updates
  glibc-headers.x86_64 2.17-196.el7_4.2 @updates
  glibc-static.i686 2.17-196.el7_4.2 @updates
  glibc-static.x86_64 2.17-196.el7_4.2 @updates
  glibc-utils.x86_64 2.17-196.el7_4.2 @updates
  [root@baby python]#

[nrgopi786]

[root@baby python]# hostnamectl
Static hostname: baby
Icon name: computer-vm
Chassis: vm
Machine ID: d8b9e596d0314c9fa0212dc31d153048
Boot ID: 6fc311d8ce764b18bba6a193fee7e6f3
Virtualization: vmware
Operating System: CentOS Linux 7 (Core)
CPE OS Name: cpe:/o:centos:centos:7
Kernel: Linux 3.10.0-693.17.1.el7.x86_64
Architecture: x86-64
[root@baby python]# cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

[root@baby python]#

[nrgopi786]

[root@baby ~]# strings /usr/lib64/python2.7/site-packages/_pylief.so | grep GLIBCXX_3.4.
GLIBCXX_3.4.20
GLIBCXX_3.4.18
GLIBCXX_3.4.11
GLIBCXX_3.4.9
GLIBCXX_3.4.14
GLIBCXX_3.4.21
_ZNSt12out_of_rangeC1EPKc@@GLIBCXX_3.4.21
_ZSt25__throw_bad_function_callv@@GLIBCXX_3.4.14
_ZNSt13runtime_errorC1EPKc@@GLIBCXX_3.4.21
_ZNSo9_M_insertIyEERSoT_@@GLIBCXX_3.4.9
_ZNSo9_M_insertImEERSoT_@@GLIBCXX_3.4.9
_ZNSo9_M_insertIPKvEERSoT_@@GLIBCXX_3.4.9
_ZNSo9_M_insertIlEERSoT_@@GLIBCXX_3.4.9
_ZSt16__ostream_insertIcSt11char_traitsIcEERSt13basic_ostreamIT_T0_ES6_PKS3_l@@GLIBCXX_3.4.9
_ZNKSt5ctypeIcE13_M_widen_initEv@@GLIBCXX_3.4.11
_ZNKSt8__detail20_Prime_rehash_policy14_M_need_rehashEmmm@@GLIBCXX_3.4.18
_ZSt24__throw_out_of_range_fmtPKcz@@GLIBCXX_3.4.20
_ZNSo9_M_insertIdEERSoT_@@GLIBCXX_3.4.9
_ZNSo9_M_insertIbEERSoT_@@GLIBCXX_3.4.9
_ZNSt13runtime_errorC2EPKc@@GLIBCXX_3.4.21

[nrgopi786]

vim setup.py.in
[root@baby python]# ./setup.py.in
/usr/lib/python2.7/site-packages/setuptools/dist.py:364: UserWarning: The version specified ('@LIEF_VERSION_MAJOR@.@LIEF_VERSION_MINOR@.@LIEF_VERSION_PATCH@') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details.
"details." % self.metadata.version
usage: setup.py.in [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py.in --help [cmd1 cmd2 ...]
or: setup.py.in --help-commands
or: setup.py.in cmd --help

error: no commands supplied

[nrgopi786]

[root@baby python]# easy_install --version
setuptools 38.5.1 from /usr/lib/python2.7/site-packages (Python 2.7)
[root@baby python]#

Is setuptoolds 38.5.1 compatible for LIEF?

[Rafiot]

I totally forgot about it, but @amuehlem is maintaining an RPM package for MISP, and iirc, it has lief pre-installed, meaning, the procedure should be somewhere: https://github.com/amuehlem/MISP-RPM

[amuehlem]

You can try it out:

yum install https://certrepo.switch.ch/certrepo/misp/lief-python-0.8.3-1.el7.x86_64.rpm

The RPMs are currently not GPG signed, but I can enable this (will have to coordinate with CIRCL about the key to use).

the complete install procedure is in the lief.spec file https://github.com/amuehlem/MISP-RPM/blob/master/SPECS/lief.spec

[nrgopi786]

How long it would take for getting GPG signed for this package?

How to install setup tools 36.0.0?

[amuehlem]

see the install instructions , you can enable our repository, where all the dependencies can be found. I'm currently testing my RPMs on a test server, any input is appreciated.

I'll check, gpg signing shouldn't be a big issue, its more a question of which key to use. But I think I'll use a own gpg key as long as we provide the packages from our repository.

[nrgopi786]

Hi

It seems to be bug on the lief module where minimum version required is mapped as GLIBCXX 3.4.20 which is not available in centos but in ubuntu

root@misp:/usr/lib/x86_64-linux-gnu# strings libstdc++.so.6.0.24 | grep GLIBCXX
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_3.4.20
GLIBCXX_3.4.21
GLIBCXX_3.4.22
GLIBCXX_3.4.23
GLIBCXX_3.4.24
GLIBCXX_DEBUG_MESSAGE_LENGTH
root@misp:/usr/lib/x86_64-linux-gnu# cat /etc/os-release
NAME="Ubuntu"
VERSION="17.10 (Artful Aardvark)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 17.10"
VERSION_ID="17.10"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=artful
UBUNTU_CODENAME=artful
root@misp:/usr/lib/x86_64-linux-gnu#

[amuehlem]

That's exactly why I've compiled the lief package for CentOS 7. Using my repository, I can install MISP on a CentOS 7 system and all dependencies (pymisp, pydeep, magic and lief) are green.

I've updated my repository and signed all packages with a GPG key, the newest repository release installs the key and has gpgcheck enabled.

yum install https://certrepo.switch.ch/certrepo/misp/misp-release-1.0-5.el7.noarch.rpm

See the complete installation instructions for more details

[SteveClement]

@nrgopi786 I suppose this is resolved.
Thank to @amuehlem for the help!

[iwitz]

Hi,
I have the same issue on RHEL 7.5 and RHEL 7.6. Installing a pre-compiled package that doesn't come from the official RedHat repositories is not an option in my organization.
I followed the instructions in section 11 of the RHEL installation instruction. After that I can import LIEF successfully within a Python 2 interpreter, but when I try to import LIEF within a Python 3 interpreter I get the following :

# scl enable rh-python36 'python3'
Python 3.6.3 (default, Jan  9 2018, 10:19:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import lief
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/lief-0.9.0-py3.6-linux-x86_64.egg/lief/__init__.py", line 4, in <module>
    import _pylief
ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/lief-0.9.0-py3.6-linux-x86_64.egg/_pylief.cpython-36m-x86_64-linux-gnu.so)
>>> quit()

And LIEF isn't detected by MISP in the diagnostics page. It's likely due to the fact that the RHEL installation instructions install all of MISP's dependencies with Python 3 and enable the use of Python 3 by PHP-FPM, but compile LIEF with parameters for Python 2. ( scl enable devtoolset-7 'bash -c "cmake3 \ -DLIEF_PYTHON_API=on \ -DLIEF_DOC=off \ -DCMAKE_INSTALL_PREFIX=$LIEF_INSTALL \ -DCMAKE_BUILD_TYPE=Release \ -DPYTHON_VERSION=2.7 \ .."' )

I've been trying to install LIEF with Python 3 lately, I'll submit pacth to the installation instructions when I pull it off.

[Rafiot]

Woops, good catch. I don't have a way to test it myself, but does it work if you replace -DPYTHON_VERSION=2.7 by -DPYTHON_VERSION=3.6?

[iwitz]

It doesn't, at least it hasn't worked the couple of times I've tried.

[Rafiot]

And what does thins one say (modulo the version of libstdc++ you're using):
root@misp:/usr/lib/x86_64-linux-gnu# strings libstdc++.so.6.0.24 | grep GLIBCXX

[iwitz]

strings /usr/lib64/libstdc++.so.6.0.19 | grep GLIBCXX
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_DEBUG_MESSAGE_LENGTH

I did compile LIEF with devtoolset-7 enabled though so theoretically that shouldn't be an issue ( or at least it wasn't when compiling LIEF for Python 2 ).

UPDATE : I managed to build it and install it, I'm trying to reproduce it so I can document it

[Rafiot]

Excellent, thanks!