Misp: STIX, CyBox and mixbox not installed correctly in RHEL 7.5

[jezkerwin]

Work environment

| Questions | Answers
|---------------------------|--------------------
| Type of issue | Bug
| OS version (server) | RedHat 7.5
| OS version (client) | Windows 10 Enterprise
| PHP version | 7.1 RH Software Collections
| MISP version / git hash | v2.4.92 (6eb4e49f2c62d05e671e3482ac1b21004ce58ee3)
| Browser | Firefox 60

Expected behavior

Diagnostic page reports that Stix, Cybox and Mixbox are installed correctly
Exporting an event as STIX 1 doesn't return "An Error Has Occured'

Actual behavior

Diagnostic page shows that the libraries are not installed
"STIX and Cybox libraries…STIX or CyBox or mixbox library not installed correctly"
Exporting events as STIX result in an error being returned

Stix, Cybox and Mixbox can be imported as a python module in both python2 and rh-python36

Steps to reproduce the behavior

[root@misp01 MISP]# python
Python 2.7.5 (default, Feb 20 2018, 09:19:12)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import stix
import cybox
import mixbox

[root@misp01 MISP]# scl enable rh-python36 python
Python 3.6.3 (default, Jan 9 2018, 10:19:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.

import stix
import cybox
import mixbox

Logs, screenshots, configuration dump, ...

[RichieB2B]

You need to enable the use of python3 within php, see the documentation update at https://github.com/MISP/MISP/pull/3323

[jezkerwin]

Thanks for the suggestion. It's still not working for me.

diagnostic page still can't find the modules and when exporting an event as STIX XML I get the following error.

An Internal Error Has Occurred.An Internal Error Has Occurred./events/stix/download/1.xml

[RichieB2B]

What do app/tmp/logs/exec-errors.log and app/tmp/logs/error.log say about this?

[jezkerwin]

app/tmp/logs/error.log says the following

2018-06-07 00:37:43 Error: [Exception] There was an issue generating the STIX export.
Request URL: /events/stix/download/1.xml
Stack Trace:

0 [internal function]: EventsController->stix('download', '1')

1 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Controller.php(491): ReflectionMethod->invokeArgs(Object(EventsController), Array)

2 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(193): Controller->invokeAction(Object(CakeRequest))

3 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(167): Dispatcher->_invoke(Object(EventsController), Object(CakeRequest))

4 /var/www/MISP/app/webroot/index.php(92): Dispatcher->dispatch(Object(CakeRequest), Object(CakeResponse))

5 {main}

2018-06-08 22:03:25 Error: [Exception] There was an issue generating the STIX export.
Request URL: /events/stix/download/1.xml
Stack Trace:

0 [internal function]: EventsController->stix('download', '1')

1 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Controller.php(491): ReflectionMethod->invokeArgs(Object(EventsController), Array)

2 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(193): Controller->invokeAction(Object(CakeRequest))

3 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(167): Dispatcher->_invoke(Object(EventsController), Object(CakeRequest))

4 /var/www/MISP/app/webroot/index.php(92): Dispatcher->dispatch(Object(CakeRequest), Object(CakeResponse))

5 {main}

exec-errors.log just says the following.

sh: python3: command not found

[RichieB2B]

There you go: php can't find the python3 command. Try explicitly including it in the env[PATH] inside /etc/opt/rh/rh-php71/php-fpm.d/www.conf and then restart rh-php71-php-fpm.service
Also, double check clear_env is set to no in `/etc/opt/rh/rh-php71/php-fpm.d/www.conf

[jezkerwin]

Adding the path to www.conf seems to have worked. the diagnostic page and downloading an event as STIX is now working.

Shouldn't adding this command make adding the path not necessary.

echo 'source scl_source enable rh-python36' >> /etc/opt/rh/rh-php71/sysconfig/php-fpm

That file on my system states the following:

`Additional environment file for php-fpm

This file is deprecated when systemd is used and
will be removed in the future

With systemd >= 204 you can simply drop a file with the
suffix .conf in /etc/systemd/system/rh-php71-php-fpm.service.d, with
[Service]
Environment=FOO=bar

See systemd documentation.
man systemd.unit
man systemd.exec

source scl_source enable rh-python36
`

[RichieB2B]

We had similar issues getting this to work. I agree that setting the correct path in /etc/opt/rh/rh-php71/sysconfig/php-fpm and setting clear_env is set to no in /etc/opt/rh/rh-php71/php-fpm.d/www.conf should be enough. That is how it is currently works for us, but before we had to modify env[PATH] as well.

[iglocska]

Yet again, proof that RHEL is evil. Thanks @RichieB2B for being an effing hero again!!!

[jezkerwin]

RHEL isn't evil, its just a different way of doing things 😁

Thanks @richieb2b for your help. It even fixed the diagnostics not seeing the php-cli

I might do some research and look to migrate to systemd 😋

[RichieB2B]

I’m glad it’s working now. Can you share how you finally solved it? Do we need to update the documentation?

[jezkerwin]

It was your suggestion to add the path to the python3 executable from scl into the path in php-fpm file.
I also added the path to the scl version of php-cli in there as well. That solved the php diagnostic issue.

[adulau]

@jezkerwin Thank you for the feedback. Thanks to @RichieB2B our RHEL expert ;-)

[iglocska]

Candid video of @RichieB2B fixing RHEL issues:

https://www.youtube.com/watch?v=YgYEuJ5u1K0

[RichieB2B]

Yeah, if you forget to pray to the crimson titfer gods before troubleshooting all is lost. I'm just glad nobody is reporting issues with MISP on FreeBSD..

[iglocska]

Don't give them ideas!...

[gulsoomro]

hello guys i am having some problem in mapping stix2 logs,to convert from common format logs to stix2 formate logs. Stix2 provide SDO object for attack type Malware if we occur some other type of attack like (Sip type, Brute force,db,recon) then how to map that type of logs?

[jezkerwin]

I'd suggest raising a separate issue.