Openshot-qt: OpenShot 2.4.4 AppImage does not start on RHEL 7.6

[toracat]

Describe the bug:
OpenShot 2.4.4 AppImage does not start on RHEL 7.6

Steps to reproduce the behavior:

1. Run OpenShot-v2.4.4-x86_64.AppImage on RHEL 7.6
2. See error

Expected behavior:
It should start without an error.

System Details:

• OpenShot Version: 2.4.4
• Operating System / Distro: Red Hat Enterprise Linux 7.6

Log Files:
Traceback (most recent call last):
File "/home/gitlab-runner/builds/5cd61c66/0/OpenShot/openshot-qt/openshot.py", line 18, in swig_import_helper
File "/usr/lib/python3.4/imp.py", line 297, in find_module
ImportError: No module named '_openshot'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/cx_Freeze-4.3.4-py3.4-linux-x86_64.egg/cx_Freeze/initscripts/Console.py", line 27, in
File "openshot_qt/launch.py", line 55, in
from classes.app import OpenShotApp
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 2237, in _find_and_load
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 2226, in _find_and_load_unlocked
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 1200, in _load_unlocked
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 1129, in _exec
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 1471, in exec_module
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 321, in _call_with_frames_removed
File "/tmp/.mount_JW4CX8/usr/bin/classes/app.py", line 41, in
from classes import info, settings, project_data, updates, language, ui_util, logger_libopenshot
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 2284, in _handle_fromlist
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 321, in _call_with_frames_removed
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 2237, in _find_and_load
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 2226, in _find_and_load_unlocked
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 1200, in _load_unlocked
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 1129, in _exec
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 1471, in exec_module
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 321, in _call_with_frames_removed
File "/tmp/.mount_JW4CX8/usr/bin/classes/logger_libopenshot.py", line 31, in
import openshot
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 2237, in _find_and_load
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 2226, in _find_and_load_unlocked
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 1191, in _load_unlocked
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 1161, in _load_backward_compatible
File "/home/gitlab-runner/builds/5cd61c66/0/OpenShot/openshot-qt/openshot.py", line 28, in
File "/home/gitlab-runner/builds/5cd61c66/0/OpenShot/openshot-qt/openshot.py", line 20, in swig_import_helper
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 2237, in _find_and_load
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 2226, in _find_and_load_unlocked
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 1191, in _load_unlocked
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 1161, in _load_backward_compatible
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 539, in _check_name_wrapper
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 1715, in load_module
File "/usr/lib/python3.4/importlib/_bootstrap.py", line 321, in _call_with_frames_removed
ImportError: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /tmp/.mount_JW4CX8/usr/bin/libresvg.so)

[mSparks43]

Same here (CentOS7), reverted to version 2.4.3

[uberlinuxguy]

I would like to add to this that I experienced the same issue.

CentOS Linux release 7.5.1804 (Core)
kernel 4.19.4-1.el7.elrepo.x86_64

I am not sure what other useful information is needed to help investigate the issue, but if anything is required, I will gladly help.

I'm not familiar with AppImages but I think I might have to look into this.

[toracat]

According to the error log:

ImportError: /lib64/libc.so.6: version 'GLIBC_2.18' not found

OpenShot 4.4.4 apparently requires glibc 2.18. The glibc version in RHEL/CentOS 7 is 2.17. So, I suppose 2.4.3 is the last one that runs on EL7.

RHEL 8 (beta at this moment) has glibc 2.28. Therefore, to run newer versions of OpenShot, it may be necessary to update the OS to EL8 once it is released.

[uberlinuxguy]

I thought that too but I also thought the point of AppImages was to not rely on the system and that the app was self contained

[toracat]

Yes, that came to my mind, too. It is still possible that this is a packaging issue...

[toracat]

Unfortunately my original guess seems to be right. There is this description:

Some core libraries, such as glibc, tend to break compatibility with older base systems quite frequently, which means that binaries will run on newer, but not on older base systems than the one the binaries were compiled on.

[ferdnyc]

It looks like, as of four days ago, python3-sip and python3-PyQt5 are available in EPEL7. That means I should now be able to package OpenShot 2.4.4 for EPEL7 in RPM Fusion. I'll work on that this weekend, since the AppImage is no longer compatible.

[toracat]

ferdnyc, that will be fantastic!

[ferdnyc]

Okay, these are _completely_ untested because I don't have an EL7 system to test them on (they were built with mock on F29), but here are a set of OpenShot 2.4.4 RPMs for EL7 x86_64.

You'll need to --enablerepo=updates-testing when installing these using dnf/yum. Some of the packages they require haven't been pushed to updates yet, and may not be for another two weeks. After
this update comes out of testing, they should be installable normally, and I can publish them via RPM Fusion.

Dropbox and Google Drive sharing links, because the 60MB archive is way too big for GitHub comments.

https://www.dropbox.com/s/p8g8ffp402w3bbb/openshot-el7-rpms.tar.gz?dl=0

https://drive.google.com/open?id=1l9CEemvIXt1avSYpu2FQzNlzC2st4GhL

ETA: Updated links, see below

https://www.dropbox.com/s/6u784xpo41eikfq/openshot-el7-rpms2.tar.gz?dl=0

https://drive.google.com/file/d/1eMFkMkS_3F_xuyAwhGK4iQfcW44AOQzK/view?usp=sharing

[toracat]

Thank you. I tried it on a RHEL 7 system. The installation fails due to dependency issues. The following packages are required:

python36-httplib2
python36-qt5-webkit
python36-zmq

[ferdnyc]

@toracat Thanks for testing! python36-qt5-webkit _should_ be part of orion's python-qt5 update, hopefully. python-httplib2 and python-zmq exist in EPEL7, but only in their python 2 versions. I'll get in touch with their maintainers and ask about python 3 builds.

[ferdnyc]

Okay, I've submitted requests for both of those packages.

In the meantime, here are python36-zmq and python36-httplib2, again built in mock on my F29 box:

zmq_and_httplib2.tar.gz

As well as orion's PyQt5 rpms:
python36-qt5.tar.gz

And the necessary SIP rpms for PyQt5:
sip.tar.gz

I actually _tested_ the install this time, in Mock (like I should've in the first place), and that was everything necessary to successfully install the OpenShot RPMs.

If you have trouble with errors like:

Error: Package: python36-qt5-base-5.12.1-2.el7.x86_64 (/python36-qt5-base-5.12.1-2.el7.x86_64)
           Requires: python36-pyqt5-sip-api(12) >= 12.6

Try installing _just_ the RPMs from sip.tar.gz by themselves, first, and _then_ the rest of the RPMs. The SIP RPMs install some macros that seem to affect the dependency resolution of the other RPMs, so it seems like they might have to be in place before attempting to install the rest.

I think that was just a bad test on my part, I re-checked and it works fine to install everything all at once.

[toracat]

With the 3 required packages installed, openshot was finally installed without an error. However running it produced the following error:

Loaded modules from installed directory: /usr/lib/python3.6/site-packages/openshot_qt
launch:INFO ------------------------------------------------
launch:INFO OpenShot (version 2.4.4)
launch:INFO ------------------------------------------------
app:INFO openshot-qt version: 2.4.4
app:INFO libopenshot version: 0.2.3
app:INFO platform: Linux-3.10.0-957.12.1.el7.x86_64-x86_64-with-redhat-7.6-Maipo
app:INFO processor: x86_64
app:INFO machine: x86_64
app:INFO python version: 3.6.6
app:INFO qt5 version: 5.9.2
app:INFO pyqt5 version: 5.12.1
logger:ERROR Traceback (most recent call last):
logger:ERROR File "/usr/bin/openshot-qt", line 11, in
logger:ERROR load_entry_point('openshot-qt==2.4.4', 'gui_scripts', 'openshot-qt')()
logger:ERROR File "/usr/lib/python3.6/site-packages/openshot_qt/launch.py", line 102, in main
logger:ERROR app = OpenShotApp(argv)
logger:ERROR File "/usr/lib/python3.6/site-packages/openshot_qt/classes/app.py", line 87, in __init__
logger:ERROR from classes import exceptions
logger:ERROR File "/usr/lib/python3.6/site-packages/openshot_qt/classes/exceptions.py", line 30, in
logger:ERROR from classes.metrics import track_exception_stacktrace
logger:ERROR File "/usr/lib/python3.6/site-packages/openshot_qt/classes/metrics.py", line 30, in
logger:ERROR import requests
logger:ERROR ModuleNotFoundError
logger:ERROR :
logger:ERROR No module named 'requests'

[ferdnyc]

Hmm, seems we have an undeclared dependency on python-requests. It looks like python36-requests is in EPEL7, can you try a dnf install python36-requests and see if that fixes it? I'll add the dependency to OpenShot's .spec file for the next set of builds.

[ferdnyc]

I looked through the rest of the imports, and I _think_ the only other hidden dependency is simplejson. All the rest look to be either declared, or standard Python 3 libraries. So, you may need a dnf install python36-simplejson as well.

[ferdnyc]

Here are links to an updated set of OpenShot RPMs with the python36-simplejson and python36-requests dependencies declared, so dnf/yum should pull them in automatically.

https://www.dropbox.com/s/6u784xpo41eikfq/openshot-el7-rpms2.tar.gz?dl=0

https://drive.google.com/file/d/1eMFkMkS_3F_xuyAwhGK4iQfcW44AOQzK/view?usp=sharing

[toracat]

I have a report with mixed results.

First, after installation of python36-requests, openshot successfully started. So, this is all good.

Second, to test an updated set of packages from openshot-el7-rpms2.tar.gz, I used a fresh system that had not seen any of the related package. This one failed to install with several missing required libraries:

Error: Package: libopenshot-0.2.3-1.el7.x86_64 (/libopenshot-0.2.3-1.el7.x86_64)
Requires: libswresample.so.2(LIBSWRESAMPLE_2)(64bit)
Requires: libavcodec.so.57(LIBAVCODEC_57)(64bit)
Requires: libswscale.so.4()(64bit)
Requires: libswresample.so.2()(64bit)
Requires: libavcodec.so.57()(64bit)
Requires: libswscale.so.4(LIBSWSCALE_4)(64bit)
Requires: libavformat.so.57(LIBAVFORMAT_57)(64bit)
Requires: libavutil.so.55(LIBAVUTIL_55)(64bit)
Requires: libavutil.so.55()(64bit)
Requires: libavformat.so.57()(64bit)

This is what is happening:

While struggling with the installation earlier, I built the packages myself, but by using the fc28 srpms. The libopenshot package built from the fc28 sources could be installed by using ffmpeg-libs-2.8.15-2.el7 from the nux-dextop repo as dependency. I believe yours is built from the fc29 sources. It apparently requires newer versions of the above libraries which are not available for EL7.

[ferdnyc]

@toracat The libopenshot RPMs in my tarballs were built against ffmpeg-devel from RPM Fusion. (The SRPMs for various releases are typically identical. Dependency versions are determined at compile time, based on the versions supplied by the -devel packages used for the build.)

The compiled libopenshot would require an equivalent ffmpeg-libs, which _should_ be pulled in automatically if the rpmfusion-free-updates repo is enabled during install. (RPM Fusion is considered a prereq for OpenShot, since it depends on packages not in the Fedora/EPEL repos.) Our current FFMpeg build for EL7 is ffmpeg-libs-3.4.6-1.el7.x86_64.rpm, found here: ffmpeg-libs: RPM Fusion EL 7 Free x86_64

But building from SRPM using ffmpeg-devel from nux-desktop should work as well, and would result in a libopenshot compatible with the corresponding nux-desktop ffmpeg-libs.

[toracat]

That explains it. Thanks.

[toracat]

OK, with the rpmfusion-free-updates repo installed, openshot installed successfully. However, it did not run. The error was:
ModuleNotFoundError: No module named 'pkg_resources'

I then installed python36-setuptools and openshot now starts normally.

Perhaps you need to add Requires: python36-setuptools to openshot ? Or is it supposed to be pulled in by some other package?

[ferdnyc]

Perhaps you need to add Requires: python36-setuptools to openshot ? Or is it supposed to be pulled in by some other package?

Nope, that's another hidden dependency I missed. It was in the BuildRequires, but I didn't catch that it was needed at runtime as well. Thanks for letting me know, I'll add it in.

[toracat]

There is one other thing I want you to consider...

A number of RHEL/CentOS/SL desktop users have the nux-dextop repo enabled. They would encounter the same problem as I did, that is, nux's ffmpeg-libs satisfies the dependencies. So your version will not install. It may be a good idea to add ffmpeg-libs > 3 (or some such), I suppose.

[ferdnyc]

@toracat That's fine, the nux ffmpeg-libs won't actually satisfy the dependency — as you discovered, the RPM Fusion libopenshot...

Requires: libswresample.so.2(LIBSWRESAMPLE_2)(64bit)
Requires: libavcodec.so.57(LIBAVCODEC_57)(64bit)
Requires: libswscale.so.4()(64bit)
Requires: libswresample.so.2()(64bit)
Requires: libavcodec.so.57()(64bit)
Requires: libswscale.so.4(LIBSWSCALE_4)(64bit)
Requires: libavformat.so.57(LIBAVFORMAT_57)(64bit)
Requires: libavutil.so.55(LIBAVUTIL_55)(64bit)
Requires: libavutil.so.55()(64bit)
Requires: libavformat.so.57()(64bit)

None of which nux ffmpeg-libs provides.

Those dependencies won't be satisfied unless RPM Fusion is enabled. And since this OpenShot package is intended for RPM Fusion, that'll be a given when installing it. It's only because we're doing an end-run around the normal repo setup, that things got a little hairy.

Now, since nux-desktop also publishes an ffmpeg-libs, then there _may_ be issues for users trying to use nux and RPM Fusion together. RPM Fusion ffmpeg-libs-3.4.6 and nux ffmpeg-libs-2.8.15 probably can't be installed side-by-side. But that's never been a supported configuration, and there's really not much we can do about that. RPMs in the RPM Fusion repos are built to require only distro packages (EPEL7, in this case) or other RPM Fusion packages. (Which is why I can't release official RPM Fusion builds of OpenShot for EL7 yet, until those last few dependencies have made their way to EPEL7.)

[ferdnyc]

Things have been progressing on the EPEL side, BTW.

• The python-qt5 update has 5 days left in testing before it reaches stable. (Feel free to give it karma if you're using it on EL7, anyone, since that'll help it get through the process faster. The python36-qt5 packages I provided above were all from that update, so if you're using those you're using the official test build.)

• python-httplib2 is now in testing as well, not sure if that'll be subject to the same 2-week testing period or if it'll come out quicker. (Again, karma would be great to help expedite the process. But you'd have to install and test the build from that update candidate, it's not the same as the one I shared here.)

• python-zmq should be along presently, the maintainers have no objection to my request for a build, they're just still nailing down logistics.

[toracat]

Added karma to python-qt5 (yours) and python-httplib2 (downloaded and installed).

[ferdnyc]

Update: To my very great surprise, openshot-2.4.4 is actually now _available_ in RPM Fusion free for EL7. I'm not sure exactly why, but @sergiomb2 built it a few days ago, and it's progressed to the updates repo.

However, python36-zmq has not yet made it into EPEL7, so it's still missing one dependency to actually be installable/usable. We're hopefully close to getting that sorted out. (I'm trying to strike a balance between wanting to get this sorted out ASAP for the OpenShot users interested in running 2.4.4 on EL7, vs. _not_ wanting to be too pushy or impatient with the maintainers of the required EPEL7 packages.)

Once the package is built and submitted as an update, it'll still take 2 weeks to mature in testing before it moves to stable, unless we can scare up 3 EL7 users to submit positive karma and move it along faster. Regardless, even 2 weeks isn't forever, and at least there'd be a finish line in sight. So, hopefully we can at least get that clock _started_ ticking.

[sergiomb2]

Hello ,
yes , Orion made packages for epel7 only of python3-pillow [1] , which allow python3-pygame [2], more python3-sip [3] and python3-qt5 [4]
So I could build openshot ... , sorry I missed that you are already on it . My motivation was keep RPMFusion git branches aligned (If I'm explain well ) .
Thanks

[1]
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-2819a72a78

[2]
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-02008e7514

[3]
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-b56adcd6f9

[4]
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-f5aa6f723a

[ferdnyc]

@sergiomb2 Not at all, thanks for taking care of it. I was just caught by surprise when I saw it go by in the Update Report. But it's one less thing for me to do, and it makes sense as this way it'll already be waiting in the repo, for whenever python-zmq finally lands in EPEL7.

[sergiomb2]

python-zmq is in epel-testing [1] we may starting testing :smile:

[1]
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-f8c7028b60

[ferdnyc]

@h1z1 Yeah, the AppImage situation is unlikely to change.

Thanks for the report on ZeroMQ. Both openshot and libopenshot are correctly depended on python3.6-zmq and cppzmq/zeromq, so as you say it looks like an upstream issue. I'll look into it, but it'd be a big help if you could supply the output of a few commands:

ldd /usr/lib64/python3.6/site-packages/_openshot.so
ldd /usr/lib64/libopenshot.so.17
rpm -ql zeromq

Thanks.

[ferdnyc]

Not zlib. ZeroMQ. Corrected.

[ferdnyc]

Actually, make that ldd /usr/lib64/libopenshot.so.17, you won't have a libopenshot.so unless you installed libopenshot-devel.

[sergiomb2]

I'll leave the political hillarity of appimage aside, tried this today though. The appimage still fails due to glib. The RPM fails to install due to a missing dependency on zeromq itself. Likely more an upstream problem
log output

I did a pull request [1] that was merged but was build [2]

[1]
https://src.fedoraproject.org/rpms/python-zmq/pull-request/4#request_diff
[2]
https://src.fedoraproject.org/rpms/python-zmq/commits/epel7

[ferdnyc]

Aha! I missed that.

Still seems odd, though, as libopenshot also has a dependency on libzmq.so.5()(64bit) — maybe it needs the same explicit Requires? (Or whatever this sclo-cassandra3-zeromq is needs to have that Provides: disabled.)

Anyway, I can easily rebuild libopenshot with a Requires: zeromq (or should that be Requires: zeromq%{isa}?), if that'll fix it. Can you try doing a yum install zeromq.x86_64, @h1z1, and see if that sorts it out?

[ferdnyc]

@h1z1

#  rpm -qa |grep zeromq 
zeromq3-3.2.5-1.el7.x86_64
zeromq-4.1.4-6.el7.x86_64
sclo-cassandra3-zeromq-4.1.6-7.el7.x86_64
#  rpm -qf /usr/lib64/libopenshot.so.17  /usr/lib64/python3.6/site-packages/_openshot.so
libopenshot-0.2.3-2.20190406git101f25a.el7.x86_64
python36-libopenshot-0.2.3-2.20190406git101f25a.el7.x86_64

Just so I'm clear, that's _after_ you manually installed zeromq, right? (Which wasn't installed originally?)

# ldd /usr/lib64/libopenshot.so.17  
    libavcodec.so.57 => not found
    libavformat.so.57 => not found
    libavutil.so.55 => not found
    libswscale.so.4 => /lib64/libswscale.so.4 (0x00007ff9f140e000)
    libswresample.so.2 => /lib64/libswresample.so.2 (0x00007ff9f11f1000)
    libavutil.so.55 => not found
    libavutil.so.55 => not found

As mentioned to @toracat above, you're going to need to upgrade ffmpeg-libs to RPMFusion's, looks like you have a different version installed from elsewhere.
https://download1.rpmfusion.org/free/el/updates/7/x86_64/repoview/ffmpeg-libs.html

[SuslikV]

@h1z1 Yes, there is reason on exact version of FFmpeg (until you don't compile your libopenshot from the source).

[h1z1]

@SuslikV I meant API

[ferdnyc]

It's a matter of what version libopenshot is compiled with, as @SuslikV says. If you compile libopenshot from source, it's compatible with any FFmpeg 2.x-4.x, but RPM Fusion libopenshot is built against RPM Fusion ffmpeg-libs, which for EPEL7 is currently 3.4.6.

[ferdnyc]

That's correct, and as I said it can be compiled with any FFmpeg version 2.x or greater.

But the libopenshot code that uses FFmpeg APIs adjusts to match the specific APIs of the libav____.so shared library versions it's compiled and linked with. To use a different FFmpeg version, libopenshot needs to be recompiled to match those APIs, and linked to those shared libraries. The process of compiling libopenshot matches it to a specific ffmpeg-libs (and, actually, ffmpeg-devel) version.

Whatever ffmpeg-libs you have installed on your system is probably compatible with libopenshot, but _not_ with RPM Fusion's libopenshot RPM. You'd have to install the corresponding ffmpeg-devel package, and build a matching libopenshot from source.

[SuslikV]

@h1z1 Any sane version off FFmpeg will work. No special functions required. All stuff specific to the some build is escaped in the code itself.

Probably, you don't understand what OpenShot is and what it depends on. Same as AppImage "wrapper".

[h1z1]

Sure. In the mean time I've recompiled Openshot as I said but you can think what you like.

Have a lovely day.

[ferdnyc]

@h1z1

Sorry, the nature of your question wasn't clear until your most recent (deleted) comments. (I apologize for not grasping what you were asking. When you asked "Is there a reason FFmpeg 3.4.6 is required?", I thought you were asking about the RPM Fusion binary package.)

As far as libopenshot compatibility, I'm not sure what exactly is the _lowest_ version of FFmpeg that's compatible, but it may even be earlier than ffmpeg 2.0. That being said, ffmpeg itself lacks some features when you go back that far.

Hardware decoding and encoding is only available in libopenshot with FFmpeg 3.2 or later.

Support for the H.265 codec was also spotty with older versions — to be honest I'm not sure what the minimum version on that is, either. I know that libopenshot built with FFmpeg 3.4.x or later has generally produced good results (if ffmpeg was built with --enable-libx265). FFmpeg 1.x or 2.x has generally been problematic.

[SuslikV]

Sure. In the mean time I've recompiled Openshot as I said but you can think what you like.

Have a lovely day.

@h1z1 , Good luck.

[stale[bot]]

Thank you so much for submitting an issue to help improve OpenShot Video Editor. We are sorry about this, but this particular issue has gone unnoticed for quite some time. To help keep the OpenShot GitHub Issue Tracker organized and focused, we must ensure that every issue is correctly labelled and triaged, to get the proper attention.
This issue will be closed, as it meets the following criteria: - No activity in the past 180 days - No one is assigned to this issue
We'd like to ask you to help us out and determine whether this issue should be reopened. - If this issue is reporting a bug, please can you attempt to reproduce on the latest daily build to help us to understand whether the bug still needs our attention. - If this issue is proposing a new feature, please can you verify whether the feature proposal is still relevant.
Thanks again for your help!