Server: Vote: Skip Ubuntu 17.10 support and focus on 18.04 LTS instead

[dotarmin]

Hi!

Background

I have been trying to run CasparCG Server 2.2.0 on Ubuntu 17.10 without any success. After it has been built using Docker using the Docker-files provided and extracted from Docker I get the following error when trying to run the server by running run.sh.

[0306/154319.679913:ERROR:nss_util.cc(650)] Error initializing NSS with a persistent database (sql:/home/caspar/.pki/nssdb): libsoftokn3.so: cannot open shared object file: No such file or directory

I have been looking around for the error without any success but what I have came across is that a solution is to install libnss3-nssdb package which isn't available for Ubuntu 17.10 according to Ubuntu packages and Repopackages.

@baltedewit has been helpful with this issue but neither of us have any solution right now. The interesting thing here is that on some computer it actually runs fine and some it don't. It seems like some linux package does actually include the libnss3-nssdb in 17.10 but what package it is we don't know right now.

Suggestion

My suggestion is to skip support for Ubuntu 17.10 and focus on 18.04 LTS when it's released and maybe even wait until summer when major bugs has been fixed in 18.04. Users who wanna run CasparCG Server 2.2.0 on Linux 17.10 or other versions/dists can do it but should't depend on support for those versions.
Another thing beside this issue is that we should put focus on LTS variants only.

Vote

👍 : Focus on Ubuntu 18.04 LTS instead
👎 : Stay on Ubuntu 17.10

/Armin

[Julusian]

I started having a play with building for 18.04 last night. and have something that runs. I hit the same error on attempting to run, which was just some files that weren't being copied.
https://github.com/Julusian/CasparCG-Server/commit/2eb56d7e5ec8d2b6473ff14591b61520fb75cfbf#diff-38d13f5bb32606e35ea06563ff190d35

[walterav1984]

@dotarmin
Could you also start a "vote/poll" on which desktop environment to run or just ubuntu server with a window manager? Because I like @Julusian his notice that openbox gave him more performance than Unity>Gnome3 will give by default. Although I'm not sure openbox used vsync...

and I'm pretty sure that Unity was using 15-20% of the GTX960 just idle. I ended up swapping Unity out for Openbox, which cut that down to <5%.
https://github.com/CasparCG/server/issues/877#issuecomment-368837920

@Julusian I will try your 18.04 patches, I also had a little success since this week on Ubuntu 18.04 with nvidia-390 proprietary driver(needed for OpenGL4.5 older versions are not an option) on a Geforce GTX560ti and with @sesse his help from the BlackmagicDesign forum with DesktopVideo 10.9.10 driver is also working on the linux 4.15 kernel in 18.04 for me.

What I'm trying to say is that 18.04 is getting more and more a workable option. However I noticed that it could take a month or more after 18.04 LTS release before Docker-CE will make a supported "bionic" option, right now I'm using the "artful" option in apt!

Instructions for getting BMD Decklink/Intensity DesktopVideo driver supported.

cd tofolderwithextractedbmddebs
sudo dkpg - i *.deb
sudo apt-get -f install

Than edit both bm_util.c and bm_util.h in /usr/src/blackmagic-io-10.9.10a2.orig with @sesse his diff from https://forum.blackmagicdesign.com/viewtopic.php?f=3&t=68382:

diff -u blackmagic-io-10.9.10a2.orig/bm_util.c blackmagic-io-10.9.10a2/bm_util.c
--- blackmagic-io-10.9.10a2.orig/bm_util.c   2018-01-08 02:46:14.000000000 +0100
+++ blackmagic-io-10.9.10a2/bm_util.c   2018-01-31 23:27:01.543121803 +0100
@@ -411,39 +411,45 @@
    return jiffies_to_usecs(jiffies);
 }

+static void timer_thunk(struct timer_list* kernel_timer)
+{
+   bm_timer_t* timer = container_of(kernel_timer, bm_timer_t, kernel_timer);
+   (*timer->callback)(timer->data);
+}
+
 bm_timer_t* bm_timer_alloc(bm_timer_callback_t callback, void* data)
 {
    bm_timer_t* timer = bm_kmalloc(sizeof(bm_timer_t));
    if (!timer)
       return NULL;

-   init_timer(timer);
-   timer->function = (void(*)(unsigned long))callback;
-   timer->data = (long)data;
+   timer->callback = callback;
+   timer->data = data;
+   timer_setup(&timer->kernel_timer, timer_thunk, 0);
    return timer;
 }

 void bm_timer_free(bm_timer_t* timer)
 {
-   del_timer(timer);
+   del_timer(&timer->kernel_timer);
    bm_kfree(timer);
 }

 void bm_timer_expire_in(bm_timer_t* timer, uint64_t ns)
 {
-   timer->expires = jiffies + (ns * HZ + NSEC_PER_SEC - 1) / NSEC_PER_SEC;
-   add_timer(timer);
+   timer->kernel_timer.expires = jiffies + (ns * HZ + NSEC_PER_SEC - 1) / NSEC_PER_SEC;
+   add_timer(&timer->kernel_timer);
 }

 void bm_timer_expire_at(bm_timer_t* timer, uint64_t ns)
 {
-   timer->expires = (ns * HZ + NSEC_PER_SEC - 1) / NSEC_PER_SEC;
-   add_timer(timer);
+   timer->kernel_timer.expires = (ns * HZ + NSEC_PER_SEC - 1) / NSEC_PER_SEC;
+   add_timer(&timer->kernel_timer);
 }

 void bm_timer_cancel(bm_timer_t* timer)
 {
-   del_timer(timer);
+   del_timer(&timer->kernel_timer);
 }

 // Event waiting
diff -u blackmagic-io-10.9.10a2.orig/bm_util.h blackmagic-io-10.9.10a2/bm_util.h
--- blackmagic-io-10.9.10a2.orig/bm_util.h   2018-01-08 02:46:14.000000000 +0100
+++ blackmagic-io-10.9.10a2/bm_util.h   2018-01-31 23:27:01.543121803 +0100
@@ -112,8 +112,12 @@

 // Timer
 struct timer_list;
-typedef struct timer_list bm_timer_t;
 typedef void (*bm_timer_callback_t)(void*);
+typedef struct {
+   struct timer_list kernel_timer;
+   bm_timer_callback_t callback;
+   void *data;
+} bm_timer_t;

 uint64_t bm_absolute_time_us(void);

Finally rebuild the kernel packages and your Decklink/Intensity card should work.
sudo dpkg-reconfigure desktopvideo

[Julusian]

@walterav1984 I believe that ubuntu calls the drivers nvidia-384, even though once installed they claim to be 390... I did a fresh install of 18.04 last night, so went straight for latest everything, so didn't have any issues with opengl or blackmagic versions. The Blackmagic 10.9.11 driver works against the 4.15 kernel without any changes making that easier.
Yes, I too installed the artful version of docker. I tried the ubuntu repo version first, but its too old.

I havent actually tried using caspar, all I know is that it starts up without error with the default config so far. I was experimenting with running it in docker, but that does not look like a realistic option as all the info I can find on doing it says that the docker container needs the nvidia driver inside the container, and it needs to be matching version to the host which kills image portability.

[walterav1984]

Thanks for mentioning docker version, and that BMD DesktopVideo driver 10.9.11 doesn't need patching for 18.04.

I think you're correct about nvidia-384, its now a meta package however it says 390.25(Nvidia settings info) last week there was a real nvidia-390 which gave boot loop errors.

PS: tried to build your docker version but it says:
https://github.com/Julusian/CasparCG-Server/commit/2eb56d7e5ec8d2b6473ff14591b61520fb75cfbf#diff-38d13f5bb32606e35ea06563ff190d35
manifest for casparcg/base:1.2.0 not found

[Julusian]

Ah yes, I had to rebuild the base image, so bumped the version. I have just pushed another commit which adds a build-base-images script to help build that as well as ffmpeg and boost.

[dotarmin]

I started having a play with building for 18.04 last night. and have something that runs.

Keep up the good work @Julusian ! 👍 I think focusing on 18.04 is the right way and LTS version from there and onwards.

I hit the same error on attempting to run, which was just some files that weren't being copied.

I did try this, I located the libsoftokn3.so and copied but it didn't help in my case unfortunately.

[dotarmin]

Could you also start a "vote/poll" on which desktop environment to run or just ubuntu server with a window manager? Because I like @Julusian his notice that openbox gave him more performance than Unity>Gnome3 will give by default. Although I'm not sure openbox used vsync...

@walterav1984 Agree but we have to wait just a bit with these kind of things and focus on getting one stable Linux version and then take it from there. But the idea is great! Can you just create a plain issue with this idea so it won't be forgotten.

/Armin

[walterav1984]

@Julusian
Your latest pull https://github.com/Julusian/CasparCG-Server/commit/2f7078a773dc1e09e1adb44096963be052011c31 fixes the manifest error and builds, when running tools/linux/build-base-images and than tools/linux/build-in-docker.

However extracting the docker build or run interactive from dockerbuild still doens't work for me.
Sadly no error messages, for extracted docker build just stalling cursor after ./run.sh.

Concerning running the dockerbuild will introduce more challenges with GPU driver shipment/compatibility and maybe decklink although both have been done according to other github projects.

[Julusian]

@walterav1984 There is a typo in the build-in-docker script where it tags it as casparcg/server3, so you are likely extracting the wrong image... I've amended the commit that broke that.

[walterav1984]

A quick rebuild with removing the "3" character shows a new error when run both with launch-interactive as extracted docker build, Type "q" to close application. will try a total rebuild if that might help.

[Julusian]

That doesn't look like an error, just normal startup to me, this is what I get

julus@julus-archimedes:~/caspar-build-dev/casparcg_server$ ./run.sh
Type "q" to close application.
Logging [info] or higher severity to log/

[2018-03-07 12:52:22.084] [info]    ############################################################################
[2018-03-07 12:52:22.085] [info]    CasparCG Server is distributed by the Swedish Broadcasting Corporation (SVT)
[2018-03-07 12:52:22.085] [info]    under the GNU General Public License GPLv3 or higher.
[2018-03-07 12:52:22.085] [info]    Please see LICENSE.TXT for details.
[2018-03-07 12:52:22.085] [info]    http://www.casparcg.com/
[2018-03-07 12:52:22.085] [info]    ############################################################################
[2018-03-07 12:52:22.085] [info]    Starting CasparCG Video and Graphics Playout Server 2.2.0.3344 2f7078a77 Beta 1
[2018-03-07 12:52:22.086] [info]    Initialized ffmpeg module.
[2018-03-07 12:52:22.086] [info]    Initialized oal module.
[2018-03-07 12:52:22.086] [info]    Initialized decklink module.
[2018-03-07 12:52:22.086] [info]    Initialized screen module.

[walterav1984]

@Julusian
OMG it runs, I had to use sudo ./run.sh instead of ./run.shand its playing AMB and CG1080i50 demo loops!!!

Update:I'm running the docker extracted files!

[walterav1984]

BMD Intensity 4K hdmi with decklink consumer @720p5000 or PAL also plays the demo videos with audio, just like the screen consumer.

[dotarmin]

Okey, it seems obvious that community together agreed on skipping the support for Linux in CasparCG Server 2.2.0, and specifically Ubuntu 17.10 to focus on 18.04 LTS instead.

We could however have some sort of pre-evaluation period with the work Julian has done with making it run under 18.04 and building with Docker. If we do that, we have to be clear it's not supported by the community but is available for those users who wanna give it a shot.

@Julusian Can you do a PR?

@Julusian @ronag comments?

[baltedewit]

having to run with sudo sounds like a bad idea. there's probably a file permission issue when copying from docker that should be fixed.

skipping the support for Linux in CasparCG Server 2.2.0, and specifically Ubuntu 17.10 to focus on 18.04 LTS instead

it's not that it doesn't work, Linux just comes with a big old disclaimer: YMMV. my build for Debian is humming along quite nicely, and by building outside docker I didn't get any weird behaviour like missing db's and having to run through sudo. but it does take some knowledge about the platform to get it set up.
we do need more time to figure out how to set up these releases though, especially now that we're building through docker. Would maintaining a package be an option?

[dotarmin]

@baltedewit having to run with sudo sounds like a bad idea. there's probably a file permission issue when copying from docker that should be fixed.

Docker daemon runs as root and needs therefore sudo. It is possible to to manage Docker without sudo permissions by creating a Docker group and adding your user it. More information here.

it's not that it doesn't work

Agree, I don't blame Linux or so just to be clear 😄, but putting focus on LTS versions is still the main goal in my opinion.

[baltedewit]

Docker daemon runs as root and needs therefore sudo.

Docker is only used at compile time, runtime should _never_ require sudo unless you're doing things that require full access to the os (like the docker deamon).

[dotarmin]

Docker is only used at compile time, runtime should never require sudo unless you're doing things that require full access to the os (like the docker deamon).

You're absolutely right, but to manage Docker and run Docker commands such as docker build it requires sudo by default and as you say, it shouldn't ask for it at runtime. I misunderstood the original sentence, sorry. I thought it was related to compiletime and not runtime.

[Julusian]

Can you do a PR?

Yes, I shall do so soon.

Docker is only used at compile time, runtime should never require sudo

Im not sure why sudo was needed. I don't think I needed to do that so that is very strange.

it's not that it doesn't work

Yes, I too primarily run debian so will likely end up doing a lot of dev and testing work from there. But for any production deployments I shall be looking to whatever we support.
One thing to note is that we build against a lot of system libraries now, which is a potential source for differences/incompatabilities/bugs as each distro will have slightly different versions of each library.

Has anyone tried to run the 18.04 build on an older version of Ubuntu? I am curious to know if it runs.

by building outside docker I didn't get any weird behaviour like missing db's and having to run through sudo.

I bet that if you were to copy the build to another machine you would have that same missing db issue. It likely only works because it was a package you had to install to build it, but isnt being bundled up with the result.

Would maintaining a package be an option

Do you mean a full apt repo, or just packaging as deb files?

[k0d]

@dotarmin docker requires sudo by default, so you should add your user to the docker group (something like this... sudo usermod -a -G docker dotarmin"

In terms of 18.04 support instead of 17.10....I am in total support of that...but as I didn't want to install it on my laptop when I was doing the development and testing recently, I targeted 17.10. I don't know exactly when...but I will upgrade my laptop to 18.04 as soon as I possibly can.

There seems to be a docker image ready for 18.04 (devel)...so I'll look into testing out the build asap.

[baltedewit]

Do you mean a full apt repo, or just packaging as deb files?

I was thinking of a full apt repo, but I don't know what exactly the implications are. I think a deb file would still suffer from the issue that we're not using system versions of ffmpeg and boost, right?

[premultiply]

A deb-Package would be the best solution for users.

[walterav1984]

@premultiply
Make that a snap, even a dump/classic snap can be installed from the ubuntu snap store without adding ppa's or manually downloading debs. But snaps are kinda hard to make I noticed if you would like them to run besides the platform your building on...

[Julusian]

The more I think about it, the less sure I am on which way to go.
I believe an apt repo isn't much more difficult than just building deb files (I did it once in the past). And it needs to be hosted somewhere.
Building deb files sounds reasonable, but there are some questions on to do it:

• put all the required libs into the deb
• make the deb reference other packages and require them to be installed (really ties us to the one release). This will also make it a lot harder to install offline. any custom built libs would need to be embedded in this, or put into their own deb.
• what about people who want multiple versions? It could be done with a new package name for each version (casparcg-2.2/casparcg-2.3/casparcg-dev) and a meta-package to install the latest branch like gcc, kernel and many other packages.
  For a dev/testing environment, being able to have many versions is very useful, but I can see that debs/snap would be easier for production.

I had a quick look into snap and flatpak, and from what I could tell it is pretty much impossible to control the version you have installed. you are supposed to have latest which I dont think that is a very good way to go for us, as it means the user has no control over their version.

[Julusian]

@walterav1984 #917 Could your issue when not running as root be this?

[baltedewit]

snap / flatpack

They're containerized (kind of like docker) so I don't think we can get access to niche drivers like Decklinks. Our implementation for the mixer _might_ also not work, depending on how they've implemented gpu acceleration.

put all the required libs into the deb

That's basically like putting CasparCG together with boost and ffmpeg into a tar file, except you get to access system directories and auto-execute an install script.

make the deb reference other packages and require them to be installed

That's mostly what I'm thinking of. So long as we don't use system versions of boost and ffmpeg we require other "packages" anyway, so it would make sense to me to at least provide a relatively easy way to install them. An apt repo would make it even easier for the user, I think.
Not certain if it would make it a _lot_ harder to install offline. A flash drive with the required deb files would do, right? Currently we don't really document dependencies either, so you go from being in the dark to a sort of dark grey.

multiple versions

Yeah, my idea is basically meta packages. It's pretty convenient I think. Also, quite easy to symlink against if you want to change versions. Concerning dev builds: we could still contain those within a single folder I think?

Question remains: how much value does it add over a script that executes the docker commands and extracts the builded files?

• We take away the building from the user's machine
• We have a mechanism for providing updates (would require full apt repo)
• Hopefully it makes the install easier for the user

Are these worth the extra effort of figuring out dependency management (I'm thinkin apt repo, providing libraries in the form of ffmpeg-3.4-gitrev) and hosting the apt repo (yes, since we already have to host files anyway)

but there are some questions on to do it:

We would also have to think about where the config file and media directories would live.

[k0d]

We are statically building against ffmpeg and boost so that simplifies things.

There is already a script to extract CCG to a folder from the Docker image. It takes too many libs just now...which causes the nss issue...but I will be updating that soon.

I think the plan to create a single folder is the best way still, it should be quite portable that way.

I have plenty of experince with apt/deb packaging, so if we find thats the best route...we can do that.

[walterav1984]

@Julusian
The sudo requirement probably came from running all the build scripts and extraction with sudo since I didn't put my normal user to the docker group. Doing a chmod -R 777 on the extracted caspar-server folder makes casparcg run without sudo for me.

make the deb reference other packages and require them to be installed

This makes it indeed tight to one ubuntu release, but when done right/perfect a snap can be build to support all releases!

@baltedewit

About snap/flatpack They're containerized (kind of like docker) so I don't think we can get access to niche drivers like Decklinks.

Can't talk for flatpack but "shotcut" snap runs with decklink output for me and can use the gpu, but currently its a classic/dump snap.

Currently we don't really document dependencies either, so you go from being in the dark to a sort of dark grey.

For a dump/classic snap it also means dependency wise that its more gray than black but still not white. A perfect snap with exact dependencies would not only run on multiple ubuntu releases but also other linux distros. But its a lot of effort but not if there is a fully dependent deb has been made before.

We would also have to think about where the config file and media directories would live.

This is a very important point when standardizing into any packaging/container format.

Multiple versions for snap are still on the roadmap its called parallel installs.. It has been referenced again just a month ago by a developer with coming soon.

Without the need for a custom repo or ppa its possible to make separate snap builds for each github branch stable/development and auto import those gits into launchpad and that will autobuild the snap and publish on snapcraft into the ubuntu snap store?

[Julusian]

Not certain if it would make it a lot harder to install offline.

So I had a play around with deb packaging last night (https://github.com/Julusian/casparcg-deb). Turns out that the dependency tree if we go for referencing libs is pretty large. It's not yet referencing any CEF deps and maybe overestimates what is needed for ffmpeg (I made it reference libavformat out of laziness, even though we use our own)
My conclusion from this is that it would be way too painful to use for an offline install.

The following additional packages will be installed:
  dbus fontconfig fontconfig-config fonts-dejavu-core gcc-8-base
  i965-va-driver krb5-locales libaacs0 libapparmor1 libasound2 libasound2-data
  libass9 libasyncns0 libavcodec57 libavformat57 libavutil55 libbdplus0
  libbluray2 libbsd0 libc-bin libc6 libcairo2 libchromaprint1 libcroco3
  libcrystalhd3 libdatrie1 libdbus-1-3 libdrm-amdgpu1 libdrm-common
  libdrm-intel1 libdrm-nouveau2 libdrm-radeon1 libdrm2 libedit2 libelf1
  libexpat1 libfdk-aac1 libflac8 libfontconfig1 libfreeimage3 libfreetype6
  libfribidi0 libgcc1 libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-bin
  libgdk-pixbuf2.0-common libgl1 libgl1-mesa-dri libgl1-mesa-glx libglapi-mesa
  libglew2.0 libglib2.0-0 libglib2.0-data libglvnd0 libglx-mesa0 libglx0
  libgme0 libgomp1 libgraphite2-3 libgsm1 libgssapi-krb5-2 libharfbuzz0b
  libicu-le-hb0 libicu60 libilmbase12 libjbig0 libjpeg-turbo8 libjpeg8 libjxr0
  libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0 liblcms2-2 libllvm5.0
  libmp3lame0 libmpg123-0 libnspr4 libnss3 libnuma1 libogg0 libopenal-data
  libopenal1 libopenexr22 libopenjp2-7 libopenmpt0 libopus0 libpango-1.0-0
  libpangocairo-1.0-0 libpangoft2-1.0-0 libpciaccess0 libpixman-1-0
  libpng16-16 libpulse0 libraw16 librsvg2-2 librsvg2-common librtmp1
  libsensors4 libsfml-graphics2.4 libsfml-system2.4 libsfml-window2.4
  libshine3 libsnappy1v5 libsndfile1 libsndio6.1 libsoxr0 libspeex1
  libssh-gcrypt-4 libssl1.1 libstdc++6 libswresample2 libtbb2 libthai-data
  libthai0 libtheora0 libtiff5 libtwolame0 libva-drm2 libva-x11-2 libva2
  libvdpau1 libvorbis0a libvorbisenc2 libvorbisfile3 libvpx5 libwavpack1
  libwebp6 libwebpmux3 libwrap0 libx11-6 libx11-data libx11-xcb1 libx264-152
  libx265-146 libxau6 libxcb-dri2-0 libxcb-dri3-0 libxcb-glx0 libxcb-present0
  libxcb-render0 libxcb-shape0 libxcb-shm0 libxcb-sync1 libxcb-xfixes0 libxcb1
  libxdamage1 libxdmcp6 libxext6 libxfixes3 libxml2 libxrandr2 libxrender1
  libxshmfence1 libxvidcore4 libxxf86vm1 libzvbi-common libzvbi0
  mesa-va-drivers mesa-vdpau-drivers shared-mime-info ucf va-driver-all
  vdpau-driver-all xdg-user-dirs

5 upgraded, 160 newly installed, 0 to remove and 30 not upgraded.

From this I am leaning towards packaging the deps into the deb. Makes it large and technically the 'wrong' way to do it, but is a lot simpler for us to maintain and for the user to install.
Doing it all in one with apt/ppa doesn't feel like it has much benefit over a tar/zip file though, except for making the install and update process slightly easier.

Either way, I think we should be releasing tar/zip files alongside whatever else we want to do.

We would also have to think about where the config file and media directories would live.

For a start the directories should probably be relative to the config file, rather than the current working directory, but that should probably get its own ticket.

It takes too many libs just now...which causes the nss issue

@k0d Does it take too many? I had a look at the script and it copies everything that the exec is linked against, and the nss lib is not referenced there I assume because it is runtime referenced.

I have plenty of experince with apt/deb packaging, so if we find thats the best route...we can do that.

Great, I am just googling and trying things currently :)

Multiple versions for snap are still on the roadmap its called parallel installs..

I think we need to wait for this to be done before we can really consider snap, but it is good to know it is in the plans.

[dimitry-ishenko]

Having switched my laptop to Ubuntu recently (been using Gentoo for over 10 years), in my extremely humble opinion, PPA is the way to go. It can even be hosted on launchpad.net.

My conclusion from this is that it would be way too painful to use for an offline install.

@Julusian there are ways to make it less painful:
http://www.tuxradar.com/answers/517
https://stackoverflow.com/questions/22008193/how-to-list-download-the-recursive-dependencies-of-a-debian-package

If you are planning to target LTS versions (which is the way to go), you should be able to build against system installed ffmpeg and boost and will only need to update once every 2 years. Or is there another reason for not using sysdeps? Worst case, you can provide needed deps in the PPA

Are these worth the extra effort of figuring out dependency management

It's actually not that hard either :smile: I've done it few times for my Gentoo builds.

[Julusian]

there are ways to make it less painful:

Ooh, that is good to know. Im sure ill want that myself in future.
While that is less painful, its still not ideal. It does rely on the user having a machine running the same release machine available to download the deps on.

you should be able to build against system installed ffmpeg and boost

Ideally yes, but we are using a newer version of ffmpeg than is available in the repos, and potentially compiled with different options (to match what we use on windows).
The version of boost in 18.04 is too old and doesn't contain some asio components that we are using, so I cant see that changing.

We are statically building against ffmpeg

@k0d can we dynamically link against ffmpeg? The media scanner will need ffmpeg and ffprobe, so it might be a better option.
That also means we can seperate out a ffmpeg deb and media-scanner deb more cleanly

[dimitry-ishenko]

The version of boost in 18.04 is too old and doesn't contain some asio components that we are using, so I cant see that changing.

@Julusian oh yes, this brings up 2 things:

1. Asio can be used as a standalone library: https://think-async.com/Asio/AsioStandalone without any dependency on boost.
   I've actually done this in my Firmata implementation: https://github.com/dimitry-ishenko/firmata.
   If you use it as a standalone lib, you can just add it as a submodule to the server git repo and have no runtime deps.

2. Last time I perused the server code, I've noticed many boost usages that are now available in the std library (ie, c++11). If somebody goes through the code, they should be able to reduce dependency on boost and perhaps it won't matter that the boost version in 18.04 is not the latest.

Again, I have successfully compiled the server on Gentoo Linux against system version of boost, which was different than the bundled one. I've even sent a PR at one point for that, but it bitrotted and got lost in the midst of things.

PS. I am willing to be that somebody to go through the code, but my workload at the moment is already more than I can handle, so it might take me a while.

[Julusian]

I have raised a PR for the build portion of this #919
It would be good to not close this issue yet, until there is a plan for packaging (and potentially using system boost)

[dotarmin]

@k0d docker requires sudo by default, so you should add your user to the docker
group (something like this... sudo usermod -a -G docker dotarmin"

I know? :)

@julusian The more I think about it, the less sure I am on which way to go.

Keeping it simple is the key here. We have also to consider that users who's going to use the Linux build knows the OS it-self and can manage to run it self or by having clear instructions on how to do so.

I don't see a reason for going for a deb-package when we have Docker for building it on Linux and a way to extract the package from Docker. It's just matter of having it well documented for other users and taking the "step".

Regarding the builds, when we support 18.04 LTS and continue to use Docker for building we could just upload an archive file just as we do for Windows builds and have it located under builds.casparcg.com.

@julian It would be good to not close this issue yet, until there is a plan for packaging (and potentially using system boost)

Wouldn't it be enough with just a tar file that is unpacked and then execute the run.sh script? I presume Linux users now how to execute a shell script? Even if the intention is really nice with having a deb-package or even an apt-package it could be a nightmare to maintain worst case scenario. Just some thoughts.

[k0d]

I love packaging...apt repos are awesome...but also require more maintenance to support multiple OS versions. So I think that we should just use the archive file system that we do already. We statically link to FFMPEG and Boost to keep it as simple as we can...and it really doesn't matter as much about the media-scanner ffmpeg version as it does for the CCG server, so I think it's fine to not require the same.

I believe the system that we've started to setup now, using Docker to build it and being able to just extract the build out with the included script, is the way we should go forward. Of course it still needs work...and targeting 17.10 like we do now is only a temporary thing until we get round to 18.04 in a very short time.

@Julusian the script does take too many libs for certain....it causes the NSS issue for sure...I just didn't have time to look into which I shouldn't include. It probably needs a specific list instead of just automatically grabbing all libs.

If someone wants to test running it without any libs in the ./lib folder...and then adding them one by one until it runs....please do so and provide a list to me, then I'll fix it. Otherwise it'll take a bit longer until I find time to do it...

[dimitry-ishenko]

IMHO ppa + deb pkg(s) is the simplest option from users' perspective. All they would need to do is add-apt-repository ... and apt install .... This way they will also be able to get updates and bugfixes with apt update/upgrade.
From developer's perspective, if you are planning to only target LTS, you only need to worry about single platform.
Just my 2 cents.

[k0d]

@dimitry-ishenko I can agree that being able to apt update/upgrade is nice...but isn't a large part of the user-base of this project in a different situation? "If it works, don't fix it" applies very much to broadcasting and accidental upgrades might be too big of a risk. I probably can't describe this 100% as I don't have long time experience in it...

We have to consider maintenance of the repos etc also....an unmaintained repo is worse than a slightly more complicated install process...and really...downloading a .tar.gz file and opening it...is not really that hard right??

[dotarmin]

@k0d I believe the system that we've started to setup now, using Docker to build it and being able to just extract the build out with the included script, is the way we should go forward. Of course it still needs work...and targeting 17.10 like we do now is only a temporary thing until we get round to 18.04 in a very short time.

I agree!

@dimitry-ishenko Just my 2 cents.

Well, it's two good cents then for sure! :)
But who will maintain the repo? I love repos myself and would like that for the project but the issue with maintaining that is still a fact that could potentially get out of scope easily.

[ronag]

We've merged 18. I think this can be closed?

[dimitry-ishenko]

@kod @dotarmin you guys are funny. I feel like you are trying to put the onus of dealing with installation issues on the end-users. :stuck_out_tongue_closed_eyes:

As a 10+ year Gentoo Linux user and systems administrator (running over 400 Gentoo boxen at our various facilities) I had a fair share of manually installing and patching software. I love Gentoo and I loved doing that, but recently I've realized that I spend too much time tweaking things, because they don't often work out of the box.

Creating .deb package is not that much more work than making .tar.gz.

A lot of users are afraid of terminals and command prompts. There are tons of GUIs for installing debian repos and packages. By making it as easy as possible for the users to install the package, even at the expense of slightly more work for the developers, you will be able to attract more end users.

[dimitry-ishenko]

Honestly, I think you will end up spending less time maintaining the repo and making it "just work" for the users, than dealing with installation problems from a .tar.gz archive.

[k0d]

@dimitry-ishenko

1. Feel free to contribute the scripts to build the debian package(s) and submit a PR
2. IMHO: Installation issues (for this type of project) are really the responsibility of the end-user....and if they want help...that's what the forum is for. We will provide installation instructions of course, and they will likely work if the end user follows the guidelines, which probably need to be updated....but will likely mean very similar hardware and a non-custom install of Ubuntu.

@ronag I will do a test of the 18.04 updates...then give my input, but feel free to close this issue and I'll open another if I find anything that needs fixing.

[dotarmin]

@dimitry-ishenko you guys are funny. I feel like you are trying to put the onus of dealing with installation issues on the end-users. 😝

I'm not saying anything haha!

A lot of users are afraid of terminals and command prompts.

I can understand that but what's the purpose of using Linux if not using the terminal? 😃

[dotarmin]

This issue is fixed We will support 18.04 and a PR has been merged just as @ronag says. We should go for the archive files for now, this could on the other hand change in the future.