Mumble: Backwards-compatibility strategy

[Krzmbrzl]

The current policy is to keep compatibility with the oldest not yet EOL LTS version of Ubuntu (at this time this is Ubuntu 16.04). This means that we will have to make sure that Mumble compiles on these old systems which restricts us in what we can use of 3rdParty libraries (e.g. OpenSSL - see #3918).

This thread shall serve as a discussion whether or not this strategy should be maintained or if it might be a better idea to change it.

---

Until this is the version that is supported by the oldest still supported Ubuntu release, it'll probably take quite some time. And before that we can't make use of it.

This might be a question better suited to its own thread, but what is the rationale for this? Is there data to suggest that a significant number of people use Mumble on the oldest supported non-ESM Ubuntu which means that Mumble is limited to technology that can be more than 5 years behind? I really question the value of this sort of legacy support when it comes to things like this and #3918. Wouldn't it benefit more people to move the baseline so that newer technologies can be used? Not trying to insult the work put in this pull request; just trying to offer some food for thought.

_Originally posted by @TredwellGit in https://github.com/mumble-voip/mumble/pull/4076#issuecomment-615933677_

[Krzmbrzl]

Actually I don't know the reason. It was this way when I joined the team and I didn't question it.

@davidebeatrici @Kissaki do we have some user-stats that include data about how many people actually use our backwards-compatibility to its full extend?

[Kissaki]

I disagree with the limitation that was concluded in #4076. We have always Qt-version-define-guarded newer Qt features and Qt changes. I would also want to do that with a Markdown interpreter if Qt provides one. At the very least to look into what it provides, and what the differences to a custom one would be, or if we can use the new one in older versions (by duplicating/copying it).

As for dropping backwards compatibility I agree that should it be a goal to drop old versions when no longer in use/supported. I am not sure what is being questioned here though. It is hard to answer and talk about generic, unconcrete topics or claims.

From my understanding the super old CentOS environment we use for our static binary package (server) has always been a big blocker for dropping older functionality even if we would have done or at least considered doing so otherwise.

We collect anonymous statistics but atm it just lands in a database and the website presenting it was somewhat broken and removed/dropped.

I do think we can consider and target dropping stuff as soon as (popular) distros drop support for them. Like I have advocated ignoring Windows 7 now.

I presume the CentOS environment for our static binaries is still a thing though, and still a blocker. There were technical reasons why we could not upgrade. But maybe there is a better solution to be found here.

[Krzmbrzl]

We have always Qt-version-define-guarded newer Qt features and Qt changes.

Changes yes, but I don't think I have seen this being done to an entire feature so that that particular feature is only available if the program was compiled with a specific minimum Qt version...

I am not sure what is being questioned here though.

The question is: Should we remove support for older versions earlier (aka not regarding the oldest Ubuntu LTS as the backwards compatibility target). This would allow us to move to a newer cpp standard and to more recent OpenSSL versions which would probably eliminate the problems encountered during the discussion about the new crypto stuff.

We collect anonymous statistics but atm it just lands in a database and the website presenting it was somewhat broken and removed/dropped.

Is the source code of the old system available somewhere so I (or someone else) could try fixing it up? 'cause I think such a webpage would be really interesting (not only for the present issue).

There were technical reasons why we could not upgrade.

I suppose @davidebeatrici knows more about this?

[Kissaki]

No, we do not have source code. I did want to work on it, but only briefly did so in the past. There is still some stuff to do on the new website, but possibly showing some statistics or at least having it for us is something I would like to do afterwards. The storage is undocumented. We post to our service and it is put into a MySQL database. The first step is to find out and document that schema.

It was mainly mkrautz who worked on it in the past. Maybe davide knows something.

[toby63]

@Krzmbrzl @Kissaki :
Regarding Backwards Compatibility:
I favor using newer software instead of supporting very old distros.
Like I said in #3918 comment, older distros could (at least partially) use backports to still support the use of mumble.

Another idea: Some (or even many) software projects maintain different versions of their software, so they have one tree for backwards compatability, but I guess for mumble that would be to demanding and also maybe problematic for two-way use.
Nonetheless you could think about that, if basic functionality is still provided in "backwards compatability" versions, users of these versions could still use it, without the new stuff (UI, extended functionality etc.).

Regarding the analysis of anonymous statistics of OS:
I would not rely on that.
Or do you get data about the amount of users who disabled the option to provide information?

@Krzmbrzl : Could you rewrite the first post, so more people understand what this is about at first sight?

[Krzmbrzl]

Could you rewrite the first post, so more people understand what this is about at first sight?

Done

Like I said in #3918 comment, older distros could (at least partially) use backports to still support the use of mumble.

This kinda feels like a patch-up that we should not rely on. If we decide to no longer support such old distros, then it would of course be nice if users of those distros could use backports to still use Mumble.
However I would not want to make our decision on that matter based on the assumption that such backports will be readily available for Mumble to work on older distros.

On a personal note I think I tend to agree with the perspective of preferring to drop the support for distros older than let's say 3 years or something like that in favor for being able to use newer features more readily.

As a workaround to use Mumble on older distros, I think we could provide static binaries in form of snap packages or AppImages. I think these should run on any dstribution refardless of what is in their repositories...

[toby63]

@Krzmbrzl

Done

Thank you.

This kinda feels like a patch-up that we should not rely on. If we dice to no longer support such old distros, then it would of course be nice if users of those distros could use backports to still use Mumble.
However I would not want to make our decision on that matter based on the assumption that such backports will be readily available for Mumble to work on older distros.

Well thats a difficult discussion.
In case of openssl for example the situation should be clear, 1.0.2 is EOL, so it is imo the responsibility of the distros to switch to a newer version and you (the mumble developers) should not use EOL software.
In general it is always a question of competence, should the distro include software or should the software developers respect the distros decision?
Its an everlasting problem, for that reason snap and appimage exist, because developers didn't always want to wait or discuss everything with package maintainers/distros.
My personal oppinion is that developers should keep a hard but reasonably fair line, that means push new software, but not too hard.
Always considering the effort for package maintainers; in case of openssl it is also low effort (few dependencies etc.), so it can be pushed imo.

preferring to drop the support for distros older than let's say 3 years [...]

Yes, if you look at the Mumble package in ubuntu for example, we see that bionic has at least v1.2.19 included, but xenial is completely outdated (v1.2.12-1ubuntu1).
Bionic also has openssl v1.1.1-1 included.
I would advocate for bionic (also a LTS-Release) being the new line for backwards support, so as a general rule, always the second available LTS-version of ubuntu.
Update: see discussion below.

we could provide static binaries in form of snap packages or AppImages

Use snap packages please, because those can receive "automatic" updates.
A snap package of mumble is by the way already available: https://snapcraft.io/mumble
See also github page.

[davidebeatrici]

Now that 20.04 is out, 18.04 will be our baseline.

[Krzmbrzl]

The point with this discussion is to potentially find a solution i which we won't use the oldest Ubuntu LTS as a baseline, 'cause in a few years we'll have the same issues with 18.04 that we're currently having with 16.04 :point_up:

[toby63]

@Krzmbrzl @davidebeatrici

The point with this discussion is to potentially find a solution i which we won't use the oldest Ubuntu LTS as a baseline, 'cause in a few years we'll have the same issues with 18.04 that we're currently having with 16.04

I agree, five years (which is the standard support time for ubuntu LTS releases) are too long.
Xenial Xerus (16.04) started in October 2015 (ReleaseSchedule).
And now you want to switch to Bionic (18.04) which started 2017 (ReleaseSchedule Bionic).

This means i have to revert my former comment:

I would advocate for bionic (also a LTS-Release) being the new line for backwards support, so as a general rule, always the second available LTS-version of ubuntu.

As this would mean waiting for five years again.

I recommend backwards support for maximum 2 (two) years.
This is still like a decade in software ages, but it would be reasonable.
So instead of looking at a distributions release circle, you look at the software you use and implement features that are still supported by two year old versions of this software.
But beware:
The most important thing is to ensure software security:
I want to qoute your website:

Stay private and secure
always encrypted communication

This in ensured by:

• using up-to-date software
• which includes the newest fixes

Everything else, including backwards support has to be secondary.

So if an old feature/API etc. is dropped, this should not mean to not update the software depencies, it must mean to drop support for older versions and making an exception of the two-year-backwards goal.

Additional Notes:

• QT is dropping LTS-versions for normal users (official blog):

Long-term-supported (LTS) releases and the offline installer will become available to commercial licensees only

Edit: As some people see bold text as offensive, I removed the bold text.

[Krzmbrzl]

So if an old feature/API etc. is dropped, this should not mean to not update the software depencies, it must mean to drop support for older versions and making an exception of the two-year-backwards goal.

This I disagree with. If we decide on a strategy then we have to stick to that strategy. Otherwise SysAdmins won't be able to plan with the software. If you install a software that guarantees support for time x, then you really should get that support for that time. No exceptions there.

My preference would however be to decrease this time frame to something around 2-3 years. And as I suggested above: For older version we could simply provide static builds (in form of AppImages and/or snap packages).

[toby63]

@Krzmbrzl

This I disagree with. If we decide on a strategy then we have to stick to that strategy. Otherwise SysAdmins won't be able to plan with the software. If you install a software that guarantees support for time x, then you really should get that support for that time. No exceptions there.

Well it's all your decision.
But normally there is preparation time, so if a software drops features or API this is mostly foreseeable, but maybe not for two years, maybe just for 6 months or 1 year.
And once again I want to point out the shared responsibility aspect:
So the Package Maintainers, SysAdmins (and the developers of third party software) have to do their part.
If they are e.g. too stubborn to upgrade/patch or adapt, this is not necessarily your no.1 concern, but security of all users is the no.1 concern imo.

For older version we could simply provide static builds (in form of AppImages and/or snap packages).

• This could also be a solution for the situations above.
• But I am still a bit sceptical, because this approach is only secure, if there are regular dependency updates (provided by you or the snapcrafters).
  Nonetheless it fixes the problem for outdated dependencies on other platforms.

[Kissaki]

We discussed this (backwards compatibility strategy) yesterday in our team meeting.

We want to move forward faster, without being limited as much by old OSes/distros.

We want to support the latest Ubuntu LTS release. If there are older LTS releases the package maintainer (of the Ubuntu package) can still backport important fixes.

If problematic we want to drop our ermine static server binary package. Nowadays there are multiple app package/image technologies which we should embrace/use and focus on instead. Back when we introduced the static binaries those were not a thing. Nowadays it may no longer be necessary. (If it still is we will find out from feedback and we will have to tackle it anew, and hopefully with an updated and simpler technology stack.)

[toby63]

We want to support the latest Ubuntu LTS release.

That means Ubuntu 20.04 LTS (Focal Fossa) right now?
So we would have a backwards-strategy of two years?
(Because every two years a new LTS-Release is released)

If problematic we want to drop our ermine static server binary package.

:+1:

Nowadays there are multiple app package/image technologies which we should embrace/use and focus on instead.

Curious which one you prefer.
I remember a discussion about flatpaks (in a forum of another software project), someone posted this link:
https://flatpak.org/faq/#Can_Flatpak_be_used_on_servers_too_

[Krzmbrzl]

That means Ubuntu 20.04 LTS (Focal Fossa) right now?
So we would have a backwards-strategy of two years?

Yes and yes