Vm: Talk with HTTP/2

[tiagoefreitas]

The installer says Talk doesn't work with HTTP/2, but this can be fixed according to this :

https://github.com/nextcloud/spreed/issues/602#issuecomment-384468807

Its a configuration issue, so it may not be fixed automatically just by upgrading Talk.

[enoch85]

As @Schnookeuz pointed out, using the mpm_event worker in combination with php-fpm is the recommended way of running Nextcloud Talk.

Problem is, we don't run php-fpm. The VM is built and designed to work for everything. Adding HTTP/2 in general may break other things. That's why we made it a choice and not per default in the installer scripts. But if you as a end user want to run HTTP/2, then it's up to you.

I already use HTTP/2 in my Nginx Reverse Proxy so I don't need it on the backend. That's one of the reasons it's harder for me to try it live. But if you find a way to activate it without breaking anything else, then be my guest and write a PR. :+1:

Would be awesome, since HTTP/2 is better.

[shadoxx]

@enoch85
I can confirm that running Apache 2.4 with php-fpm doesn't break anything else related to Nextcloud. I've ran a heavily used production instance for more than six months in this configuration without issue. It is light on resources and blazing fast.
I'd be happy to make a PR once I've understood how the VM is built. It looks like the script located in the root of the repo called nextcloud_install_production.sh does the heavy lifting for getting everything configured?

[enoch85]

@shadoxx Thanks for your offer. I'm not convinced though. Same as HTTP/2 I don't know of any of the scripts will break on the change to php-fpm and I don't have time to test all the scripts.

You are very welcome with a PR though. I understand if it will take time as you need to check all the scripts and all the lines.

It looks like the script located in the root of the repo called nextcloud_install_production.sh does the heavy lifting for getting everything configured?

Yes that's correct.

The main design is that everything is built on the install script and the setup script. That's what this VM started with. Since then everything is divided into smaller pieces, and a while ago we also added lib.sh where all the $vars and functions are stored so that they can be fetched upon every load.

[shadoxx]

You are very welcome with a PR though. I understand if it will take time as you need to check all the scripts and all the lines.

Understood. So, to test this script as is, do I just spin up a fresh Ubuntu VM and run it? I just learned about this project through the reference to my comment in another issue. After a cursory look, it does look like it'll take me a bit of time to get everything working. Definitely do-able though.

[enoch85]

Definitely do-able though.

That's great news!

Can yo tell me what the biggest difference between php-fpm and the Apache module (cli) is?

[shadoxx]

I think the differences have been explained better elsewhere, so I will quote a better source: https://serverfault.com/a/645765

mod_php is running a PHP as Apache module - that is PHP request is run under Apache process with everything that goes with it - Apache processes are defined by Apache configuration, PHP is run with Apache permission etc.

PHP-FPM is PHP's FastCGI implementation; PHP-FPM runs as a standalone FastCGI server and Apache connects to the server using Apache's module, usually mod_fcgid or mod_fastcgi; I personally think this is much better than running as mod_php, but it depends on your requirements and is also a little more complex; in this configuration, permission, processes related stuff & everything is run under PHP configuration, PHP user etc. & Apache connects to PHP as to a server; in this configuration it is also possible to have pool of PHP servers and to have PHP server on physically different machine than Apache. They say this is almost as fast as using Apache's module and there are benefits of better control over PHP configuration.

So, basically:

CGI, Fast-CGI are protocols; CGI is slow, Fast-CGI is much faster
mod_php (with underscore) and PHP-FPM are two basic ways how to run PHP
mod_SuPHP is similar to mod_php but can change the user/group that the process runs under

tl;dr Improved security, better scaleability, lighter on resources.

[enoch85]

tl;dr Improved security, better scaleability, lighter on resources.

And more complex, harder to debug for Linux noobs which this VM is aiming for - even though it's used by Linux PROs as well.

If you can give me a POC that works better than the original one, then let's do it. We might even use a different branch and release one PHP-FPM version just to get it tested thoroughly before we put it in master.

[enoch85]

OK, I've looked at some videos and did some reading - digging into the differences. Apparently PHP-FPM is recommended for Apache 2.4 which is used in the VM. So, I'm fine with changing.

@shadoxx Please move forward with your PR. Thank you very much!

[enoch85]

Oh, and btw. To answer your question:

do I just spin up a fresh Ubuntu VM and run it?

Yes, when I test something new I usually does it on a Digitalocean VPS, or localhost on a local VM.

So basically, just install Ubuntu 18.04, once installed do a snapshot so that you can revert the installation process when you test the scripts. Make changes and revert to a fresh install of Ubuntu everytime you want to test your changes. Make sure to include all the scripts and choose all the extras once you do to make sure nothing breaks.

Also, a recommendation would be to check all the scripts, and search for anything with PHP or Apache in it, to see where it's used or set.

[enoch85]

@shadoxx Actually, if you start a PR we can work on it together. I have nothing planned for this weekend so I can help you with all the changes.

[tiagoefreitas]

The collaboration going on looks good 👍

If php-fpm is the recommended way of running Nextcloud Talk, and this is the "official" vm, it makes sense to use the best possible configuration.

I installed with HTTP/2 because I don't need talk video for now and thought a future update of talk would solve the issue.

But now I see it a vm upgrade is needed, do you think you will release one that does all the necessary changes, or a complete reinstall of another branch will be necessary?
In that case I may as well disable HTTP/2 for now, I can check the code that enabled it, or is it better to reinstall to do that?

Is there a recommended way to move all user data to another install in case I need to start fresh later?
Just getting started and want to minimize future changes and downtime.

Thank you

[enoch85]

@tiagoefreitas Well, you have to keep in mind that the Nextcloud VM isn't mainly made to work with Talk, there are so many other scripts and factors that needs to be weighed in. If PHP-FPM breaks 10 scripts but solves one problem, then I wouldn't go forward, but now it seems like it's recommended anyhow by Apache since it's the new way of dealing with PHP in general.

The change to PHP-FPM will happen in the whole VM, and not just for Talk, meaning that it will require some effort to test it properly before put in production. I'm thinking we'll make a seperate branch called "php-fpm" or similar so that we can test everything several times before we put it in master.

With the upcoming release of Nextcloud 14, adding PHP-FPM to the VM would be a major change and suitable for a major version bump.

Then we have backwards compability to keep in mind as well. How do we ensure that ppl running Nextcloud 12 won't break anything by using the update script for example?

So with all that said, there are a lot to think of when making a change like this. It affects so many users since this VM is downloaded about 150 times per day.

Is there a recommended way to move all user data to another install in case I need to start fresh later?
Just getting started and want to minimize future changes and downtime.

Just do a regular import and export of the DB and data if you want to start fresh, noting too complicated. It's documented here and here.

[enoch85]

@shadoxx Started to work on a separate branch. Feel free to dig in!

[enoch85]

@tiagoefreitas OK, so you can help us test the PR. Just install everything on a separate VM and test if Talk works with HTTP/2: https://github.com/nextcloud/vm/pull/591

You need to install everything with this script: https://raw.githubusercontent.com/nextcloud/vm/php-fpm/nextcloud_install_production.sh

[enoch85]

Talk works with PHP-FPM and HTTP/2. So once we merge that PR this issue will be solved.

[tiagoefreitas]

Testing this now.
The installer still asks if you want to enable HTTP/2, do you want to ask when using PHP-FPM?

[enoch85]

do you want to ask when using PHP-FPM?

Yes, since it's not from from the Ubuntu repos and may cause issues down the road. Even if Ondrej Sury is a trusted source, I don't want to use PPAs that are outside of the Ubuntu standard repo.

I also don't know if HTTP/2 breaks anything else.

[tiagoefreitas]

When it reached the part where it installs onlyoffice, it failed because nextcloud is giving "service unavailable".
At the end it said "Congratulations! You have successfully installed Nextcloud! " but still not working.
Will look at logs and let you know.

[enoch85]

@tiagoefreitas Did you run the SSL script? It always fails because the repo is wrong somewhere, but succeeds when you download the latest script from this repo and run it.

So, once we merge it to master it will work. I tested several times yesterday.

[tiagoefreitas]

I downloaded the php-fpm script and ran it...what else was needed? Yes I activated SSL when it asked during install.
This is the apache2 error log:

[Mon Aug 06 17:08:12.001514 2018] [proxy:error] [pid 1986:tid 139869042882304] (2)No such file or directory: AH02454: FCGI: attempt to connect to Unix domain socket /run/php/php7.2-fpm.sock (*) failed
[Mon Aug 06 17:08:12.001552 2018] [proxy_fcgi:error] [pid 1986:tid 139869042882304] [client 62.28.29.102:56914] AH01079: failed to make connection to backend: httpd-UDS

[enoch85]

@tiagoefreitas Download the latest activate-ssl.sh script and run it. It will solve the problem. The download the onlyoffice.sh script as well and run it manually.

[tiagoefreitas]

I used activate-ssh from php-fpm branch and it worked, its accessible over https.

Then I tried onlyoffice.sh from php-fpm branch and it says it cannot reach Nextcloud over https...

[enoch85]

Restart both apache and php-fpm and then try again.

[tiagoefreitas]

It installed successfully but does not work, and if I go to the onlyoffice url it opens nextcloud. Not sure if I used the master or php-fqm branch..

If I run it again:
Warning: the "docker" command appears to already exist on this system.

If you already have Docker installed, this script can cause trouble, which is
why we're displaying this warning and provide the opportunity to cancel the
installation.

[enoch85]

I've already tested Collabora and OnlyOffice, and it works. So you must be doing something wrong.

You must use everything from the php-fpm branch, else it will fail.

[tiagoefreitas]

ok I may have made a mistake downloading onlyoffice.sh from master branch,
can I run it again or have to redo everything from scratch?
On Mon, 6 Aug 2018 at 19:46, Daniel Hansson [e-mail]
wrote:

I've already tested Collabora and OnlyOffice, and it works. SO you must be
doing something wrong.

You must use everything from the php-fpm branch, else it will fail.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nextcloud/vm/issues/588#issuecomment-410812147, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AK6WxoIaowXcvOYqinmshSqujeA7mysAks5uOI7ygaJpZM4Vsjci
.

[enoch85]

You should be able to run it again. I don't remember if we trash the old docker stuff or not. Try, if it works then great. If not, then you have to start over.

Thanks for testing btw! :+1:

[tiagoefreitas]

It failed, I will try again.
I see you are making commits to php-fpm now, I want to test it again should I wait a bit for further
commit it go ahead?

Will not configure SSL this time, looking into using a reverse proxy instead (caddy or traefik).

[enoch85]

I see you are making commits to php-fpm now, I want to test it again should I wait a bit for further
commit it go ahead?

You can go ahead. It's about done now. Will merge once it's properly tested.

[enoch85]

Will not configure SSL this time, looking into using a reverse proxy instead (caddy or traefik).

Btw, I would recommend Nginx instead. I have scripts for that too. :)

[thejohnha]

@enoch85 Just out of curiosity, I followed your instructions on techandme to set up a reverse proxy with nginx. But ever since I haven't been able to get Nextcloud to automatically renew its SSL cert. I have to point the router to the nextcloud instance directly (and not the reverse proxy) to renew the cert, then point the router back to the nginx reverse proxy. Is this the same for you, or do you have a better way to renew the certs? Thx - sorry for the off-topic.

[tiagoefreitas]

Where are the scripts, can you send?

I was thinking of caddy or traefik because they use lets encrypt automatically and http/2 by default, and also automatically configure themselves for docker container apps.

I want to add more apps and it would make it easier as I don't want to spend too much time in sysadmin that I never did much before.

However I just found this:
https://github.com/jwilder/nginx-proxy
https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion

May use that instead...

[tiagoefreitas]

Why would the ssl certs for nextcloud be needed when using a proxy?

[enoch85]

@thejohnha You need to put the SSL certs on your Proxy as that is the one talking to the outside.

The guide is not updated, since I use the scripts instead, and due to lack of time.

@tiagoefreitas If you send me an email I can send them to you. The script is built so that everything is deployed automatically, SSL and all that as well. I use them myself to deploy new hosts on my server.

Why would the ssl certs for nextcloud be needed when using a proxy?

You need them on the Proxy, not on the backend.

Btw, how is testing going? Any progress?

[tiagoefreitas]

It appear Ubuntu 18.04 includes apache2 with mod_http2, so an external source should not be needed.

Maybe you did it before this fix bellow?

https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1658469

[enoch85]

Uuh, missed that one. Can you try to enable it without using the PPA?

[tiagoefreitas]

I can remove the ppa and revert to stock apache2 and test.

[enoch85]

I tested it myself, and you're right, it's not from PPA anymore. I will enable it per default. :+1: