ToDo:
Hey there,
as I use dietpi mainly for my nextcloud instance, I want to optimize the installation process a bid to give a better plug & play feeling with less left necessary/recommended configuration.
Most information about how nextcloud should be set up, I get from the current admin manual.
Before creating a related pull request, I want to collect and discuss the possible changes/additions here.
localhost
, second "domain" is *
, I have no idea what the result of this is. Maybe allow all domains for the start? I always replace that with my actual online domain of course..user.ini
is created with max upload/post file sizes and memory limit. Actually this is not necessary with apache+mod_php, as then .htaccess
will be used, which are then auto created by nextcloud. I don't know which one will be preferred in case.APCu is set for local memcache.
What I didn't see so far:
Several apache and php modules need to be installed/enabled: https://docs.nextcloud.com/server/12/admin_manual/installation/source_installation.html#example-installation-on-ubuntu-16-04-lts-server
All necessary ones are actually there, but maybe it make sense to check this again during nextcloud installation, just in case and if something changes about that/changed by admin?
What is missing so far:
Nextcloud cronjob, which is highly recommended: https://docs.nextcloud.com/server/12/admin_manual/configuration_server/background_jobs_configuration.html#cron
There is also an occ command to change this for nextcloud side: https://docs.nextcloud.com/server/12/admin_manual/configuration_server/occ_command.html#background-jobs-selector
Further possible improvments:
port 0
unixsocket /var/run/redis/redis.sock
unixsocketperm 770
apc.enable_cli=1
Same could be also done for ownCloud, but to make a start and because I use Nextcloud, I started with this.
Additions, discussion, corrections are highly recommended, as well as hints, which parts may differ on different platforms. I only have a Raspberry Pi 2 here to test/assure that certain configurations are working.
After decision what can/should be reasonable implemented and how, I will create a pull request. #1280
Anyone any opinion on this?
Otherwise I would start to create a pull request including all changes that are necessary in my view (until 3.).
Changes in 4. do also affect other web services, so I am not too sure if this is a good idea. Maybe it could be explained during installation process and just done upon user agreement + non manual set value for the related settings.
Configuring Redis, if it is already installed, to (also) listen to redis.sock and then configure Nextcloud config.php accordingly otherwise would not hurt. To force/recommend it's installation on the other hand, should be discussed.
As Nextcloud 12 is installed, it needs at least PHP 5.6 thus debian/raspbian Jessie, am I right? So the installation script would need to check for this distro or stretch and check/install the related php packages (5.6 vs 7.0), otherwise refuse installation with error message that recommends to upgrade to at least Jessie.
Am I right, that for all dietpi platforms but RPi the debian.org repos are used? debian.org and raspbian.org use the exactly same package names+versions for all webserver/php related stuff, so we can handle them similar.
@MichaIng Thank you for bringing this up for discussion. I will be willing to help test and contribute as I have time with your project.
Here's a up-to-date guide by one of the lead developers of Nextcloud. That walks you through the complete set up of Nextcloud 12, Nextcloud A+, SSL A+, Ubuntu 16.04.2 LTS 64Bit, nginx 1.13.3, mariadb 10.0.29, PHP 7.1.7, ufw, fail2ban, redis-server, postfix and more.
Any recommendations I might have are found in that guide. Such as...
Rather than reposing my recommendations here, update your post after reading the guide and we'll go from there. It contains everything I would recommend. A couple of amendments would be utilizing Redis 4 as it brings many enhancements on single board computing platforms. In addition alternate directory the cache file because it should not be on an SD card because high r/w but on either a hhd/mmc medium. I created a temp folder in /mnt/dietpi_userdata/ for ngx_cache_purge and php.
The guide caters to for oDroid C2 and raspberry pi 3, but walks you through detail configuration process to utilize any single-board computer. I have adapted successfully in its commands PHP 5.6 on debian/raspbian Jessie but not on dietpi the platform due to the following
There are some outstanding issues experienced with dietpi and web server configuration. Many of the issues you mentioned can be fixed through further configuration of standard files nextcloud and its dependencies. Unfortunately, at the time of writing how dietpi handles web server configuration it prevents a secure fully functional deployment of nextcloud (or any other web service.)
For simplicity dietpi with all web servers essentially runs _all_ website that's dropped into /var/www folder via a _global_ configuration file. For example Nginx "/etc/nginx/sites-enabled/default" if we are to properly deploy nextcloud on dietpi nextcloud each websites is going to need its own configuration for each web server. Based on my limited knowledge we cannot have a global config that functions as it does in dietpi and have separate for config's websites more advanced configuration congruently.
@Fourdee this issue is not going to go away, we might want to open up a separate ticket?
We should be able to come up with a template that work for most websites and webservers. Thus allowing for advanced configuration as needed such as nextcloud.
@LexiconCode Thanks for your participation!
I read through the guide of Rieger. It contains indeed many useful/necessary steps to run Nextcloud stable and secure. But it is in some parts quite specific for the setup with ubuntu server and some therefore specific versions/configurations of webserver, php and database. Many of the provided steps are already covered by the default related debian/raspbian packages, certbot, the dietpi-software installation process of webserver stacks or by Nextclouds .htaccess files, in case apache with mod_php is used. Others rely on installing additional packages (including redis, which I also would always recommend), where I would keep it more modular/optional instead of installing a quite big bunch of software together with Nextcloud by default. Also the e.g. webserver/php config modifications might be perfect for a just-Nextcloud server, but might impair or even break other parallel installed web services. Same for the additional improvements I listed in under 4.
above. I am still not sure if it is a good idea to adjust these settings. Many magic checks could be necessary to prevent incompatibilities etc. I would prefer to do most/all of that via modular/Nextcloud specific config files, e.g. an separate nextcloud.conf
, the .htaccess
or .user.ini
where possible.
By the way, does nginx provide a possibility to set site/location specific settings in a separate config file? I use apache so far and do not have an environment to test nginx setups, so I would highly appreciate if someone else could provide the nginx specific configuration part.
@Fourdee Did I watch right, that you added Debian + Raspbian Stretch now as default (and also all dietpi-platforms are debian/raspbian repo based), so with the following update there will be no Jessie server left? For RPi it makes sense, as finally the stretch repo is there for archive.raspberrypi.org and also on archive.raspbian.org stretch is now the stable repo: http://archive.raspbian.org/raspbian/dists/
Funny thing, that their official image ist still Jessie, but should not take long anymore: https://www.raspberrypi.org/downloads/raspbian/
Generally I also would also add all mentioned improvements in separate commits/pull requests to keep them separately testable and discussable π .
Let me go through the single steps of Riegers guide and your recommendations now...
Riegers Nextcloud installation guide: https://www.c-rieger.de/nextcloud-installation-guide/
Nginx installation
Installation is done here by manual build from scratch ubuntu source. For simplicity and (current+future) compatibility I would stay with the debian/raspbian repo packages, as dietpi already does it within dietpi-software.
I am not sure about the ngx_cache_purge
module. If someone knows more about that, we could discuss (better in a separate issue) to add it by default to the dietpi-software nginx installation?
worker_processes
setting is already done by dietpi-software: https://github.com/Fourdee/DietPi/blob/master/dietpi/dietpi-software#L7504
About worker_connections
and the other provided settings: same as 2.
π as I do not know much about nginx. Again if there is a solid way to add Nextcloud specific settings into a separate config file, I guess this would be the best way to go: Ah /etc/nginx/conf.d/nextcloud.conf
is possible π
https://docs.nextcloud.com/server/12/admin_manual/installation/nginx.html#nextcloud-in-a-subdir-of-nginx also is a good source to compare to Riegers guide.
PHP installation
PHP7.1 is available on ubuntu repo, for debian/raspbian stretch its just PHP7.0, but I guess at least in terms of performance the step from 5.X -> 7.0 was the most beneficial one and good enough to leave it there?
PHP-FPM configuration is already done by dietpi-software: https://github.com/Fourdee/DietPi/blob/master/dietpi/dietpi-software#L7575
pm.max_requests
however is missing there, could be discussed to add, better again in a separate issue.
APCu and OPCache configuration also done by dietpi-software: https://github.com/Fourdee/DietPi/blob/master/dietpi/dietpi-software#L7600-L7639
Both configurations together meet more/less what is given by Nextcloud admin manual and linked in OT. I wonder that in dietpi-software as well as in Riegers guide, some of the module related settings are not moved to the actual module.ini.
All opcache.<setting>
should be moved into opcache.ini
and
all apc.<setting>
should be moved into apc.ini
to avoid syntax errors on module absence.
Also I am wondering, if it is possible to move all the settings, that might collide with other web services, could be moved into Nextclouds .htaccess/.user.ini. E.g, my /path/to/nextcloud/.htaccess containts:
<IfModule mod_php7.c>
php_value upload_max_filesize 1.9G
php_value post_max_size 1.9G
php_value memory_limit 512M
php_value mbstring.func_overload 0
php_value default_charset 'UTF-8'
php_value output_buffering 0
<IfModule mod_env.c>
SetEnv htaccessWorking true
</IfModule>
</IfModule>
Would be great if we could add some settings there, paired with checks for activated PHP modules
(inside the here checked apache modules). But I did not try yet, if this is working well for APCu/OPCache related settings.
MySQL/MariaDB installation
default-character-set = utf8mb4
is there,skip-name-resolve
actually doesn't have any influence, if the database is only accessed locally, which we assume for SOC servers π . All the other settings given by Riegers guide are in some cases already implemented into mariadb default (on debian/raspbian package) and/or strongly rely on the actual usage of Nextcloud/the whole server. So I would not change much more there, as it is a balance question between low memory consumption and database performance/capability. The default settings give a good balance here from my point of view./etc/php/7.0/mods-available/pdo_mysql.ini
as recommended by Nextcloud admin manual (see OT 4.
). Could be also added to the nextcloud folder specific config file?'mysql.utf8mb4' => true,
Letβs Encrypt / HTTPS configuration
As using Nextcloud/the server just internally and opening both to the web are two different things, I would leave all that SSL related stuff within Letβs Encrypt installation. For apache as well as nginx there exist packages for debian/raspbian stretch: https://packages.debian.org/stretch/python-certbot-apache | https://packages.debian.org/stretch/python-certbot-nginx
At least for apache with certbot --apache --rsa-key-size 4096
these give me easy and fast a well secure default configuration with direct A+ ssllab rating and A scan.nextcloud.com rating (where the only missing thing is __Host-Prefix hardening, which is not possible if nextcloud is reachable via my.domain/nextcloud). The only thing I add is: Redirect permanent / https://my.domain/
because I don't know the difference to (given by certbot/letsencrypt):
RewriteEngine on
RewriteCond %{SERVER_NAME} =my.domain
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,QSA,R=permanent]
and thought double is better π .
The letsencrypt default ssl configuration also is moreless the same than provided by https://mozilla.github.io/server-side-tls/ssl-config-generator/
I guess for the nginx package it should be the same, but should be indeed checked.
By talking about this: dietpi goes a quite complicated way by building certbot from source and creates much data overhead with this, as least it looks like this while watching the whole installation/running process. Why not switching to the mentioned debian/raspbian packages?
Redis installation
I am still not sure how to handle this. I could imagine the following alternatives:
You say version 4 brings some benefits for SOCs over version 3, which is currently installed via debian/raspbian stretch repo? If repo package is wanted, we would need to go for sid: https://packages.debian.org/sid/redis-server which is just available for debian, but not yet for raspbian. If this or manual building from source is working well with stretch, would need to be tested carefully.
Other steps of the guide
crontab -u www-data -e
?Uff long text here. enough for today.
By the way I just recognized this: https://github.com/Fourdee/DietPi/issues/1034
@Fourdee as the topics mentioned there are also covered here, you could close that opened issue in favour of this π .
Btw. https://www.codem.com.au/blog/per-directory-php-nginx-configuration/
Is there anything against using this for nginx/php nextcloud specific configurations as well as .htaccess for apache? We could move all mentioned settings into these files. While discussing, if we find some good for the whole server setup. we can still move them into the php.ini afterwards. But as I said: I like to keep things modular π .
@MichaIng
@Fourdee Did I watch right, that you added Debian + Raspbian Stretch now as default (and also all dietpi-platforms are debian/raspbian repo based), so with the following update there will be no Jessie server left? For RPi it makes sense, as finally the stretch repo is there for archive.raspberrypi.org and also on archive.raspbian.org stretch is now the stable repo: http://archive.raspbian.org/raspbian/dists/
Funny thing, that their official image ist still Jessie, but should not take long anymore: https://www.raspberrypi.org/downloads/raspbian/
Plan is to switch to Stretch for all our devices. However, the RPi stretch image is still in testing:
https://github.com/Fourdee/DietPi/issues/475
Other devices (such as ARMbian built images), I'll check later down the line. Once the main boards are updated to stretch.
Ah okay, because in PREP_SYSTEM_FOR_DIETPI.sh you set everything to DISTRO='stretch'
. But I see that you adjust this for image creation: https://github.com/Fourdee/DietPi/blob/testing/dietpi/func/dietpi-set_software#L57-L58
So we still need to take care about especially different php verions (5.6 <> 7.0) an therefore different locations of php.ini etc. If we move all adjustments to .htaccess/.user.ini of course that would not matter again.
About using .htaccess
/.user.ini
http://php.net/manual/en/configuration.changes.php
So should work at least with apache+mod_php. Instead of manipulating the .htaccess which comes with nextcloud, it could be also done via nextcloud vhost inside <directory>
statement. But still, I don't find a way to check for installed/activated php modules there. Will check, if syntax error arrives in case, or if the php settings are available/set anyway, in case without effect.
http://php.net/manual/en/configuration.file.per-user.php
And the .user.ini
seems to be actually made for php values, as you even do not need php_flag
/php_value
. So this should work for php_fpm/nginx then.
@MichaIng
By default, dietpi currently installs mysql with nextcloud, where I would clearly prefer mariadb. But the best would be the possibility to generally choose the preferred database, as it is possible for webserver already and discussed here: #894
MariaDB already comes well preconfigured:
Forgot to mention, MariaDB is now default in Stretch:
https://www.debian.org/releases/stable/armhf/release-notes/ch-whats-new.en.html#mariadb-replaces-mysql
Ah nice info. I was waiting for MariaDB to become default/preferred by more instances:
I mean okay, who want's to install again a meta-package of a meta-package.. with default-mysql-server
, especially where the name again implies that it is "mysql". But okay.
The other possibility would be to completely switch DietPi to MariaDB and just remove MySQL. For web server the option make sense, because all web servers have clear differences and pros/cons. For databases (also for Nextcloud) there are also different possibilities: MySQL/MariaDB 5.5+; PostgreSQL; Oracle 11g
, but for MySQL and MariaDB there is from my point of view no pro/con, MariaDB ist just the preferred better default configured and faster developing alternative, which is fully compatible with all known mysql commands/api. Just my opinion as I say, and seems also the opinion of debian devs ;).
I have a problem with the plugin/app Ebook-reader. It doesn't work, no matter web server installed (apache, nginx, lighttpd). When I try to open a comic, cbr or cbz format, blank page appears with some left buttons at left (but not loaded completly).
The error is:
Refused to execute script from 'http://192.168.114.128/nextcloud/index.php/apps/files_reader/vendor/bitjs/archive/unrar.js' because its MIME type ('text/html') is not executable, and strict MIME type checking is enabled.
The error happens both in my Raspberry Pi 3 and VM image (using Jessie build).
The plugin works fine in the web demo version on Nextcloud (using the same cbr file).
I can't find how to fix it.
Anyone can help me?
Regards
After further investigations, and try and error, I have found:
If I change the file
/var/www/nextcloud/lib/public/AppFramework/Http/EmptyContentSecurityPolicy.php
and change:
$policy .= "base-uri 'none';";
to
$policy .= "base-uri 'self;";
it works.
But I don't know if I'm opening a big security hole....
Finally Stretch became stable for Raspbian: https://www.raspberrypi.org/downloads/raspbian/
So we can concentrate on Stretch (PHP7 etc.) packages π . I hope I find some time the coming week to test the use of .user.ini/.htaccess for all PHP module related configurations.
Finally I was able to do some tests about setting PHP values just for nextcloud/the nextcloud location.
By reading this more carefully I got, that in .user.ini
/.htaccess
you can just set a small amount of PHP settings. Mostly we want to adjust values here, that just can be set inside php.ini
(respectively the conf.d etc. sub configs) or "httpd.conf" [ 1 2 3 ]. The letter means in our case apache2.conf
. I tried around and finally was able to set the values just for the nextcloud instance by giving them in nextcloud vhost as following:
Alias /nextcloud "/var/www/nextcloud/"
<Directory /var/www/nextcloud/>
Options +FollowSymlinks
AllowOverride All
<IfModule mod_dav.c>
Dav off
</IfModule>
SetEnv HOME /var/www/nextcloud
SetEnv HTTP_HOME /var/www/nextcloud
Satisfy Any
# OPCache configuration
#php_admin_value opcache.enable 1
php_admin_value opcache.enable_cli 1
php_admin_value opcache.interned_strings_buffer 8
php_admin_value opcache.max_accelerated_files 10000
php_admin_value opcache.memory_consumption 128
php_admin_value opcache.save_comments 1
php_admin_value opcache.revalidate_freq 1
# MySQL/MariaDB configuration
php_admin_value mysql.allow_local_infile 1
php_admin_value mysql.allow_persistent 1
php_admin_value mysql.cache_size 2000
php_admin_value mysql.max_persistent -1
php_admin_value mysql.max_links -1
php_admin_value mysql.default_port 0
php_admin_value mysql.default_socket /var/run/mysqld/mysqld.sock
#php_value mysql.default_host
#php_value mysql.default_user
#php_value mysql.default_password
php_admin_value mysql.connect_timeout 60
php_admin_value mysql.trace_mode 0
# APCu configuration
php_admin_value apc.enable_cli 1
</Directory>
_β¬: One correction about it: "opcache.enable 1" inside vhost produces an nextcloud log error. I thought it was solved by removing the duplicate in php.ini/opcache.ini but it was not/reappearing. Seems this value can/should not be set outside of default php config files: https://support.plesk.com/hc/en-us/articles/115001622189-A-lot-of-errors-in-php-fpm-log-ERROR-Unable-to-set-php-value-opcache-enable-
But opcache still worked as intended with the correct local settings, which I also tested by copying the dietpi's opcache.php into the nextcloud folder. And it defaults to "1"/enabled anyway, so can be just left out._
I was able to verify, that these settings are just present for the nextcloud directory by checken inside and outside via phpinfo()
. The mysql settings are by the way obsolete, as they default like this and I am not very sure, if they are used that way for "pdo_mysql" module, which is used by nextcloud. The "cache_size" I do even only find for "mysqli", which is not used by nextcloud.
The good thing about this method: I do not see any errors if wrong php values are set, respectively a module is not present/active. Wrong values seem just to be ignored, so no checks for active modules are necessary here.
For php_fpm/nginx there is something similar possible: http://php.net/manual/en/install.fpm.configuration.php (see bottom)
But I do not have the time to set up this for testing. I hope someone with php_fpm and/or nginx finds the time to test it.
Still opened questions are about:
HTTP/2: It still breaks the spreed video calls app and might break some other web services as well as old browser versions. I did not yet test, if this can at least be activated only for nextcloud inside the <Directory>
directive.
Install Redis automatically or not or give the choice?
I found this solution to create the http users crontab via script: crontab -u www-data -l 2>/dev/null | grep -q '/var/www/nextcloud/cron.php' || echo "*/15 * * * * php /var/www/nextcloud/cron.php" | crontab -u www-data -
Does this look good? We would also need a related command to remove the line for nextcloud uninstall script. Otherwise we can also just edit the users files in /var/spool/cron/crontabs
which has the same effect. As we test the changes, there is no risk that something wrong will be added.
_β¬: Also found that, working on the official nextcloud vm machine: https://github.com/nextcloud/vm/blob/master/nextcloud_install_production.sh#L286_
@dhernan3 Sorry for the late reply. I would assume your error to be on the apps/nextclouds side. Check its github page or help.nextcloud.com to find some help about that. Could be related to some missing optional php modules? https://docs.nextcloud.com/server/12/admin_manual/installation/source_installation.html#prerequisites-for-manual-installation
Btw: I still face some opened questions about using the nextcloud vhost/apache conf for php module settings (see above), mostly about how opcache is used. Explained them here: https://help.nextcloud.com/t/use-htaccess-user-ini-for-all-nextcloud-related-php-configuration/18492/5?u=michaing
I'm still stuck in how to figure out if everything is really working as intended and will not produce any errors later on. At least my own nextcloud runs without error since then and the 5 apache processes just use each about ~30 MB ram, so no ~100 MB blocked, like stated within the link π .
Hi,
I will install Diet-Pi STRETCH tomorrow on m'y Rpi3 ans will try to help you with Nextcloud :)
it would be good to bring 4-bits support out of the box
It needs :
Yeah, I read this everywhere, but could you explain, what is the actual benefit of this change according to Nextcloud? As it seems possible to change this afterwards, I will test it later on my own system.
the actual benefits is that it brings full unicode support (emoji included).
For sole people it's not mandatory but if you use the _news_ app on Nextcloud for example you will need unicode support to properly render some feeds.
but if you use the news app on Nextcloud for example you will need unicode support to properly render some feeds
Ah okay that makes sense. I was asking myself where the hell emoji's (also mentioned in nc admin manual) could be needed inside a database :laughing:.
Added it to Phase Il plan.
β¬: Okay, first mysql/mariadb config needs to be changed:
# Enable 4-byte support for e.g. Nextcloud
# Will be default for >= MariaDB 10.2.2
innodb_large_prefix = 1
innodb_file_format = barracuda
# Actually already defaults for >= MariaDB 5.5
innodb_file_per_table = 1
Barracuda file format is not the only thing, innodb_large_prefix
also need to be set according to admin manual: https://docs.nextcloud.com/server/12/admin_manual/configuration_database/mysql_4byte_support.html
Last setting already defaults for mariadb since long time, the others will default for mariadb >= 10.2.2, but this is not (yet) available for Stretch and will never be for Jessie repo.
Afterwards the nextcloud tables need to be migrated "manually, one by one" as the admin manual says. They offer a command helping with that, but I don't know if this is possible in a more automated way. Otherwise we would need to check, if nextcloud is/was already installed (the database exists) and enable 4-byte support just, if it is a clean installation.
By the way, the following was already set in my mariadb config. Is this somehow related?
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci
Adding value to config.php and in case updating the tables from nextcloud side would be no problem via occ command.
β¬β¬: By the way: The migration worked very well for me. I even recognized that my database size was reduces to less than half. I try to get some response what could be the reason for that: https://help.nextcloud.com/t/mariadb-4-byte-support-migration-dramatically-reduced-database-size/21684
I included nginx configuration into existing PR, thanks to @Bridouz , so ready for testing: https://github.com/Fourdee/DietPi/pull/1142
Some more points could be considered. There is a stretch package providing the nginx cache_purge module https://packages.debian.org/stretch/libnginx-mod-http-cache-purge suggested by c-riegers guide. Sadly https://www.c-rieger.de/nextcloud-installation-guide/ is not available due to ssl error at the moment. The worker_processes
/worker_connections
for my impression highly relies on the underlying system and should be not set generally for all Nextcloud instances. This is something that should be set in DietPi's global nginx configuration, if a all.
Merged for testing: https://github.com/Fourdee/DietPi/pull/1142#issuecomment-333881513
Installation fails on fresh Rpi3 DietPi V157 :
Installation aborted because :
E: Command line option 'i' [from -intl] is not understood in combination with the other options.
[Failed] The apt cache may be corrupt, apt mirror offline, or you have held broken packages. DietPi-Software will now exit.
In fact `php7.0-intl" does not exist in apt-cache.
I tried to remove that package installation but then I had another errors with :
${php_package_name}enmod ctype curl dom fileinfo gd iconv intl json mbstring posix simplexml xmlwriter xmlreader zip pdo_mysql exif opcache apcu
saying that packages like opcache
apcu
do not exist.
EDIT: I removed all the php installation stuff locally as I already have them. Installation OK
but nginx fails to restart with error :
nginx: [emerg] "location" directive is not allowed here in /etc/nginx/sites-enabled/nextcloud.conf:1
Jessie:
Oct 09 17:29:43 DietPi nginx[1080]: nginx: [emerg] invalid number of arguments in "fastcgi_param" directive in /etc/nginx/sites-dietpi/nextcloud.config:42 || Oct 09 17:29:43 DietPi nginx[1080]: nginx: configuration file /etc/nginx/nginx.conf test failed
AH00526: Syntax error on line 38 of /etc/apache2/sites-enabled/nextcloud.conf:
| php_admin_value apc.enable_cli 1
Stretch:
I'll update as I complete testing.
reported errors with lines /etc/nginx/sites-dietpi/nextcloud.config
:
fastcgi_param SCRIPT_FILENAME;
fastcgi_param PATH_INFO;
try_files /nextcloud/index.php; #All references
set "opcache.enable_cli=1";
set " \n opcache.* # all
set " \n mysql.* #all
Jessie + Lighttpd, 503 remains on nextcloud:
Creating config file /etc/php5/mods-available/intl.ini with new version
php5_invoke: Enable module intl for cli SAPI
php5_invoke: Enable module intl for cgi SAPI
php5_invoke: Enable module intl for fpm SAPI
Processing triggers for php5-fpm (5.6.30+dfsg-0+deb8u1) ...
WARNING: Module ctype ini file doesn't exist under /etc/php5/mods-available
WARNING: Module ctype ini file doesn't exist under /etc/php5/mods-available
WARNING: Module ctype ini file doesn't exist under /etc/php5/mods-available
WARNING: Module dom ini file doesn't exist under /etc/php5/mods-available
WARNING: Module dom ini file doesn't exist under /etc/php5/mods-available
WARNING: Module dom ini file doesn't exist under /etc/php5/mods-available
WARNING: Module fileinfo ini file doesn't exist under /etc/php5/mods-available
WARNING: Module fileinfo ini file doesn't exist under /etc/php5/mods-available
WARNING: Module fileinfo ini file doesn't exist under /etc/php5/mods-available
WARNING: Module iconv ini file doesn't exist under /etc/php5/mods-available
WARNING: Module iconv ini file doesn't exist under /etc/php5/mods-available
WARNING: Module iconv ini file doesn't exist under /etc/php5/mods-available
WARNING: Module mbstring ini file doesn't exist under /etc/php5/mods-available
WARNING: Module mbstring ini file doesn't exist under /etc/php5/mods-available
WARNING: Module mbstring ini file doesn't exist under /etc/php5/mods-available
WARNING: Module posix ini file doesn't exist under /etc/php5/mods-available
WARNING: Module posix ini file doesn't exist under /etc/php5/mods-available
WARNING: Module posix ini file doesn't exist under /etc/php5/mods-available
WARNING: Module simplexml ini file doesn't exist under /etc/php5/mods-available
WARNING: Module simplexml ini file doesn't exist under /etc/php5/mods-available
WARNING: Module simplexml ini file doesn't exist under /etc/php5/mods-available
WARNING: Module xmlwriter ini file doesn't exist under /etc/php5/mods-available
WARNING: Module xmlwriter ini file doesn't exist under /etc/php5/mods-available
WARNING: Module xmlwriter ini file doesn't exist under /etc/php5/mods-available
WARNING: Module xmlreader ini file doesn't exist under /etc/php5/mods-available
WARNING: Module xmlreader ini file doesn't exist under /etc/php5/mods-available
WARNING: Module xmlreader ini file doesn't exist under /etc/php5/mods-available
WARNING: Module zip ini file doesn't exist under /etc/php5/mods-available
WARNING: Module zip ini file doesn't exist under /etc/php5/mods-available
WARNING: Module zip ini file doesn't exist under /etc/php5/mods-available
WARNING: Module exif ini file doesn't exist under /etc/php5/mods-available
WARNING: Module exif ini file doesn't exist under /etc/php5/mods-available
WARNING: Module exif ini file doesn't exist under /etc/php5/mods-available
apache2 AH00526: Syntax error
#
Seems to be only allowed at the beginning of a line / without command before.
This should be also changed in nginx nextcloud.config
Nginx:
try_files /nextcloud/index.php; #All references
Found this: https://stackoverflow.com/a/17760593
Seems that somehow you need two arguments for try_files
, but /dev/null
seems a valid 2. argument/workaround. Funny since this is the official provided config from Nextcloud admin manual and worked for @Bridouz ?
Is the error the same for fastcgi_param SCRIPT_FILENAME
/PATH_INFO;
, or do they belong together? I am little confused that suddenly all of them are wrong.
set "opcache.enable_cli=1";
set " n opcache.* # all
set " n mysql.* #all
I will put all settings into one directive, as mentioned in #1172 discussion.
About the php modules: Yes, I expected errors there, as some modules were backed into the php core in PHP5 and some have different names.
Could you show me ls -l /etc/php5/mods-available/
? So I will adjust this for Jessie.
Did Lighttpd work before? As we did not change something about this, or let's say there should not have been something changed according to Lighttpd behaviour π.
Lighttp works OOTB, even if it's not currently supported by Nextcloud.
did a bit of testing today. Fixed fastcgi issues and blank nextcloud index page on nginx.
Will push a PR tomorrow.
@Fourdee
π΄ lighttpd (503 handle request error)
@Bridouz
Lighttp works OOTB, even if it's not currently supported by Nextcloud.
Can someone teach me how to setup three virtual environments to start/stop on my RPi2, one with fresh DietPi Jessie + apache, one with DietPi Stretch + nginx, one with DietPi Stretch + lighttpd, to test fresh nextcloud installations π.
No really, I hope to find some time to investigate and do the topic and help to test and push development here for fresh systems. Sadly I can't reinstall my running server too much, which is just in private single server use, but my whole time and self management relies on it π.
Will push a PR tomorrow.
π
About the Jessie php module warnings: All these modules are part of the core php5 connectors: https://packages.debian.org/de/jessie/libapache2-mod-php5 (same for php-fpm / php-cgi.
I also remember that php became more modular with version 7.0, you are able to disable/enable modules, that were fix enabled in php5. So, phpenmod the list in error log above just for Stretch environments should solve the problem. But I still hope for some ls -l /etc/php5/mods-available/
from some debian Jessie, just in case some module is just reachable via different name or what π.
Will anyway create PR for this now.
@MichaIng
After a lot of testing, talks, search online we finally enabled php/opcache params and passed the security test on Nextcloud :)
Thus no errors in log so far π
location ~ ^/nextcloud/(?:index|remote|public|cron|core/ajax/update|status|ocs/v[12]|updater/.+|ocs-provider/.+|core/templates/40[34])\.php(?:$|/) {
include fastcgi_params;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_param SCRIPT_FILENAME $request_filename;
fastcgi_param PATH_INFO $fastcgi_path_info;
fastcgi_param HTTPS on;
fastcgi_param modHeadersAvailable true;
fastcgi_param REMOTE_USER $remote_user;
fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
fastcgi_intercept_errors on;
[...]
location ~ ^/nextcloud/(?:updater|ocs-provider)(?:$|/) {
try_files $uri/ =404;
Just need to add after https://github.com/Fourdee/DietPi/blob/testing/dietpi/dietpi-software#L8665
fastcgi_param PHP_ADMIN_VALUE "opcache.enable_cli=1
opcache.interned_strings_buffer=8
opcache.max_accelerated_files=10000
opcache.memory_consumption=128
opcache.save_comments=1
opcache.revalidate_freq=1
apc.enable_cli=1";
Setting mysql params throw errors in nginx log.
Opcache config line in nextcloud/config/config.php
was also faulty as it had a syntax errors and I don't know why.
It should have been \OC\Memcache\APCu
and it was the same without **
By the way would you like me to push a commit for that fix ?
I also requested a PR for Nginx support on Certbot and removed the well-known
locations at the beginning of nextcloud.config
as it was breaking certbot cert generation?
And do we stick with enabling nginx nextcloud conf file within dietpi-software or should we create a file in dietpi/conf
?
@Bridouz
Opcache config line in nextcloud/config/config.php was also faulty as it had a syntax errors and I don't know why.
[s]Yes opcache must not be set anywhere in config.php
.[/s]
_β¬: I think I missunderstood: I remember removing backslashes, since there were three of them like \\\OC\\\Memcache\\\APCu
and I know that one is enough. But maybe the way it is added, two backslashes are merged to one since it is also used for commands line line breaks \n
etc. So I added one again which should fix it. Thx 4 hint!_
By the way would you like me to push a commit for that fix ?
Yes will do now.
I also requested a PR for Nginx support on Certbot and removed the well-known locations at the beginning of nextcloud.config as it was breaking certbot cert generation?
I am not to familiar with these well-known folders, as I never seen them. But it seems to redirect requests to the somehow default DAV endpoints to Nextclouds one. But I guess it is not too important, as anyone who configures his carddav/caldav clients for Nextcloud will get the correct path easily in related contacts and calendar app web interface.
And do we stick with enabling nginx nextcloud conf file within dietpi-software or should we create a file in dietpi/conf ?
For now I would leave it like this. I will create the PR for _Phase II_ after next DietPi release with working _Phase I_ is released. Then I think I will put configurations into separate files to clean up the Nextcloud dietpi-software part. Also I am thinking about generally finding another place to put dietpi configuration files into, outside of ramfs. @Fourdee any suggestion? I though about root user folder, but this also does not really fit... /opt
maybe?
_β¬: Added your suggestions: https://github.com/Fourdee/DietPi/pull/1186/files
As well-known fix is inside your certbot PR, I left that out and hope both PR will merge without conflicts, they should π._
@MichaIng
@Bridouz
Thanks guys, great work on this π I'll re-run local tests today/tomorrow.
Can someone teach me how to setup three virtual environments to start/stop on my RPi2, one with fresh DietPi Jessie + apache, one with DietPi Stretch + nginx, one with DietPi Stretch + lighttpd, to test fresh nextcloud installations π
gitbranch=testing
in dietpi.txt
. Also set AUTO_Install_Enable=1
if you want automated installOnce the system is installed and updated
dietpi-backup
and set Backup mode : Full
, then exit (this will include MySQL DB's in the backup/restore)dietpi-backup 1
. This create a Windows style system restore point.dietpi-software install 114
dietpi-software install 74 114
dietpi-software install 77 114
dietpi-backup -1
System is now at state of previous backup, rinse and repeat "Begin testing" section. Just make sure to update the DietPi scripts if you modify any in /DietPi/dietpi/*
, before you test.
Thx, as said, I want to do this inside a virtual environment on my RPi2 to not disturb my production installation.
Jessie:
Stretch:
Hmm, I'd disagree with hard coding 128. We are setting memory cache size based on a percentage of total RAM, which is more optimized specific to the system?
And if this was to use the full 128MB available, 256MB systems would result in swapfile reliance, reducing performance.
EDIT:
Setting value higher than 128, removes the message. So 2GB+ RAM systems (1/10 = 204.8MB~ cache) are excluded from the warning in this case.
@MichaIng
@Bridouz
128
? https://github.com/Fourdee/DietPi/issues/1067#issuecomment-336174724 | yep | https://github.com/Fourdee/DietPi/issues/1067#issuecomment-336467832The point about the setting inside /nextcloud
location directive is, that this (in case higher cache size) can not be exceeded by external scripts, it can just be filled with Nextcloud scripts. As the amount of them is limited, used opcache can hardly grow over 30 MB as just PHP scripts are cached.
My used opcache (opcache.php) is less than 20M and I have quite a lot of apps installed.
Thus the chance, that hard coding 128M would every lead to swapping is practically 0, as only reason.
But yeah the chance stays and I am also not too happy with the situation that Nextcloud recommends these hard settings that prominent. Doesn't make too much sense for me, as using a 256M machine would anyway lead to a limited possibility of using Nextcloud: just as single user home server, thus all the values can be way lower in this cases.
Finally I would leave it hard coded, as messages on admin panel would lead to people asking here or on help.nextcloud.com about this issue and even if they try to solve it by edition e.g. php.ini
/opcache.ini
they would not be able, because the setting is a bid unusual set inside webserver configuration and as php_admin_value not overwriteable.
@MichaIng
Thus the chance, that hard coding 128M would every lead to swapping is practically 0, as only reason.
But yeah the chance stays and I am also not too happy with the situation that Nextcloud recommends these hard settings that prominent.
Finally I would leave it hard coded, as messages on admin panel would lead to people asking here or on help.nextcloud.com about this issue
I agree.
Ok, lets go with what NextCloud recommends. Even though I believe its a recommendation that needs to be improved.
Even though I believe its a recommendation that needs to be improved.
I totally agree! I will create a github issue about that, when I find time.
https://github.com/Fourdee/DietPi/pull/1191
Had to correct something:
@MichaIng
I'am going to remove the Nginx NextCloud site conf temporarily (due to https://github.com/Fourdee/DietPi/issues/1067#issuecomment-336159008), as we need to release DietPi v157 today.
I'll put it back in for v158
#Nginx: https://docs.nextcloud.com/server/12/admin_manual/installation/nginx.html#nextcloud-in-a-subdir-of-nginx
if (( ${aSOFTWARE_INSTALL_STATE[85]} >= 1 )); then
cat << _EOF_ > /etc/nginx/sites-dietpi/nextcloud.config
location ^~ /nextcloud {
# set max upload size
client_max_body_size $(( $(php -r 'print(PHP_INT_MAX);') / 1024 / 1024))M;
fastcgi_buffers 64 4K;
# Enable gzip but do not remove ETag headers
gzip on;
gzip_vary on;
gzip_comp_level 4;
gzip_min_length 256;
gzip_proxied expired no-cache no-store private no_last_modified no_etag auth;
gzip_types application/atom+xml application/javascript application/json application/ld+json application/manifest+json application/rss+xml application/vnd.geo+json application/vnd.ms-fontobject application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/bmp image/svg+xml image/x-icon text/cache-manifest text/css text/plain text/vcard text/vnd.rim.location.xloc text/vtt text/x-component text/x-cross-domain-policy;
# Uncomment if your server is build with the ngx_pagespeed module
# This module is currently not supported.
#pagespeed off;
location /nextcloud {
rewrite ^ /nextcloud/index.php\$request_uri;
}
location ~ ^/nextcloud/(?:build|tests|config|lib|3rdparty|templates|data)/ {
deny all;
}
location ~ ^/nextcloud/(?:\.|autotest|occ|issue|indie|db_|console) {
deny all;
}
location ~ ^/nextcloud/(?:index|remote|public|cron|core/ajax/update|status|ocs/v[12]|updater/.+|ocs-provider/.+)\.php(?:$|/) {
include fastcgi_params;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_param SCRIPT_FILENAME \$request_filename;
fastcgi_param PATH_INFO \$fastcgi_path_info;
fastcgi_param HTTPS on;
#Avoid sending the security headers twice
fastcgi_param modHeadersAvailable true;
fastcgi_param REMOTE_USER \$remote_user;
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_intercept_errors on;
# Disable because Jessie Nginx does not support that parameter
#fastcgi_request_buffering off;
}
location ~ ^/nextcloud/(?:updater|ocs-provider)(?:$|/) {
try_files \$uri/ =404;
index index.php;
}
# Adding the cache control header for js and css files
# Make sure it is BELOW the PHP block
location ~ \.(?:css|js|woff|svg|gif)$ {
try_files \$uri /nextcloud/index.php\$uri\$is_args\$args;
add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload;";
add_header Cache-Control "public, max-age=15778463";
add_header X-Content-Type-Options nosniff;
add_header X-XSS-Protection "1; mode=block";
add_header X-Robots-Tag none;
add_header X-Download-Options noopen;
add_header X-Permitted-Cross-Domain-Policies none;
# Optional: Don't log access to assets
access_log off;
}
location ~ \.(?:png|html|ttf|ico|jpg|jpeg)$ {
try_files \$uri /nextcloud/index.php\$uri$is_args\$args;
# Optional: Don't log access to other assets
access_log off;
}
# Setting PHP values by using this method: http://php.net/manual/en/install.fpm.configuration.php Example #2
# OPCache configuration: https://docs.nextcloud.com/server/12/admin_manual/configuration_server/server_tuning.html?highlight=opcache#enable-php-opcache
# APCu configuration: To prevent cron.php produce Nextcloud log [info] entry.
#"opcache.enable=1" defaults anyway and isn't possible here: http://php.net/manual/en/opcache.configuration.php#ini.opcache.enable
fastcgi_param PHP_ADMIN_VALUE "opcache.enable_cli=1
opcache.interned_strings_buffer=8
opcache.max_accelerated_files=10000
opcache.memory_consumption=128
opcache.save_comments=1
opcache.revalidate_freq=1
apc.enable_cli=1";
}
_EOF_
# - Stretch, set php7.0
if (( $DISTRO >= 4 )); then
sed -i "s#/run/php5-fpm.sock#/run/php/php7.0-fpm.sock#g" /etc/nginx/sites-dietpi/nextcloud.config
fi
fi
Added nginx conf again by https://github.com/Fourdee/DietPi/pull/1191. Indeed the nginx nextcloud.config contains settings (fastcgi_param HTTPS on; and HSTS) that are just relevant/valid for SSL enabled systems.
If I understand right, fastcgi_param needs to be in case set for every sub location, so we can't set all of this in the default nginx ssl config, as it is possible with apache. So we need 2 versions of the nextcloud config file, one for ssl enabled systems (how do we test?) and one for systems without ssl. Or maybe we can create a delta config that just adds the ssl relevant parts to to the nextcloud location directives, which can be then enabled and disabled, without the need to directly rewrite the basic (non-ssl) nginx nextcloud.config?
Never thought the nginx config file to produce that much work π. Indeed apache is way easier to configure, as all the ssl and other general settings can be set (and are mostly by default) within the standard apache config files and active for every sub location from there.
Issue reported by user, needs investigation:
http://dietpi.com/phpbb/viewtopic.php?f=11&t=2369&p=9251#p9248
WARNING: Module pdo_mysql ini file doesn't exist under /etc/php5/mods-available
pdo_mysql
module should be installed: https://github.com/Fourdee/DietPi/blob/testing/dietpi/dietpi-software#L3716pdo_mysql
is explicitly part of the php-module package: https://packages.debian.org/de/jessie/php5-mysqlnd / https://packages.debian.org/de/jessie/php5-mysqlphp-mysqlnd
is just installed in case of mariadb? Does it not work (good) with mysql itself?Quick retest after: https://github.com/Fourdee/DietPi/pull/1191
Yes Nginx fix (ssl + non-ssl) was not yet done. I will ask on https://help.nextcloud.com for help to make a non-ssl nginx-nextcloud-config based on the one for ssl from https://docs.nextcloud.com, which we use moreless already.
Also a Lighttpd config would be nice. It works out of the box, but some security and compatibility settings could be beneficial, also disabling non-compatible modules for within nextcloud location like webdav, if this exist.
MySQL and Nginx fixes: https://github.com/Fourdee/DietPi/pull/1226
Just switched my VMs to lighttpd. Works well out of the box. No X-Headers security warnings like with Nginx, so these headers seem to be set automatically here?
I tried to add location/url based 128 opcache memory:
nano /etc/lighttpd/conf-available/99-nextcloud.conf
server.modules += ( "mod_setenv" )
$HTTP["url"] =~ "^/nextcloud" {
setenv.add-environment += ( "PHP_ADMIN_VALUE" => "opcache.memory_consumption" )
}
Lets the opcache warning on admin panel disappear. But these setting will be set globally after visiting one Nextcloud page. After fresh restart, url/opcache.php
/ url/phpinfo.php
both show the correct 32MB opcache, which is set by DietPi as default value. After I open url/nextcloud
one time, both pages (outside of /nextcloud) also show suddenly 128MB opcache. I will carefully recheck this for Nginx (as it basically serves the php values the same way), but with Apache this is definitely not the case!
_Edit:_
Okay found out the following:
url/opcache.php
shows 128 available opcache memory.Test, NextCloud + Nginx + Odroid C2:
it seems that in v158 nextcloud cannot access the MySQL database, at least in case of lighttpd. see more details here: http://dietpi.com/phpbb/viewtopic.php?f=11&t=2475&p=9573#p9573
@molnarti
Jep, sorry for introducing this, wonder that I never faced during tests...
Fixes with: https://github.com/Fourdee/DietPi/pull/1242
Next PR ready to test: https://github.com/Fourdee/DietPi/pull/1251
Brought ownCloud installation onto the level of Nextcloud installation. Both are very similar, only a few necessary/possible differences. Thus I will further improve both together from now on.
Setup Redis for transactional file locking ready to test: https://github.com/Fourdee/DietPi/pull/1261
All tasks are done, ready to test: https://github.com/Fourdee/DietPi/pull/1261
Lighttpd is a separate task. Not too hight priority as ownCloud/Nextcloud runs out of the box with it. Need someone for help, that has a bid experience with Lightpd configuration.
HTTP/2 discussion should be done separately. It is a question in how far webservers do support it (Nginx + Apache2 just since Stretch e.g.) and if we can prove, that all DietPi offered software does support it. E.g. Nextclouds video calls app does still not work with it.
Fixes and polish: https://github.com/Fourdee/DietPi/pull/1272
@Fourdee
echo '[info] ...'
what we do, to inform users and also enable easier bug tracking that way?dietpi-banner
, if Nextcloud respectively ownCloud is installed:echo -e "\e[1m occ\e[0m = Shortcut for ownClouds 'sudo -u www-data php /var/www/owncloud/occ'"
@MichaIng
As configuration provides many steps, should we echo '[info] ...' what we do, to inform users and also enable easier bug tracking that way?
Yep, great idea, the more info for the user the better for them, and, us quick debugging π
/DietPi/dietpi/func/dietpi-notify 2 'Configuring NextCloud'
Would also reduce the need for comments in dietpi-software
, replaced by info messages.
Another idea, as I already implemented occ/ncc command shortcuts: We could inform users about those, by adding them to the default dietpi-banner, if Nextcloud respectively ownCloud is installed:
echo -e "e[1m occe[0m = Shortcut for ownClouds 'sudo -u www-data php /var/www/owncloud/occ'"
or something like that.
I'd prefer to keep the banner as clean as possible. Ideally, we should put this information on the online doc:
If you can write a short description of what the commands are, when to use them and briefly why, I'll add it to online doc.
Or if you are willing to do it yourself, let me know and i'll set your account to admin ?
@Fourdee
Will add notifications and write some doc tomorrow. Yes good idea, sometimes I forget that dietpi.com + forum was also for me the first info point.
@MichaIng
I'd prefer to keep the banner as clean as possible
Although, I am adding in a Xmas tree this year :rofl: π
https://github.com/Fourdee/DietPi/commit/e30065df70498b57e063aa76678c0fb11245dea1
@MichaIng
Quick note:
dietpi-notify
before the install to inform users installing OC and please wait etc...?2017-12-20 07:07:01 (562 KB/s) - β/var/www/apc.phpβ saved [38125/38125]
ownCloud was successfully installed
@Fourdee
Yes I anyway planned to add info echos during the installation process to reflect what we are doing and give possibility to name a step, if error occurred.
Some kind of global error check function with logging and meaningful pronouncement could be nice, to have a consistent method for this, what you think?
Self reminder, as I just got mail about closed issue: https://github.com/owncloud/core/issues/25773
Consider dropping of upload size adjustment by DietPi installation script. This causes integrity check errors in case, can be overwritten by certain oc/nc tasks and the value can be adjusted easily by user from within admin panel. However on ownCloud somehow the 'additional settings' tab is empty in my test cases, thus I can't find the setting there π.
Other funny thing is that I was able to upload even 10G video files and stuff like that via web interface without any warning on 513M upload size limit π.
@MichaIng
Some kind of global error check function with logging and meaningful pronouncement could be nice, to have a consistent method for this, what you think?
Yep, I can add a G_CHECK_ERROR
which can be run after any command to:
I'll have a play around ^^
@MichaIng
Yep, I can add a G_CHECK_ERROR which can be run after any command to:
https://github.com/Fourdee/DietPi/issues/1311#issuecomment-353716344
I will close this, as major rework is done. Other tasks will need separate issues, also as this one is heavily grown π.
Most helpful comment
Self reminder, as I just got mail about closed issue: https://github.com/owncloud/core/issues/25773
Consider dropping of upload size adjustment by DietPi installation script. This causes integrity check errors in case, can be overwritten by certain oc/nc tasks and the value can be adjusted easily by user from within admin panel. However on ownCloud somehow the 'additional settings' tab is empty in my test cases, thus I can't find the setting there π.
Other funny thing is that I was able to upload even 10G video files and stuff like that via web interface without any warning on 513M upload size limit π.