Vm: Quick default installation, Static IP and External Storage

[Hamsterman]

I have been trying out the VM a few times now and succeeded on the 3rd try. I have a few "issues" which are more like enhancements.

1) Quick default installation
One thing that comes to mind is that there are too many obvious questions like selecting the if you want other mirrors or the keyboard selection.
If you want the installation to be very smooth then put a list of selections in the beginning and then an option for selecting each one of them. I think it would work for many of the question (but probably not all).

2) Static IP
I actually had some issues here. When you give the possibillity to set the IP as a static IP you do not give the possibillity to change it. I had to run the installation a few times in order to guess the MAC-adresse for my server so I could let my router assign the correct IP. Yes I know you can change it afterwards, but for a non-linux guy like me I think the other way was easier.

3) External Storage
The VM here on Github got about 20GB space which is probably not enough for most installations. Expanding the VM to a bigger size is really not a good idea since having the files on a External Storage is more flexible and better for backup. I would love to have the option to specify another storage location than the VM for the home directory of all the users at the time of the installation.

[enoch85]

1. If you have Swedish keyboard you don't want to change. If you are in the correct timezone, you don't want to change, etc. etc. ~Won't fix~ Maybe some day

2. We had that before, but it gave us more problems as users didn't know what to change and ended up in making issues like "my network isn't working". As you said you can always change it afterwards.
   Also, this is kind of what you were asking for in you first question. We made it default, and if you want to change we give you instructions how to. Won't fix

3.

Really not a good idea

Why? You can have your whole VM on another storage, and there is no difference in backing up your external storage or backing up your VM storage. Tech and Me have the whole hypervisor storage on a FreeNAS which means that the VMs itself are hosted in a "External Storage" with no disks on the serrver itself. Backups are made with both Veeam and Rsync - just as an example.

For people that want larger space, we have both a guide how to expand your disk, and we have pre-configured larger VMs. If you want external storage over NFS we have guides for that as well on https://www.techandme.se/

Feel free to make a PR to solve the issue and I'll have a look and implement it.

[thejohnha]

I actually agree with @Hamsterman on point #1. It would be very nice to be able to answer all of the script's questions in the beginning instead of having to wait for the script and continuously check in on it like a pot of risotto.

[enoch85]

@thejohnha Thanks for your input.

Things happen in a special order because it's needed. E.g SSL must be installed and running to be able to use Collabora or OnlyOffice, the while loop with all the extra apps is a section in itself, also the change of MariaDB PW must take place where it does.

I'm thinking it may be possible to have a menu in the beginning maybe and make all the different sections in the script a function instead. Though it would require quite much effort and time which I don't have right now. This is a "nice to have" improvement and nothing critical.

Feel free to make a PR and contribute back to the project. :)

[thejohnha]

By the way, thanks for all your hard work on this project, Daniel.
I was thinking it would be nice to have all of the constants in a configuration file, similar to like an httpd.conf file, with all of the answers to the script's questions. Even putting in the MariaDB PW there while the script runs. Then just delete the script afterwards.

I think it would be far more efficient if you edited the script because I'm not a programmer. But if you need sponsorship, perhaps I can buy some support credits from your website for the work.

[enoch85]

@thejohnha Thanks for the offer. The thing is I have a full time job, plus all the support I give to already paying customers.

So time is very limited right now.

[enoch85]

If someone wants to work with this, @PietsHost has a nice script here: https://github.com/PietsHost/Nextcloud-Installation-Script/blob/master/nc_install.sh that you can get inspiration from.

[thejohnha]

Thanks @enoch85.
I'm not a software engineer, but I forked your repo (at https://github.com/thejohnha/vm/) and tried the following as a simple test:

1. put a new variable near the top of nextcloud_install_production.sh like this: CREATE_NEW_USER="no"
2. nest the if statement in adduser.sh with:
   if [ $CREATE_NEW_USER != "no" ] then if [[ "no" == $(ask_yes_or_no "Do you want to create a new user?") ]]]

But that didn't seem to work. Perhaps the variable isn't getting passed correctly to adduser.sh ?

Also, I think the script will always pull lib.sh from the official repo. Is there a better way to use my fork for testing than just git clone my_fork and then running sudo bash on the script?

If you think you can give me some tips to get started I would be happy to buy a 30 or 60 support package from your website. Let me know, thanks.

[enoch85]

@thejohnha The reason why it pulls the lib.sh from master is that yo have to change this line in every script so that it pulls from your master instead. You also have to put every variable in the lib.sh that you want to use in the code.

I don't get what you want to achive with your code, do you want to skip the question if you want to create a new user? Bad idea as if the user is root or something else like empty, or just a dot the script will fail. So if you really want to skip that question and go with the default "ncadmin" then I would test all possible scenarios (there are many) and skip only if one of those scenarios doesn't match the pattern.

But, then we have users that do want to use their own users because they have that specific user in all their other systems, so leaving out that question would remove that "feature".

What I thought you all meant from start was the setup script. My thought from start was some kind of "menu" where you could set options and passwords and stuff, then just let the script run by itsself. That would be the nicest solution but far more complex than what you guys are talking about. I would guess it will take about 12 hours or more to finish.

[thejohnha]

Daniel, thank you for explaining the adduser "feature" to me. I simply chose that because it was the script's first question, I thought I would start there and try to eliminate that question as a test for myself (sorry I'm not a software engineer, I only manage software teams so I am not as good with the finer details of actual scripting when I'm by myself!).

Your idea of having a menu system in the beginning is perfect. But I won't be able to do that myself, so I am looking for a minimally viable product I can hack together. :-)

Perhaps there is a better way to add my desired variables so I can try to fully automate the script according to my use cases. Could you suggest another file where I could put my variables? Or should I change that line you mentioned in all of the scripts.

[enoch85]

@thejohnha What you will need:

1. Plenty of spare time
2. Good Googling skills
3. A passion for making the world a better place (Nextcloud makes the world a better place)

So with that said. I can help you sure. But as I said, my time is very limited and I almost get no sleep right now. I don't count myself as a software engineer either, I'm just passionate about the thought that people can host their own data in an easy way (as in downloading a file, mount it and then you're set).

So, I can give you pointers. For example, what would do is to look at this and make a menu out of that with variables as you already tried to. Whiptail has a lot to offer in different boxes and settings, so check it out.

Ask if you need anything.

[enoch85]

the function msg_box has been implemented for a while now, maybe that makes stuff easier for someone who wants to take a look at this...

[thejohnha]

Is msg_box the full screen pink menu? Perhaps an addition of a default number of seconds before it continues automatically using the default (if possible, of course. certain questions won't be possible to auto-continue like letsencrypt stuff or non-critical questions). Taking inspiration from GRUB where it automatically selects an option and continues after a few seconds. For this script, perhaps if the user doesn't select the closest mirror within 90 seconds, it will just continue with the default mirror.

[enoch85]

Take a look for your self: https://github.com/nextcloud/vm/blob/master/lib.sh#L177-L180

You could adapt the whiptail to do different stuff, and preferably make it a new function.

[thejohnha]

Wow, that's cool - thank you. I checked man whiptail but there doesn't seem to be any default time settings in there. Thanks for the education - now I know about whiptail (v ncurses etc). :-)

[enoch85]

Thanks for the education - now I know about whiptail (v ncurses etc). :-)

Great! So now would be the time to send a PR. ;)

[thejohnha]

Send a PR to the whiptail project implementing a timeout?

[enoch85]

If I understand you correctly you want a menu in the beginning of the startup script. Implement that, or whatever you want and send a PR so that I can have a look. :)

[thejohnha]

Yes understood. I just don't have the skills to do that at the moment. Thanks Daniel.

[enoch85]

If you don't try you will never succeed. When I started with Nextcloud (ownCloud) I didn't know a single line of Linux commands, look at me now. :)

[thejohnha]

You are a super-star!! I am seriously impressed. :-)

I basically just want to accept the default mirror location and keyboard. Otherwise it just slows me down when trying to deploy or test the script.

[enoch85]

Hehe, slow things down? It's only two choices. Once you've done it a thousand times you know exactly when and what to press. ;)

[enoch85]

This is something that @riegercloud could work on maybe? It would be a good first project to get to know the bash code.

What I'm thinking is a whiptail menu in the beginnning where you choose all your defaults, then when you hit enter everything will be according to what you chose. For examlple, if you choose Swedish as you keyboard then Swedish should automatically be chosen during the first-setup.

[enoch85]

So guys, the scripts has evolved and I consider them so stable that it's a matter of maintaining what's online right now. Maybe some bigger changes will come, but I'm quite happy with how it works right now.

What do you think @Hamsterman @thejohnha?

[enoch85]

We are planning a GUI (written in PHP) to deal with Point 1.

Point 2 is fixed and will be released in the next version of Nextcloud.

Point 3 is somewhat fixed (not external but still) since we use another disk for storage. If someone wants to add SMB that's easily done, but would also require setting up stuff on the other end which the scripts won't do for you.

That said, once point 1 is developed I consider this issue closed.

[thejohnha]

That said, once point 1 is developed I consider this issue closed.

Cool - thank you! I'm assuming it will have a menu of options that you can choose up front, and then the installer will do everything without needing attention?

[enoch85]

Cool - thank you! I'm assuming it will have a menu of options that you can choose up front, and then the installer will do everything without needing attention?

It's just in the planning stages still but I'm hoping that we can make it as easy as possible, yes.

[thejohnha]

Awesome! Thank you for all your hard work, Daniel!

[ggeorgg]

I also wanted to have a more straight forward setup of my server so I created a fork. Have a look here. You need to download the main.sh file and execute it with sudo bash main.sh.
There is still a lot of work to be done and just the basic things are working. At the beginning you can choose if you want to have no interaction, a simple setup, or an advanced setup. What do you think of it?
Note: I currently have some troubles with the line endings, so it may be that the script won't run at all.

[enoch85]

@ggeorgg Instead of forking this project, why don't you contribute instead? :confused:

[enoch85]

For example; you posted an issue here and reading your scripts it seems like you solved it?

[ggeorgg]

Sorry, but I first had to gather some experience in bash scripting. Moreover the changes I made are quite big I think, so I am not sure how I could bring them into this project here. Any suggestions? What plans have you made so far for the quick default installation?

Edit:
I would really like to work together to enhance this project, so don't get me wrong when I'm working on my own github repository ;)

[enoch85]

@ggeorgg What this issue is about is not "a quick default setup" per say. It more like "I want the setup to be less technical", right? In the end the user should only have to answer questions to let the scripts do its thing which is the way things work now as well, but instead of hiding the output we show it since you can see what happens at all times.

There are different philosophies on how scripting should be done. Your way (since I just tested) is answering some questions at the beginning of the script but in general, it does the same thing.

Our plan is to:

We are planning a GUI (written in PHP) to deal with Point 1.

Which basically means that you will visit your local IP and just click your way through the whole process on a web page instead of visiting your console, and only output debug messages when it fails. It will take loooong time before it's stable and my time is limited, plus I need to hire a PHP dev to do it, from my own pocket.

It's great that you are learning and I would be very happy if someone could help me maintain this as I'm alone right now and I don't have that much time as I have a full time job + a company to run at the same time.

How you would implement it is the same way as you do right now in your own fork. Fork this project, test test test test test, and when you have a working fix, send a PR to this repo. :)

[enoch85]

Also, having too many options will confuse the user. What we do here is giving a stable and easy way to install Nextcloud, on your own server with the scripts, or as a pre-configured VM.

Making things too complicated also makes the risk higher for something to break, and we don't want that. So, small steps, but stable steps are the way forward if you ask me.

If you press [enter] through the whole installation you will get a basic VM, but you can choose to add more features, like SSL, apps and such. Keep in mind that this is made for people without Linux skills so it has to work. I don't know if this VM just works really good or if users don't post issues, but it never fails on me when I install it myself to customers which is a good sign at least. :)

[ggeorgg]

@ggeorgg What this issue is about is not "a quick default setup" per say. It more like "I want the setup to be less technical", right? In the end the user should only have to answer questions to let the scripts do its thing which is the way things work now as well, but instead of hiding the output we show it since you can see what happens at all times.

I would say that this point from @Hamsterman is very important to many users:
"If you want the installation to be very smooth then put a list of selections in the beginning and then an option for selecting each one of them. I think it would work for many of the question (but probably not all)."
So in my opinion it is not the point "to be less technical". This is just nice side win.

There are different philosophies on how scripting should be done. Your way (since I just tested) is answering some questions at the beginning of the script but in general, it does the same thing.

Our plan is to:

We are planning a GUI (written in PHP) to deal with Point 1.

Which basically means that you will visit your local IP and just click your way through the whole process on a web page instead of visiting your console, and only output debug messages when it fails. It will take loooong time before it's stable and my time is limited, plus I need to hire a PHP dev to do it, from my own pocket.

A GUI written in PHP would be very nice. But it won't be possible to use it on a clean Ubuntu Server version? We don't have a webbrowser. We would have to install a desktop first.

It's great that you are learning and I would be very happy if someone could help me maintain this as I'm alone right now and I don't have that much time as I have a full time job + a company to run at the same time.

I also do have a full time job and I am already working the whole day with a computer. So for me it also would be better to not have a whole project waiting for me at home :D

How you would implement it is the same way as you do right now in your own fork. Fork this project, test test test test test, and when you have a working fix, send a PR to this repo. :)

Yes, thats what I understood. But If the changes are that big, it is more or less a complete new version...

Also, having too many options will confuse the user. What we do here is giving a stable and easy way to install Nextcloud, on your own server with the scripts, or as a pre-configured VM.

Thats right. I am perfectly with you at point. That is why I put the first questions in my script that will ask the user how many questions he or she wants to see.

Making things too complicated also makes the risk higher for something to break, and we don't want that. So, small steps, but stable steps are the way forward if you ask me.

I admit. And I have to say, that many options in my script (e.g. apache2 or NGINX) are no real options right now because NGINX is not implemented yet. But that would be cool to easily test different setups.

[enoch85]

Yes, that's what I understood. But If the changes are that big, it is more or less a complete new version...

No, it's easier to work on something that already exists and improve it than to write something brand new IMHO. Think of a way to implement your fix into the existing scripts. For example, ask the user if he|she have another disk and if so let them choose. That's basically what you do now, right?

Scripting is hard because of it's so limited and static, luckily there is if arguments, but it's still a matter of predicting what the end user wants - and there as there are alooooooooot of options then you need to prioritize - hence we chose to go with a second drive and made it compatible (so far) with KVM, Hyper-V, DigitalOcean and VMware. Because if you want to make it possible for the user to make a choice you must also think of all the different scenarios and make all of them work. You wouldn't want the script to fail in the middle of the run, because that would lead to a failed installation.

I admit. And I have to say, that many options in my script (e.g. apache2 or NGINX)

Good for testing purposes but nothing I would put in production as you must change everything depending on which web server you use. Not even the PHP-FPM implementation would look the same in any of the options, not the paths either. There are just too many things that could go wrong giving such an option.

"If you want the installation to be very smooth then put a list of selections in the beginning and then an option for selecting each one of them.

That's what we will do with the PHP GUI, and that's not intended for the install script, but for the VM and therefore the setup script.

As I said earlier, you're very welcome aboard. I'm going to Germany soon, maybe we should meet? Send me an email!

[enoch85]

Closing this as pressing enter through the script will give you a default installation.

[enoch85]

Also, there will be a GUI in the future. Can't say when though.