Vm: Bitwarden not installing although selected during initial setup

[mamama1]

Hi

as mentioned in https://github.com/bitwarden/server/issues/860#issuecomment-673750736, I just deployed a brand new VM using a 1TB ova with changed timestamp 16.07.2020 20:11 and Bitwarden was not installed although selected during the first setup.
Like I mentioned in the linked issue, I briefly saw some red text saying /home/bitwarden not found or something.

After the first run setup completed, I did run menu.sh again to install Bitwarden and that time the installation worked and Bitwarden is up and running.

Main question, as I'm getting a little bit tired of redeploying VMs and starting from scratch ;-) Is this VM now in a "supported" state where I can expect it to work or can I expect it to be in a kind of weird state now where I shouldn't put it into "production" (at home, but still...)?

Another confusing thing: Webmin is now selectable during the first run setup but even if I do not choose to install it, it is being installed anyway. That's a bit misleading...

Thanks!

PS: do you still want me to debug the issue with the previous VM iteration where the new install script (where bitwarden appearently gets installed to /home/bitwarden) didn't work on an appliance where Bitwarden was previously installed using the old scipt (where bitwarden got installed to /root) or can/should we ditch that one?

[enoch85]

After the first run setup completed, I did run menu.sh again to install Bitwarden and that time the installation worked and Bitwarden is up and running.

That tells me that you managed to install if just at the same time we pushed the PR which probably lead to that Github caching didn't catch up on all the changes in the file you got. That happens to me all the time during testing

To answer your question though, it's stable now, so go ahead and put in it production after you reinstalled it one last time. :D

Also, as you said yourself; it works with the latest script so horaay to that!

Another confusing thing: Webmin is now selectable during the first run setup but even if I do not choose to install it.

No, in your VM Webmin is installed from the beginning (before it's released). In the next release that won't be the case.We fixed the scripts which runs live now, but we can't remove Webmin from the already released VMs. :)

PS: do you still want me to debug the issue with the previous VM iteration where the new install script (where bitwarden appearently gets installed to /home/bitwarden) didn't work on an appliance where Bitwarden was previously installed using the old scipt (where bitwarden got installed to /root) or can/should we ditch that one?

Please try to reinstall it again with the latest script and post your findings, if any.

Thanks!

[enoch85]

JFTR: https://github.com/nextcloud/vm/pull/1366

[szaimen]

I briefly saw some red text saying /home/bitwarden not found or something.

Hm.... this sounds like the userfolder isn't successfully created here: https://github.com/nextcloud/vm/blob/2d323bfbc12dd816dfbe18f315c7c6bcb0419374/apps/tmbitwarden.sh#L86
and then it fails the check here: https://github.com/nextcloud/vm/blob/2d323bfbc12dd816dfbe18f315c7c6bcb0419374/apps/tmbitwarden.sh#L125

We could just add the following code in this line: https://github.com/nextcloud/vm/blob/2d323bfbc12dd816dfbe18f315c7c6bcb0419374/apps/tmbitwarden.sh#L93

mkdir -p "$BITWARDEN_HOME"
chown "$BITWARDEN_USER":"$BITWARDEN_USER" "$BITWARDEN_HOME"

@enoch85 What do you think?

[enoch85]

Sure, add it. It doesn't hurt. :)

[enoch85]

But, user $HOME should be created during the user creation, so if that failed we have another issue.

I actually removed above code yesterday.

[szaimen]

So shall I add it or not?

[mamama1]

I'm going to deploy one more brand new VM for testing the current state of everything, I'd suggest to wait with your (@szaimen ) proposed change until we know how the deployment goes now?

edit: I'll also redownload the VM image, just to be sure.

[enoch85]

Just tested, users $HOME is created. But, maybe we should add a check_command. I'll take care of that.

[mamama1]

That tells me that you managed to install if just at the same time we pushed the PR which probably lead to that Github caching didn't catch up on all the changes in the file you got. That happens to me all the time during testing

Maybe, I installed the appliance 14.08.2020 around 00:30 MESZ.

No, in your VM Webmin is installed from the beginning (before it's released). In the next release that won't be the case.We fixed the scripts which runs live now, but we can't remove Webmin from the already released VMs. :)

Makes sense :-)

Please try to reinstall it again with the latest script and post your findings, if any.

will do.

Just tested, users $HOME is created. But, maybe we should add a check_command. I'll take care of that.

Shall I wait with the deployment of the brand new VM? Has the 1 TB ova been updated since 16.07.2020 20:11? That's the timestamp my file has.

[enoch85]

All the scripts except the startup script, updates in the released VM when we push to master, so please go ahead and try.

Thanks!

[mamama1]

PS: do you still want me to debug the issue with the previous VM iteration where the new install script (where bitwarden appearently gets installed to /home/bitwarden) didn't work on an appliance where Bitwarden was previously installed using the old scipt (where bitwarden got installed to /root) or can/should we ditch that one?

Please try to reinstall it again with the latest script and post your findings, if any.

Still doesn't work, this time i got this output (did /root/bitwarden.sh stop && docker system prune -af && rm -rf /root/bwdata and hash -r):

Docker version 19.03.12, build 48a66213fe
docker-compose version 1.25.0, build unknown

1.36.1: Pulling from bitwarden/setup
Digest: sha256:5b2c43b46c03da54aecc6d19098b0574b5fb205f0cd3046019725925cadfdf29
Status: Image is up to date for bitwarden/setup:1.36.1
docker.io/bitwarden/setup:1.36.1

Migrating database.
Unhandled exception. System.Data.SqlClient.SqlException (0x80131904): Login failed for user 'sa'.
   at Bit.Setup.Program.MigrateDatabase(Int32 attempt) in /home/runner/work/server/server/util/Setup/Program.cs:line 193
   at Bit.Setup.Program.Update() in /home/runner/work/server/server/util/Setup/Program.cs:line 140
   at Bit.Setup.Program.Main(String[] args) in /home/runner/work/server/server/util/Setup/Program.cs:line 65
ClientConnectionId:b06f37f6-aa39-498e-81d0-2e64e3278395
Error Number:18456,State:1,Class:14
Sorry but something went wrong. Please report this issue to https://github.com/nextcloud/vm/issues and include the output of the error message. Thank you!
sudo -u bitwarden ./bitwarden.sh updatedb failed
Posting notification to users that are admins, this might take a while...
Posting 'Sorry but something went wrong. Please report this issue to https://github.com/nextcloud/vm/issues and include the output of the error message. Thank you!' to: lnadmin

From my point of view we can stop debugging that but I can do more tests if you want me to.
Will now turn that VM off for now and deploy a new one.

[enoch85]

@szaimen Do we need updatedb at all? I'm starting to think it's not necessary?

@mamama1 Could you please remove sudo -u bitwarden ./bitwarden.sh updatedb from the code before running the script and see if it works? You'll need to download the script manually.

If that doesn't work, please add sleep 10before sudo -u bitwarden ./bitwarden.sh updatedb. I think it's due to that the DB hasn't had time to come up yet.

[szaimen]

@szaimen Do we need updatedb at all? I'm starting to think it's not necessary?

I wonder, too!

@mamama1 Could you please remove sudo -u bitwarden ./bitwarden.sh updatedb from the code before running the script and see if it works? You'll need to download the script manually.

yeah, would be great if we wouldn't need this at all.

If that doesn't work, please add sleep 10before sudo -u bitwarden ./bitwarden.sh updatedb. I think it's due to that the DB hasn't had time to come up yet.

Sounds like a workaround that could work 👍

[mamama1]

i did add a delay of 30 seconds before the updatedb command and bitwarden installed without an error.
before that, even on a fresh appliance the installation failed with the updatedb error during the first run setup and even if I tried it a second or third time afterwards using menu.sh.

I don't know what updatedb is doing but waiting n amount of time doesn't put much confidence into this for me as time could hugely vary on different hardware...

[enoch85]

I will test this some more, but thanks for the info so far.

Right now I'm leaning towards removing it altogether.

[enoch85]

I think it's a RAM issue, you need more.

Made some changes, please try again.

Thanks!

[szaimen]

So I guess we cannot get rid of this https://github.com/nextcloud/vm/blob/2d5e65787e61382ab7ee46ffeafe0a02509e6a1f/apps/tmbitwarden.sh#L171 can we?

---

Also I think that at this line: https://github.com/nextcloud/vm/blob/2d5e65787e61382ab7ee46ffeafe0a02509e6a1f/apps/tmbitwarden.sh#L168
Would be check_command sudo -u "$BITWARDEN_USER" ./bitwarden.sh start better because you can see what happens. if we just start the service you see nothing happening, depending on the hardware for a long time (at worst probably one or two minutes)

[enoch85]

Would be check_command sudo -u "$BITWARDEN_USER" ./bitwarden.sh start better b

Sure, change it.

So I guess we cannot get rid of this

Please read the comment in the code. ;)

[enoch85]

For me, it works 90% of the times, and it's always working on the second attempt. Don't know why, but that's the case.

I haven't tested after I've added sleep.

[enoch85]

Actually, it worked now with the added sleep and increased RAM.

Please confirm @szaimen and @mamama1. I think we can close this issue now.

[enoch85]

OK, last change done now. Now the database runs as a docker volume.

Can't do more with this issue, everything else is Bitwarden related.

[enoch85]

Wrong button...

Please confirm that it works now.

[mamama1]

will do, hopefully tomorrow. btw, 8GB should be plenty of RAM... :-)

[szaimen]

OK, last change done now. Now the database runs as a docker volume.

Hm, haven't seen this change inside the script, I also don't like the idea because it will probably make backup and restore much more complicated.

[enoch85]

backup and restore much more complicated.

How?

[szaimen]

Please read the comment in the code. ;)

Ah okay, I get it :D (just a bit confusing that the comment is now not directly above the updatedb command...)

[szaimen]

How?

I don't know for sure...
But if the database is only in a dockercontainer, this is probably not possible...

[enoch85]

Ok, tested 5 times now (peew), 100% success!

[szaimen]

Nice! Congratulations! :)

[enoch85]

But if the database is only in a dockercontainer, this is probably not possible...

I'm sure you can move Docker volumes. Never done it, but everything else would be stupid imho. :) I mean why create volumes if you can't move them. Makes no sense. :)

[szaimen]

But if the database is only in a dockercontainer, this is probably not possible...

I'm sure you can move Docker volumes. Never done it, but everything else would be stupid imho. :) I mean why create volumes if you can't move them. Makes no sense. :)

No, what I mean is that if the database is not in a container and just stored anywhere in /home/bitwarden/bwdata, you can just copy the whole folder and have all important files. If on the other hand the database is inside a docker container, you need to copy the file out of the container to the same place outside of the container which is much less convenient...

[szaimen]

Where is the line where you added the database to a container?

[enoch85]

What I mean is that you can probably move the whole container and attach it to a new docker instance.

[szaimen]

This is possible, yes 👍

[szaimen]

ah okay, found it

[enoch85]

6 times now, success! Closing this issue now, please confirm that it works anyway.

Where is the line where you added the database to a container?

https://github.com/nextcloud/vm/blob/master/apps/tmbitwarden.sh#L160

[szaimen]

I still vote for reverting the usage of database_docker_volume.

This would be the backup/restore procedure: https://github.com/bitwarden/server/issues/373#issuecomment-498443578

[szaimen]

Does it provide such an improvement in install-consistency over not using this option?

[enoch85]

Does it provide such an improvement in install-consistency over not using this option?

Well, it does seem to solve the updateddb issue. :laughing:

[szaimen]

Does it provide such an improvement in install-consistency over not using this option?

Well, it does seem to solve the updateddb issue. 😆

So without this option it was still not 100% stable?

[enoch85]

Correct.

[enoch85]

@szaimen You could still experiment without it if you like. I just wanted something quick that solved the issue. But if you can make it work on-disk it would be better maybe..

[szaimen]

I've changed my mind since I've seen that this docker volume seem to be store on the host system under:
"/var/lib/docker/volumes/docker_mssql_data/_data",`

[mamama1]

regarding backup, this is how it is supposed to work, so no issues with mssql being in a container (that seems to be the default setting anyway): https://bitwarden.com/help/article/backup-on-premise/

regarding updatedb: do we know what it is doing and whether we may need it for something (maybe in the future)?

[enoch85]

@mamama1 Can you please confirm that the new script works?