Server: Issues with ./bitwarden.sh updatedb and bitwarden/mssql:1.15.1

[aNaravi]

There seem to be two related issues that are preventing me from using a locally installed version of bitwarden.

1. After running
   curl -s -o bitwarden.sh \ https://raw.githubusercontent.com/bitwarden/core/master/scripts/bitwarden.sh \ && sudo chmod u+x bitwarden.sh
./bitwarden.sh install
./bitwarden.sh start
   bitwarden was installed and running:

Status: Image is up to date for bitwarden/setup:1.15.1

bitwarden is up and running!

visit https://localhost
to update, run './bitwarden.sh updateself' and then './bitwarden.sh update'

However I got an error with ./bitwarden.sh updatedb:

Docker version 17.09.0-ce, build afdb6d4
docker-compose version 1.17.0, build ac53b73

1.15.1: Pulling from bitwarden/setup
Digest: sha256:9d4fd9ea670b2ee84eeb19278d6a80bee10f5a50796b3a5f11af1dbdd4b052cf
Status: Image is up to date for bitwarden/setup:1.15.1
Migrating database.

Unhandled Exception: System.Data.SqlClient.SqlException: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 40 - Could not open a connection to SQL Server)
at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, Object providerInfo, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling)
at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource1 retry, DbConnectionOptions userOptions) at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource1 retry)
at System.Data.SqlClient.SqlConnection.Open()
at Bit.Setup.Program.MigrateDatabase() in /Users/kyle/Projects/bitwarden/core/util/Setup/Program.cs:line 244
at Bit.Setup.Program.Main(String[] args) in /Users/kyle/Projects/bitwarden/core/util/Setup/Program.cs:line 55

I can't create an account at https://localhost/#/register (as a result of this?) Error Message: "An unexpected error has occurred."

1. I also noticed that the mysql container keeps restarting every minute or so.

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
34283bd01ae4 bitwarden/web:1.21.0 "/entrypoint.sh" About a minute ago Up About a minute 80/tcp web
31a7f9a62a7f bitwarden/mssql:1.15.1 "/entrypoint.sh" About a minute ago Restarting (1) 2 seconds ago mssql
f9c21d55f93b bitwarden/identity:1.15.1 "/entrypoint.sh" About a minute ago Up About a minute 80/tcp identity
959c19ec290a bitwarden/api:1.15.1 "/entrypoint.sh" About a minute ago Up About a minute 80/tcp api
7149b9d14566 bitwarden/icons:1.15.1 "/entrypoint.sh" About a minute ago Up About a minute 80/tcp icons
7ab690f9fbef bitwarden/nginx:1.15.1 "/entrypoint.sh" About a minute ago Up About a minute 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp nginx
0a5d0cf90785 bitwarden/attachments:1.15.1 "/entrypoint.sh" About a minute ago Up About a minute 80/tcp attachments

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
34283bd01ae4 bitwarden/web:1.21.0 "/entrypoint.sh" 4 minutes ago Up 3 minutes 80/tcp web
31a7f9a62a7f bitwarden/mssql:1.15.1 "/entrypoint.sh" 4 minutes ago Restarting (1) 15 seconds ago mssql
f9c21d55f93b bitwarden/identity:1.15.1 "/entrypoint.sh" 4 minutes ago Up 3 minutes 80/tcp identity
959c19ec290a bitwarden/api:1.15.1 "/entrypoint.sh" 4 minutes ago Up 3 minutes 80/tcp api
7149b9d14566 bitwarden/icons:1.15.1 "/entrypoint.sh" 4 minutes ago Up 3 minutes 80/tcp icons
7ab690f9fbef bitwarden/nginx:1.15.1 "/entrypoint.sh" 4 minutes ago Up 3 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp nginx
0a5d0cf90785 bitwarden/attachments:1.15.1 "/entrypoint.sh" 4 minutes ago Up 3 minutes 80/tcp attachments

[kspearrin]

Your database is not running. It's crashing for some reason. Do you have enough RAM on the machine? mssql requires 2GB

[aNaravi]

I've got 4GB RAM and 4GB Swap Space.
Since you mentioned 2GB I tried updating again while watching RAM usage (having shut down all other applications). The mssql process consumes about 300-400MB before crashing (same error message). So I don't think RAM is the issue.

[kspearrin]

What do the docker logs say when starting mssql?

[aNaravi]

Says something about invalid startup options:

2017-12-06 20:17:30.27 Server The licensing PID was successfully processed. The new edition is [Express Edition].
2017-12-06 20:17:30.81 Server Microsoft SQL Server 2017 (RTM-CU2) (KB4052574) - 14.0.3008.27 (X64)
Nov 16 2017 10:00:49
Copyright (C) 2017 Microsoft Corporation
Express Edition (64-bit) on Linux (Ubuntu 16.04.3 LTS)
2017-12-06 20:17:30.82 Server UTC adjustment: 0:00
2017-12-06 20:17:30.82 Server (c) Microsoft Corporation.
2017-12-06 20:17:30.82 Server All rights reserved.
2017-12-06 20:17:30.82 Server Server process ID is 4116.
2017-12-06 20:17:30.82 Server Logging SQL Server messages in file '/var/opt/mssql/log/errorlog'.
2017-12-06 20:17:30.82 Server Registry startup parameters:
-d /var/opt/mssql/data/master.mdf
-l /var/opt/mssql/data/mastlog.ldf
-e /var/opt/mssql/log/errorlog
2017-12-06 20:17:30.85 Server Error: 17113, Severity: 16, State: 1.
2017-12-06 20:17:30.85 Server Error 87(The parameter is incorrect.) occurred while opening file '/var/opt/mssql/data/master.mdf' to obtain configuration information at startup. An invalid startup option might have caused the error. Verify your startup options, and correct or remove them if necessary.
2017-12-06 20:17:46.65 Server Microsoft SQL Server 2017 (RTM-CU2) (KB4052574) - 14.0.3008.27 (X64)
Nov 16 2017 10:00:49
Copyright (C) 2017 Microsoft Corporation
Express Edition (64-bit) on Linux (Ubuntu 16.04.3 LTS)
2017-12-06 20:17:46.65 Server UTC adjustment: 0:00
2017-12-06 20:17:46.65 Server (c) Microsoft Corporation.
2017-12-06 20:17:46.65 Server All rights reserved.
2017-12-06 20:17:46.65 Server Server process ID is 4116.
2017-12-06 20:17:46.66 Server Logging SQL Server messages in file '/var/opt/mssql/log/errorlog'.
2017-12-06 20:17:46.66 Server Registry startup parameters:
-d /var/opt/mssql/data/master.mdf
-l /var/opt/mssql/data/mastlog.ldf
-e /var/opt/mssql/log/errorlog
2017-12-06 20:17:46.69 Server Error: 17113, Severity: 16, State: 1.
2017-12-06 20:17:46.69 Server Error 87(The parameter is incorrect.) occurred while opening file '/var/opt/mssql/data/master.mdf' to obtain configuration information at startup. An invalid startup option might have caused the error. Verify your startup options, and correct or remove them if necessary.

This error did not occur immediately after starting bitwarden. This time I didn't even try to run updatedb. The logs started showing the error when mssql's container's status started showing "Restarting (1) ..."

[kspearrin]

What is the host OS?

[aNaravi]

Xubuntu 16.04

[kspearrin]

I am not really sure. Could you try deleting ./bwdata folder and just starting over?

[aNaravi]

I've actually tried that. This is the second installation.
I was having the same problem with mssql earlier. But I had run ./bitwarden.sh update rather than ./bitwarden.sh updatedb so I think the error was different.

[kspearrin]

The update commands won't change anything. The container won't even start up.

[aNaravi]

It wasn't intentional. I'd misread the command from the article. Anyway, I did reinstall by deleting bwdata and all the images docker had downloaded. Then I ran ./bitwarden install.
Edit: I hadn't deleted the certbot image.

Edit 2: I did it all over again. Deleted bwdata and all docker images, then reinstalled. I'm still having the same problem.

[kspearrin]

Can you try starting things with docker-compose directly so we can get more error output?

docker-compose -f ./bwdata/docker/docker-compose.yml up

[aNaravi]

Creating network "docker_default" with the default driver
Creating attachments ...
Creating api ...
Creating identity ...
Creating icons ...
Creating web ...
Creating mssql ...
Creating nginx ...
Creating api
Creating attachments
Creating web
Creating icons
Creating mssql
Creating nginx
Creating mssql ... done
Attaching to web, nginx, icons, attachments, identity, api, mssql
icons | iconserver v3.4.0 (2017-05-29) (go1.8.3) - https://github.com/mat/besticon
icons | server: 2017/12/08 09:49:39.296476 Starting server on 0.0.0.0:8080...
identity | Hosting environment: Production
identity | Content root path: /app
identity | Now listening on: http://[::]:80
identity | Application started. Press Ctrl+C to shut down.
api | Hosting environment: Production
api | Content root path: /app
api | Now listening on: http://[::]:80
api | Application started. Press Ctrl+C to shut down.
attachments | Hosting environment: Production
attachments | Content root path: /etc/bitwarden/core/attachments
attachments | Now listening on: http://[::]:80
attachments | Application started. Press Ctrl+C to shut down.
web | Hosting environment: Production
web | Content root path: /app
web | Now listening on: http://[::]:80
web | Application started. Press Ctrl+C to shut down.
icons | warn: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35]
icons | No XML encryptor configured. Key {f924bdb7-eac5-4d6d-8654-4ace84e3b167} may be persisted to storage in unencrypted form.
icons | Hosting environment: Production
icons | Content root path: /app
icons | Now listening on: http://[::]:80
icons | Application started. Press Ctrl+C to shut down.

mssql | 2017-12-08 09:51:25.33 Server The licensing PID was successfully processed. The new edition is [Express Edition].
2017-12-08 09:51:26.12 Server Microsoft SQL Server 2017 (RTM-CU2) (KB4052574) - 14.0.3008.27 (X64)
mssql Nov 16 2017 10:00:49
mssql Copyright (C) 2017 Microsoft Corporation
mssql Express Edition (64-bit) on Linux (Ubuntu 16.04.3 LTS)
2017-12-08 09:51:26.12 Server UTC adjustment: 0:00
2017-12-08 09:51:26.13 Server (c) Microsoft Corporation.
2017-12-08 09:51:26.13 Server All rights reserved.
2017-12-08 09:51:26.13 Server Server process ID is 4116.
2017-12-08 09:51:26.13 Server Logging SQL Server messages in file '/var/opt/mssql/log/errorlog'.
2017-12-08 09:51:26.13 Server Registry startup parameters:
mssql -d /var/opt/mssql/data/master.mdf
mssql -l /var/opt/mssql/data/mastlog.ldf
mssql -e /var/opt/mssql/log/errorlog
2017-12-08 09:51:26.22 Server Error: 17113, Severity: 16, State: 1.
2017-12-08 09:51:26.22 Server Error 87(The parameter is incorrect.) occurred while opening file '/var/opt/mssql/data/master.mdf' to obtain configuration information at startup. An invalid startup option might have caused the error. Verify your startup options, and correct or remove them if necessary.
mssql |
mssql exited with code 1
mssql | 2017-12-08 09:51:25.33 Server The licensing PID was successfully processed. The new edition is [Express Edition].
2017-12-08 09:51:26.12 Server Microsoft SQL Server 2017 (RTM-CU2) (KB4052574) - 14.0.3008.27 (X64)
mssql Nov 16 2017 10:00:49
mssql Copyright (C) 2017 Microsoft Corporation
mssql Express Edition (64-bit) on Linux (Ubuntu 16.04.3 LTS)
2017-12-08 09:51:26.12 Server UTC adjustment: 0:00
2017-12-08 09:51:26.13 Server (c) Microsoft Corporation.
2017-12-08 09:51:26.13 Server All rights reserved.
2017-12-08 09:51:26.13 Server Server process ID is 4116.
2017-12-08 09:51:26.13 Server Logging SQL Server messages in file '/var/opt/mssql/log/errorlog'.
2017-12-08 09:51:26.13 Server Registry startup parameters:
mssql -d /var/opt/mssql/data/master.mdf
mssql -l /var/opt/mssql/data/mastlog.ldf
mssql -e /var/opt/mssql/log/errorlog
2017-12-08 09:51:26.22 Server Error: 17113, Severity: 16, State: 1.
2017-12-08 09:51:26.22 Server Error 87(The parameter is incorrect.) occurred while opening file '/var/opt/mssql/data/master.mdf' to obtain configuration information at startup. An invalid startup option might have caused the error. Verify your startup options, and correct or remove them if necessary.
2017-12-08 09:51:56.13 Server Microsoft SQL Server 2017 (RTM-CU2) (KB4052574) - 14.0.3008.27 (X64)
mssql Nov 16 2017 10:00:49
mssql Copyright (C) 2017 Microsoft Corporation
mssql Express Edition (64-bit) on Linux (Ubuntu 16.04.3 LTS)
2017-12-08 09:51:56.13 Server UTC adjustment: 0:00
2017-12-08 09:51:56.13 Server (c) Microsoft Corporation.
2017-12-08 09:51:56.13 Server All rights reserved.
2017-12-08 09:51:56.14 Server Server process ID is 4116.
2017-12-08 09:51:56.14 Server Logging SQL Server messages in file '/var/opt/mssql/log/errorlog'.
2017-12-08 09:51:56.14 Server Registry startup parameters:
mssql -d /var/opt/mssql/data/master.mdf
mssql -l /var/opt/mssql/data/mastlog.ldf
mssql -e /var/opt/mssql/log/errorlog
2017-12-08 09:51:56.14 Server Error: 17113, Severity: 16, State: 1.
2017-12-08 09:51:56.14 Server Error 87(The parameter is incorrect.) occurred while opening file '/var/opt/mssql/data/master.mdf' to obtain configuration information at startup. An invalid startup option might have caused the error. Verify your startup options, and correct or remove them if necessary.
^CGracefully stopping... (press Ctrl+C again to force)
Killing nginx ... done
Killing mssql ... done
Killing icons ... done
Killing identity ... done
Killing api ... done
Killing attachments ... done
Killing web ... done

[kspearrin]

Is there anything else running on this same server?

[aNaravi]

Not sure which server you're talking about, but I haven't started any other docker containers that are using localhost.

[kspearrin]

I've referring to the host machine. Are there other services running on this same machine that may be causing things like port conflicts? Though I suppose that shouldn't be an issues since mssql's port is not exposed outside of the docker instance.

[aNaravi]

No, there aren't any services running on my computer other than the applications that I normally use.

2017-12-08 09:51:26.22 Server Error 87(The parameter is incorrect.) occurred while opening file '/var/opt/mssql/data/master.mdf' to obtain configuration information at startup. An invalid startup option might have caused the error. Verify your startup options, and correct or remove them if necessary.

Do you have any idea what "startup options" this could be referring to?

[atomjack]

@aNaravi Are you using zfs, by any chance?

[aNaravi]

@atomjack No I'm not.

[kspearrin]

@aNaravi This is the only information I could find related to the error you are getting: https://github.com/Microsoft/mssql-docker/issues/13 , which seems to be related to unsupported file systems (which @atomjack was alluding to).

[atomjack]

Hmm weird, because I am and was getting the exact same error. Near as I can tell, the master.mdf file was getting corrupted while being copied from my bwdata directory to the docker/mssql container. I ended up having to create zvol's for bwdata and /var/lib/docker and format them ext4 in order to fix it.

I suggest copying that file out of your docker/mssql container and check the md5 on it compared to the one in bwdata/mssql/data.

@kspearrin yes, that's the issue I found that clued me in to what was going on.

[aNaravi]

My entire filesystem is ext4 and I've never used zfs.

@atomjack When you say "copy the file out of your docker/mssql container and check md5" I'm assuming you mean /var/lib/docker/container/*mssql container's code*. But I don't see the the same files as the ones in bwdata/mssql/data - I can't find the master.mdf file in mssql's container.
Output of
ls /var/lib/docker/container/b7952512b63f958dc881b41525b2ca208736aef1fe4b55b1ba957f09c73246f8

b7952512b63f958dc881b41525b2ca208736aef1fe4b55b1ba957f09c73246f8-json.log config.v2.json hostname resolv.conf shm
checkpoints hostconfig.json hosts resolv.conf.hash

[atomjack]

@aNaravi Sorry, you need to run this:

docker cp <mssql container id>:/var/opt/mssql/data/master.mdf . where <mssql container id> is the 12 character hex id of the mssql container (that you get out of docker ps). That copies it out of the container and into your local filesystem (into the current directory, so make sure you're not running it from bwdata/mssql/data).

My guess is that that master.mdf file's md5 should match the one in bwdata. Mine wasn't, and seeing that mssql-docker issue, lead me to believe the file the container was trying to use was corrupted somehow.

The md5 of both of mine is 699f798ddf5b0510b369a5b07a869ee2. So, maybe if the one in your container is different, you could try copying the one from bwdata (if it has the same md5 as mine) back into your container, and see if that fixes it? e.g. docker cp /path/to/bwdata/mssql/data/master.mdf <mssql container id>:/var/opt/mssql/data/master.mdf

[aNaravi]

@atomjack (Un)Fortunately the md5sum of both files matches. Was your mssql container restarting every minute too?

[atomjack]

@aNaravi Yes, my mssql container was restarting every minute, and IIRC I was getting the same error you were getting (The parameter is incorrect, on that master.mdf file). I initially only created a zvol for /var/lib/docker, but I was still getting the problem, and IIRC, I pulled the master.mdf file out of the container and the md5 was still different from that of the file in bwdata/mssql/data. Once I created a zvol for bwdata as well (and updated the bitwarden.sh script to ignore the fact that the directory already exists), it worked fine. I realize you're not using zfs, but I wanted to explain what I had to do to get it working.

You said the md5sum of both files match - do they match the md5 I supplied? Or is the md5 different and they just match each other?

[aNaravi]

They match each other. Both are 8102ffa2ecac53dd397c20feee97b391

[atomjack]

Ok, your master.mdf must be corrupted, then. However, I don't really know enough about mssql to feel comfortable sending you mine (plus, both of mine in bwdata and inside the container, while their md5's match each other, it's different from the one I pasted above).

@kspearrin Can you point @aNaravi into how to get a valid copy of the master.mdf file?

[kspearrin]

Sorry, but I wouldn't know. I would think that these things should be automatically generated by mssql itself when it starts up the first time. You could try posting about this on mssql docker repo as well at https://github.com/Microsoft/mssql-docker and see if they have any insight.

[rufinus]

same problem here, i removed the local path and used an unbind volume:

  mssql:
    image: bitwarden/mssql:1.16.0
    container_name: bitwarden-mssql
    restart: always
    volumes:
#      - ../mssql/data:/var/opt/mssql/data
      - /var/opt/mssql/data

this works without problem. also i changed the run.sh skript, to use "stop" instead of "down" so the volume does not get removed.

[asteinbr]

For me the same errors. Fresh installation of docker toolbox (uses docker-machine on VirtualBox hypervisor) on Windows 10 Fall Creators Update.
RAM is set to 4GB.

# more /var/opt/mssql/log/errorlog
2018-01-04 23:27:16.55 Server      Microsoft SQL Server 2017 (RTM-CU2) (KB4052574) - 14.0.3008.27 (X64)
        Nov 16 2017 10:00:49
        Copyright (C) 2017 Microsoft Corporation
        Express Edition (64-bit) on Linux (Ubuntu 16.04.3 LTS)
2018-01-04 23:27:16.57 Server      UTC adjustment: 0:00
2018-01-04 23:27:16.58 Server      (c) Microsoft Corporation.
2018-01-04 23:27:16.59 Server      All rights reserved.
2018-01-04 23:27:16.59 Server      Server process ID is 4116.
2018-01-04 23:27:16.60 Server      Logging SQL Server messages in file '/var/opt/mssql/log/errorlog'.
2018-01-04 23:27:16.61 Server      Registry startup parameters:
         -d /var/opt/mssql/data/master.mdf
         -l /var/opt/mssql/data/mastlog.ldf
         -e /var/opt/mssql/log/errorlog
2018-01-04 23:27:16.66 Server      Error: 17113, Severity: 16, State: 1.
2018-01-04 23:27:16.66 Server      Error 87(The parameter is incorrect.) occurred while opening file '/var/opt/mssql/dat
a/master.mdf' to obtain configuration information at startup. An invalid startup option might have caused the error. Ver
ify your startup options, and correct or remove them if necessary.
#

[BramVanBerkel]

Hi, I have the exact same problem on my Ubuntu 16.04 server. Are there any updates, or can I help by delivering log outputs?

[kspearrin]

@BramVanBerkel you could use a docker volume instead of mapping the volume to the host machine file system. Edit ./bwdata/docker/docker-compose.yml and set

# Parameter:MssqlDataDockerVolume=true

Then run ./bitwarden.sh update.

[BramVanBerkel]

@kspearrin Worked like a charm! Thanks a ton!
Just out of curiosity, what does this setting do?

[kspearrin]

Your database is now being stored as a docker volume rather than being mapped back to the host machine's file system.

Type docker volume ls and you can see it.

[SiebeVE]

I had the same issue. My bitwarden lived in /etc/bitwarden on a Ubuntu 18.04 server. When moved to /opt/bitwarden the issue was solved.

[kspearrin]

Users can now opt to use docker volumes if volume mapping doesn't work with sql server on the host OS. See database_docker_volume: true in config.yml.