Server: bitwarden-identity keeps restarting

[wezmag]

Hi,
I have been running self hosted bitwarden on Synology NAS for a few months without any issues.
However, I updated bitwarden as usual tonight and something went wrong.
Identity server keeps restarting.
When I ran docker logs [container id], I got this message.

Unhandled Exception: System.ArgumentNullException: Value cannot be null.
Parameter name: certificate
   at Microsoft.Extensions.DependencyInjection.IdentityServerBuilderExtensionsCrypto.AddSigningCredential(IIdentityServerBuilder builder, X509Certificate2 certificate)
   at Bit.Core.Utilities.ServiceCollectionExtensions.AddCustomIdentityServerServices(IServiceCollection services, IHostingEnvironment env, GlobalSettings globalSettings) in /home/appveyor/projects/server/src/Core/Utilities/ServiceCollectionExtensions.cs:line 340
   at Bit.Identity.Startup.ConfigureServices(IServiceCollection services) in /home/appveyor/projects/server/src/Identity/Startup.cs:line 61
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Hosting.Conve ntionBasedStartup.ConfigureServices(IServiceCollection services)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.EnsureApplicationServices()
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.Initialize()
   at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build()
   at Bit.Identity.Program.Main(String[] args) in /home/appveyor/projects/server/src/Identity/Program.cs:line 13

Did a little bit of research, looks like there is something wrong with identity.pfx file.
I can find the file located in bwdata/identity folder, and globalSettings__identityServer__certificatePassword in global.override.env is also not missing.
What can I do now to fix the issue?
Please kindly help.
Thank you very much.

BTW, I am currently running 1.31.1. I update bitwarden every Friday night.

[kspearrin]

Weird. Can you can exec bash into the identity container and see if the mapped volume for the bwdata/identity directory is available? It should be at /etc/bitwarden/identity.

[wezmag]

@kspearrin
I was able to exec bash into the identity container and checked the mapped volume right before the container restarts. The /etc/bitwarden/identity is there and identity.pfx also exists.

root@Apollo:/volume1/docker# docker exec -it 28c0 bash
root@28c02e7dd8ce:/app# cd /etc/bitwarden/identity
root@28c02e7dd8ce:/etc/bitwarden/identity# ls
identity.pfx

However, I don't think I can stay in the container any longer. It kicks me out while restart.

Anyway, I got another issue today. The bitwarden-mssql container also keeps restarting. The error message is as below:

The SQL Server End-User License Agreement (EULA) must be accepted before SQL
Server can start. The license terms for this product can be downloaded from
http://go.microsoft.com/fwlink/?LinkId=746388.

You can accept the EULA by specifying the --accept-eula command line option,
setting the ACCEPT_EULA environment variable, or using the mssql-conf tool.

I checked the mssql.override.env file. The content is as below

ACCEPT_EULA=Y
MSSQL_PID=Express
SA_PASSWORD=my_password

Would it be possible that global.override.env and mssql.override.env are ignored?

[kspearrin]

Sounds like your env files may not be getting mapped for whatever reason. What is the contents of your ./bwdata/docker/docker-compose.yml ?

[wezmag]

Here is my docker-compose.yml file.

#
# Useful references:
# https://docs.docker.com/compose/compose-file/
# https://docs.docker.com/compose/reference/overview/#use--f-to-specify-name-and-path-of-one-or-more-compose-files
# https://docs.docker.com/compose/reference/envvars/
#
#########################################################################
# WARNING: This file is generated. Do not make changes to this file.    #
# They will be overwritten on update. If you want to make additions to  #
# this file, you can create a `docker-compose.override.yml` file in the #
# same directory and it will be merged into this file at runtime. You   #
# can also manage various settings used in this file from the           #
# ./bwdata/config.yml file for your installation.                       #
#########################################################################

version: '3'

services:
  mssql:
    image: bitwarden/mssql:1.31.1
    container_name: bitwarden-mssql
    restart: always
    volumes:
      - ../mssql/data:/var/opt/mssql/data
      - ../logs/mssql:/var/opt/mssql/log
      - ../mssql/backups:/etc/bitwarden/mssql/backups
    env_file:
      - mssql.env
      - ../env/uid.env
      - ../env/mssql.override.env

  web:
    image: bitwarden/web:2.11.0
    container_name: bitwarden-web
    restart: always
    volumes:
      - ../web:/etc/bitwarden/web
    env_file:
      - global.env
      - ../env/uid.env

  attachments:
    image: bitwarden/attachments:1.31.1
    container_name: bitwarden-attachments
    restart: always
    volumes:
      - ../core/attachments:/etc/bitwarden/core/attachments
    env_file:
      - global.env
      - ../env/uid.env

  api:
    image: bitwarden/api:1.31.1
    container_name: bitwarden-api
    restart: always
    volumes:
      - ../core:/etc/bitwarden/core
      - ../ca-certificates:/etc/bitwarden/ca-certificates
      - ../logs/api:/etc/bitwarden/logs
    env_file:
      - global.env
      - ../env/uid.env
      - ../env/global.override.env

  identity:
    image: bitwarden/identity:1.31.1
    container_name: bitwarden-identity
    restart: always
    volumes:
      - ../identity:/etc/bitwarden/identity
      - ../core:/etc/bitwarden/core
      - ../ca-certificates:/etc/bitwarden/ca-certificates
      - ../logs/identity:/etc/bitwarden/logs
    env_file:
      - global.env
      - ../env/uid.env
      - ../env/global.override.env

  admin:
    image: bitwarden/admin:1.31.1
    container_name: bitwarden-admin
    restart: always
    depends_on:
      - mssql
    volumes:
      - ../core:/etc/bitwarden/core
      - ../ca-certificates:/etc/bitwarden/ca-certificates
      - ../logs/admin:/etc/bitwarden/logs
    env_file:
      - global.env
      - ../env/uid.env
      - ../env/global.override.env

  icons:
    image: bitwarden/icons:1.31.1
    container_name: bitwarden-icons
    restart: always
    volumes:
      - ../ca-certificates:/etc/bitwarden/ca-certificates
      - ../logs/icons:/etc/bitwarden/logs
    env_file:
      - global.env
      - ../env/uid.env

  notifications:
    image: bitwarden/notifications:1.31.1
    container_name: bitwarden-notifications
    restart: always
    volumes:
      - ../ca-certificates:/etc/bitwarden/ca-certificates
      - ../logs/notifications:/etc/bitwarden/logs
    env_file:
      - global.env
      - ../env/uid.env
      - ../env/global.override.env

  events:
    image: bitwarden/events:1.31.1
    container_name: bitwarden-events
    restart: always
    volumes:
      - ../ca-certificates:/etc/bitwarden/ca-certificates
      - ../logs/events:/etc/bitwarden/logs
    env_file:
      - global.env
      - ../env/uid.env
      - ../env/global.override.env

  nginx:
    image: bitwarden/nginx:1.31.1
    container_name: bitwarden-nginx
    restart: always
    depends_on:
      - web
      - admin
      - api
      - identity
    ports:
      - '8123:8080'
    volumes:
      - ../nginx:/etc/bitwarden/nginx
      - ../letsencrypt:/etc/letsencrypt
      - ../ssl:/etc/ssl
      - ../logs/nginx:/var/log/nginx
    env_file:
      - ../env/uid.env

[wezmag]

And the file permissions...

root@Apollo:/volume1/docker/bwdata/env# ls -l
total 12
-rw-------  1 65534 65534 1851 Aug 27 20:39 global.override.env
-rw-------  1 65534 65534   77 Aug 27 21:05 mssql.override.env
-rwxr-xr-x+ 1 65534 65534   32 Aug 27 19:35 uid.env

[Gitsko]

I have exactly the same problem, the same error message, but only for the identity docker container

[djvodka]

I am setting Bitwarden up for the first time with the docker and have few bugs as well.
File permissions is the same as wezmag and docker-compose.yml also.

I fixed the error "SQL EULA" by adding "ACCEPT_EULA = Y" under environment in bitward-mssql (docker container)

but when I run "./bitwarden.sh updatedb" then I get SA login error.
I solved that by finding my sa password from "global.override .env" file and adding it in the same way as "ACCEPT_EULA" just with "SA_PASSWORD".

And now I'm stuck with "bitwarden-identity" - Unhandled Exception: System.ArgumentNullException: Value cannot be null.

[jogger67]

I've got the same issue since I've updated the Synology DS918+ to DSM 6.2.2-24922-3, updated Docker to version 18.09.6, build 8cdf373 and finally I've run ./bitwarden update and now the bitwarden-identity container keeps restarting and I can't login anymore to my bitwarden website.

[williambherman]

It seems likely that everyone here and also in #557 are running bitwarden on Synology. It seems to be an issue with the recent Synology Docker package update having a bug which means environment variables don't get passed to the container https://community.synology.com/enu/forum/15/post/128146. It seems the only workaround is to revert back to the older Docker package.

[wezmag]

@williambherman
Thanks for the information.
I just rolled back to the previous version of Docker package (17.05.0-0401). Everything works as expected.
For those who need Docker package 17.05.0-0401, you can download it here.
https://archive.synology.com/download/Package/spk/Docker/17.05.0-0401/