Teslamate: DB Connection lost - TM crashed

[airmaxx]

Describe the bug
It happend to me twice, that Teslamate crashed. I don't know why the database connection is not available. After Teslamate crashed I can restart it without problemens. There is no need for restarting PostgreSQL.

Expected behavior
a) The exceptption gets catched and Teslamate waits until it can resume DB connection
b) Question: How to set "DBConnection.start_link/2" so that TM waits longer

Relevant entries from the logs

16:51:06.789 [error] GenStateMachine TeslaMate.Terrain terminating
** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2131ms. You can configure how long requests wait in the queue using :queue_target and :queue_interval. See DBConnection.start_link/2 for more information
    (ecto_sql) lib/ecto/adapters/sql.ex:629: Ecto.Adapters.SQL.raise_sql_call_error/1
    (ecto_sql) lib/ecto/adapters/sql.ex:562: Ecto.Adapters.SQL.execute/5
    (ecto) lib/ecto/repo/queryable.ex:177: Ecto.Repo.Queryable.execute/4
    (ecto) lib/ecto/repo/queryable.ex:17: Ecto.Repo.Queryable.all/3
    (teslamate) lib/teslamate/log.ex:138: TeslaMate.Log.get_positions_without_elevation/2
    (teslamate) lib/teslamate/terrain.ex:75: TeslaMate.Terrain.handle_event/4
    (stdlib) gen_statem.erl:1161: :gen_statem.loop_state_callback/11
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
16:51:06.789 [error] GenServer TeslaMate.Api terminating
** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2999ms. You can configure how long requests wait in the queue using :queue_target and :queue_interval. See DBConnection.start_link/2 for more information
    (ecto_sql) lib/ecto/adapters/sql.ex:629: Ecto.Adapters.SQL.raise_sql_call_error/1
    (ecto_sql) lib/ecto/adapters/sql.ex:562: Ecto.Adapters.SQL.execute/5
    (ecto) lib/ecto/repo/queryable.ex:177: Ecto.Repo.Queryable.execute/4
    (ecto) lib/ecto/repo/queryable.ex:17: Ecto.Repo.Queryable.all/3
    (teslamate) lib/teslamate/auth.ex:22: TeslaMate.Auth.get_tokens/0
    (teslamate) lib/teslamate/api.ex:179: TeslaMate.Api.handle_continue/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:388: :gen_server.loop/7
Last message: {:continue, :sign_in}
16:51:06.792 [info] Application teslamate exited: TeslaMate.Application.start(:normal, []) returned an error: shutdown: failed to start child: TeslaMate.Vehicles
    ** (EXIT) exited in: GenServer.call(TeslaMate.Api, :list, 35000)
        ** (EXIT) an exception was raised:
            ** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2999ms. You can configure how long requests wait in the queue using :queue_target and :queue_interval. See DBConnection.start_link/2 for more information
                (ecto_sql) lib/ecto/adapters/sql.ex:629: Ecto.Adapters.SQL.raise_sql_call_error/1
                (ecto_sql) lib/ecto/adapters/sql.ex:562: Ecto.Adapters.SQL.execute/5
                (ecto) lib/ecto/repo/queryable.ex:177: Ecto.Repo.Queryable.execute/4
                (ecto) lib/ecto/repo/queryable.ex:17: Ecto.Repo.Queryable.all/3
                (teslamate) lib/teslamate/auth.ex:22: TeslaMate.Auth.get_tokens/0
                (teslamate) lib/teslamate/api.ex:179: TeslaMate.Api.handle_continue/2
                (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
                (stdlib) gen_server.erl:388: :gen_server.loop/7
{"Kernel pid terminated",application_controller,"{application_start_failure,teslamate,{{shutdown,{failed_to_start_child,'Elixir.TeslaMate.Vehicles',{{#{'__exception__' => true,'__struct__' => 'Elixir.DBConnection.ConnectionError',message => <<\"connection not available and request was dropped from queue after 2999ms. You can configure how long requests wait in the queue using :queue_target and :queue_interval. See DBConnection.start_link/2 for more information\">>,severity => error},[{'Elixir.Ecto.Adapters.SQL',raise_sql_call_error,1,[{file,\"lib/ecto/adapters/sql.ex\"},{line,629}]},{'Elixir.Ecto.Adapters.SQL',execute,5,[{file,\"lib/ecto/adapters/sql.ex\"},{line,562}]},{'Elixir.Ecto.Repo.Queryable',execute,4,[{file,\"lib/ecto/repo/queryable.ex\"},{line,177}]},{'Elixir.Ecto.Repo.Queryable',all,3,[{file,\"lib/ecto/repo/queryable.ex\"},{line,17}]},{'Elixir.TeslaMate.Auth',get_tokens,0,[{file,\"lib/teslamate/auth.ex\"},{line,22}]},{'Elixir.TeslaMate.Api',handle_continue,2,[{file,\"lib/teslamate/api.ex\"},{line,179}]},{gen_server,try_dispatch,4,[{file,\"gen_server.erl\"},{line,637}]},{gen_server,loop,7,[{file,\"gen_server.erl\"},{line,388}]}]},{'Elixir.GenServer',call,['Elixir.TeslaMate.Api',list,35000]}}}},{'Elixir.TeslaMate.Application',start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,teslamate,{{shutdown,{failed_to_start_child,'Elixir.TeslaMate.Vehicles',{{#{'__exception__' => true,'__struct__' => 'Elixir.D

Operating environment (please complete the following information):

• OS: Ubuntu
• Version: manually installed without docker, current verson from git

[adriankumpf]

Is more than one car linked to your Tesla account? I'm guessing that the database connection pool is to small for some reason. Could you test if increasing the pool size helps? This is done with the environment variable DATABASE_POOL_SIZE.

[airmaxx]

No only one car.

I'm going to set DATABASE_POOL_SIZE to 10 and see if it helps.

[lowwww]

Running into the same problem (also with the increased database pool size).
Running the docker on my synology.

[adriankumpf]

@lowwww could you please add the log output, too?

[lowwww]

@adriankumpf I've attached the logs of all dockers.

logs.zip

[adriankumpf]

@airmaxx Is there any news on this? Can we close this?

[adriankumpf]

@lowwww does this issue still exist with v1.14? If not, does starting the docker services sequentially work (https://github.com/adriankumpf/teslamate/issues/335#issuecomment-570339216)?

[airmaxx]

@airmaxx Is there any news on this? Can we close this?

For me the issue did not occur again after increasing database pool size and updating to 1.14

For me this issue can be closed. Thanks for the excellent work.