Ecto: Running mix test fails without an existing database

Created on 2 Mar 2016 · 10Comments · Source: elixir-ecto/ecto

Ecto 2.0.0-beta.1
Postgrex 0.11.1

After migrating to 2.0.0-beta.1 all CI builds started to fail during mix test, with the error that the database didn't exist. Running mix test a second time or creating the db beforehand always works. It is easy enough to work around, but is confusing and provides a poor first time experience. The explicit sequencing of create/migrate/start sandbox mode is common and is what Phoenix generates by default.

This can be reproduced in any existing app by dropping the test db and running mix test.

I'd love to fix this, but I'm not sure where the race condition is coming in.

Source

sorentwo

Most helpful comment

Yes, the fix is to remove those from test_helper and do it on your aliases in your mix.exs:

"test": ["ecto.create --quiet", "ecto.migrate", "test"]

Otherwise we will start the app without a database and it will fail. :) Thank you for the report, we will tackle this when we migrate Phoenix!

josevalim on 3 Mar 2016

👍8

All 10 comments

Can you paste precisely the error here and what it is in your test helper file? Because this has been supposedly fixed.

josevalim on 2 Mar 2016

Here is the end of test_helper.exs:

ExUnit.start

Mix.Task.run "ecto.create", ~w(-r Px.Repo --quiet)
Mix.Task.run "ecto.migrate", ~w(-r Px.Repo --quiet)
Ecto.Adapters.SQL.Sandbox.mode(Px.Repo, :manual)

Here's the stack trace (with individual identical errors for each pid removed) :

14:29:42.594 [error] Postgrex.Protocol (#PID<0.978.0>) failed to connect: ** (Postgrex.Error) FATAL (invalid_catalog_name): database "px_test" does not exist
** (DBConnection.Error) connection not available
    (db_connection) lib/db_connection.ex:817: DBConnection.checkout/2
    (db_connection) lib/db_connection.ex:717: DBConnection.run/3
    (db_connection) lib/db_connection.ex:957: DBConnection.run_meter/3
    (db_connection) lib/db_connection.ex:421: DBConnection.query/4
    (ecto) lib/ecto/adapters/sql.ex:178: Ecto.Adapters.SQL.query!/5
    (ecto) lib/ecto/adapters/postgres.ex:56: anonymous fn/4 in Ecto.Adapters.Postgres.execute_ddl/3
    (elixir) lib/enum.ex:1473: Enum."-reduce/3-lists^foldl/2-0-"/3
    (ecto) lib/ecto/adapters/postgres.ex:56: Ecto.Adapters.Postgres.execute_ddl/3

14:29:42.676 [error] GenServer #PID<0.1007.0> terminating
** (DBConnection.Error) connection not available
    (db_connection) lib/db_connection.ex:817: DBConnection.checkout/2
    (db_connection) lib/db_connection.ex:717: DBConnection.run/3
    (db_connection) lib/db_connection.ex:957: DBConnection.run_meter/3
    (db_connection) lib/db_connection.ex:421: DBConnection.query/4
    (ecto) lib/ecto/adapters/sql.ex:178: Ecto.Adapters.SQL.query!/5
    (ecto) lib/ecto/adapters/postgres.ex:56: anonymous fn/4 in Ecto.Adapters.Postgres.execute_ddl/3
    (elixir) lib/enum.ex:1473: Enum."-reduce/3-lists^foldl/2-0-"/3
    (ecto) lib/ecto/adapters/postgres.ex:56: Ecto.Adapters.Postgres.execute_ddl/3

sorentwo on 2 Mar 2016

Is it an umbrella project by any chance? do you also have a github repo that reproduces the error?

josevalim on 2 Mar 2016

Nope, it isn't an umbrella project. I only have a private repo, so I'll create a new one to recreate it.

sorentwo on 2 Mar 2016

@josevalim https://github.com/sorentwo/mix-test-failure demonstrates the failure. After cloning and installing deps simply run mix test, it fails with the same message as above. Running it a second time will work.

Aside from updating the generated phoenix-ecto code for Ecto 2.0 all I've done is generate a model with a migration.

sorentwo on 2 Mar 2016

Yes, the fix is to remove those from test_helper and do it on your aliases in your mix.exs:

"test": ["ecto.create --quiet", "ecto.migrate", "test"]

Otherwise we will start the app without a database and it will fail. :) Thank you for the report, we will tackle this when we migrate Phoenix!

josevalim on 3 Mar 2016

👍8

Indeed it is, thanks!

sorentwo on 3 Mar 2016

what about umbrella apps ?

is correct to setup the aliases into the umbrella mix.exs?
seems is the only way to make it work, if I setup aliases into each app it won't work,
it bails out with error "(Mix) No repository available (project MyProject.Mixfile has no :app configured). Please pass a repo with the -r option."

xadhoom on 5 Apr 2016

I'm having trouble using this in an umbrella app. I have app A which defines schemas and has migrations and I have app B that depends on app A (in_umbrella: true). If I add the alias to mix.exs for both of those apps and then run the tests separately, everything works. However, when I run mix test in the umbrella app, it fails.

Randomized with seed 54646
==> AppA
....................................................

Finished in 0.5 seconds
52 tests, 0 failures

Randomized with seed 188921
** (DBConnection.OwnershipError) cannot find ownership process for #PID<0.46.0>.

When using ownership, you must manage connections in one
of the three ways:

  * By explicitly checking out a connection
  * By explicitly allowing a spawned process
  * By running the pool in shared mode

The first two options require every new process to explicitly
check a connection out or be allowed by calling checkout or
allow respectively.

The third option requires a {:shared, pid} mode to be set.
If using shared mode in tests, make sure your tests are not
async.

If you are reading this error, it means you have not done one
of the steps above or that the owner process has crashed.

See Ecto.Adapters.SQL.Sandbox docs for more information.
    (db_connection) lib/db_connection.ex:722: DBConnection.checkout/2
    (db_connection) lib/db_connection.ex:623: DBConnection.run/3
    (db_connection) lib/db_connection.ex:845: DBConnection.run_meter/3
    (db_connection) lib/db_connection.ex:464: DBConnection.prepare_execute/4
    (ecto) lib/ecto/adapters/postgres/connection.ex:82: Ecto.Adapters.Postgres.Connection.execute/4
    (ecto) lib/ecto/adapters/sql.ex:228: Ecto.Adapters.SQL.sql_call/6
    (ecto) lib/ecto/adapters/sql.ex:178: Ecto.Adapters.SQL.query!/5
    (ecto) lib/ecto/adapters/postgres.ex:69: Ecto.Adapters.Postgres.execute_ddl/3
    (ecto) lib/ecto/migrator.ex:43: Ecto.Migrator.migrated_versions/2
    (ecto) lib/ecto/migrator.ex:142: Ecto.Migrator.run/4
    (ecto) lib/mix/tasks/ecto.migrate.ex:74: anonymous fn/4 in Mix.Tasks.Ecto.Migrate.run/2
    (elixir) lib/enum.ex:651: Enum."-each/2-lists^foreach/1-0-"/2
    (elixir) lib/enum.ex:651: Enum.each/2
    (mix) lib/mix/task.ex:296: Mix.Task.run_task/3
    (mix) lib/mix/task.ex:328: Mix.Task.run_alias/3
    (mix) lib/mix/task.ex:261: Mix.Task.run/2
    (mix) lib/mix/project.ex:211: Mix.Project.in_project/4
    (elixir) lib/file.ex:1145: File.cd!/2
    (mix) lib/mix/task.ex:387: anonymous fn/4 in Mix.Task.recur/1
    (elixir) lib/enum.ex:1623: Enum."-reduce/3-lists^foldl/2-0-"/3

The problem appears to be in the ecto.migrate section of app B's alias. If I remove that, it works again, however that means that app B won't automatically setup the DB.