Ecto: Umbrella app mix test migration alias error

[hunterboerner]

Environment

• Elixir version (elixir -v):

Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.3.1

• Database and version: PostgreSQL 9.4.4
• Ecto version (mix deps):

* ecto 2.0.2 (Hex package) (mix)
  locked at 2.0.2 (ecto) b02331c1

• Database adapter and version (mix deps):

* postgrex 0.11.2 (Hex package) (mix)
  locked at 0.11.2 (postgrex) 139755c1

• Operating system: macOS El Capitan
  

  Current behavior

I'm having trouble using this in an umbrella app. I have app A which defines schemas and has migrations and I have app B that depends on app A (in_umbrella: true). If I add the alias to mix.exs for both of those apps and then run the tests separately, everything works. However, when I run mix test in the umbrella app, it fails.

Randomized with seed 54646
==> AppA
....................................................

Finished in 0.5 seconds
52 tests, 0 failures

Randomized with seed 188921
** (DBConnection.OwnershipError) cannot find ownership process for #PID<0.46.0>.

When using ownership, you must manage connections in one
of the three ways:

  * By explicitly checking out a connection
  * By explicitly allowing a spawned process
  * By running the pool in shared mode

The first two options require every new process to explicitly
check a connection out or be allowed by calling checkout or
allow respectively.

The third option requires a {:shared, pid} mode to be set.
If using shared mode in tests, make sure your tests are not
async.

If you are reading this error, it means you have not done one
of the steps above or that the owner process has crashed.

See Ecto.Adapters.SQL.Sandbox docs for more information.
    (db_connection) lib/db_connection.ex:722: DBConnection.checkout/2
    (db_connection) lib/db_connection.ex:623: DBConnection.run/3
    (db_connection) lib/db_connection.ex:845: DBConnection.run_meter/3
    (db_connection) lib/db_connection.ex:464: DBConnection.prepare_execute/4
    (ecto) lib/ecto/adapters/postgres/connection.ex:82: Ecto.Adapters.Postgres.Connection.execute/4
    (ecto) lib/ecto/adapters/sql.ex:228: Ecto.Adapters.SQL.sql_call/6
    (ecto) lib/ecto/adapters/sql.ex:178: Ecto.Adapters.SQL.query!/5
    (ecto) lib/ecto/adapters/postgres.ex:69: Ecto.Adapters.Postgres.execute_ddl/3
    (ecto) lib/ecto/migrator.ex:43: Ecto.Migrator.migrated_versions/2
    (ecto) lib/ecto/migrator.ex:142: Ecto.Migrator.run/4
    (ecto) lib/mix/tasks/ecto.migrate.ex:74: anonymous fn/4 in Mix.Tasks.Ecto.Migrate.run/2
    (elixir) lib/enum.ex:651: Enum."-each/2-lists^foreach/1-0-"/2
    (elixir) lib/enum.ex:651: Enum.each/2
    (mix) lib/mix/task.ex:296: Mix.Task.run_task/3
    (mix) lib/mix/task.ex:328: Mix.Task.run_alias/3
    (mix) lib/mix/task.ex:261: Mix.Task.run/2
    (mix) lib/mix/project.ex:211: Mix.Project.in_project/4
    (elixir) lib/file.ex:1145: File.cd!/2
    (mix) lib/mix/task.ex:387: anonymous fn/4 in Mix.Task.recur/1
    (elixir) lib/enum.ex:1623: Enum."-reduce/3-lists^foldl/2-0-"/3

The problem appears to be in the ecto.migrate section of app B's alias. If I remove that, it works again, however that means that app B won't automatically setup the DB. I've noticed that the pid in this line never seems to change:

cannot find ownership process for #PID<0.46.0>.

Minimal app that reproduces the error: https://github.com/hunterboerner/ecto_sandbox_bug.

Expected behavior

It should not crash.

^{Created new issue as per @josevalim's instructions https://github.com/elixir-ecto/ecto/issues/1288#issuecomment-230359309}

[michalmuskala]

Adding the test alias to the root mix.exs fixes the issue:

defmodule EctoSandboxBug.Mixfile do
  use Mix.Project

  def project do
    [apps_path: "apps",
     build_embedded: Mix.env == :prod,
     start_permanent: Mix.env == :prod,
     deps: deps(),
     aliases: aliases()]
  end

  defp deps do
    []
  end

  defp aliases do
    ["test": ["ecto.create", "ecto.migrate", "test"]]
  end
end

This is not the root cause, though.

[hunterboerner]

@michalmuskala will that apply when running mix test in the sub apps?

EDIT: Also, is that in addition to the aliases in the sub apps?

[michalmuskala]

You'd have that alias in the main, top-level mix.exs, as well as in each sub app. I'm not sure where this issue stems from, though.

[josevalim]

The problem is that when you run tests in the first app, it puts the repository into ownership mode. Then when you try to use the second application it will try to run migrations in ownership mode and complain the process was not authorized.

Putting the aliases at the umbrella root solves it because you run migrations for all projects upfront. Because this is happening from inside the migrator, I will see if there is a best way to solve this, but using the aliases in the umbrella is a good way to go for now.

[josevalim]

Fixed, thank you for the reproducing app @hunterboerner , really helpful!

[coryodaniel]

I see that this was merged, but I'm having the same issue. My case is a little different as I am doing integration tests w/ Phoenix.

Erlang/OTP 19 [erts-8.0.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.3.2

* ecto 2.0.5 (Hex package) (mix)
  locked at 2.0.5 (ecto) 7f4c79ac
* phoenix_ecto 3.0.1 (Hex package) (mix)
  locked at 3.0.1 (phoenix_ecto) 42eb486e
  ok
* phoenix 1.2.1 (Hex package) (mix)
  locked at 1.2.1 (phoenix) 6dc59224
  ok

What is currently happening is in my test I am creating a record in postgres, then during the HTTP request, when the controller attempts to select it raises with the message:

Server: localhost:4001 (http)
Request: GET /images/aa426257-ea07-4c60-9ab3-02dfa5f013aa
** (exit) an exception was raised:
    ** (DBConnection.OwnershipError) cannot find ownership process for #PID<0.600.0>.

When using ownership, you must manage connections in one
of the three ways:

  * By explicitly checking out a connection
  * By explicitly allowing a spawned process
  * By running the pool in shared mode

[coryodaniel]

A side note, may be of help. If I set my tests to async: false, everything works as expected. async: true raises this error. I found that async: true DOES work though if its not an umbrella app, and is instead just a vanilla phoenix app

[coryodaniel]

Here is an example of the 'acme_bank' umbrella app failing if a request going over HTTP to the test server is async:

https://github.com/coryodaniel/acme_bank/tree/sbox_bug_async_integration_test

You can flip async between false and true and see the issue.

https://github.com/coryodaniel/acme_bank/blob/sbox_bug_async_integration_test/apps/bank_web/test/controllers/account_controller_test.exs#L2-L5

Thanks for any in sight! I'm currently just running my tests as async: false to get around this!

[josevalim]

I cannot reproduce the error. Adding :async true and running mix test works fine.

[coryodaniel]

From the app or the umbrella app? Is there any other system information that may be helpful?

[josevalim]

I tried both and nothing. :(

[coryodaniel]

Is there anything else version-wise you think could cause it? Would a dump of deps.tree help?

[josevalim]

The repository has a lock file, so we are getting the same dependencies. The only difference is that I am running on Elixir master and you are on 1.3.2. You may try 1.3.3 or master and see if the problem is solved.

[sntran]

I forked @hunterboerner 's repo and updated it with codes relevant more to Ecto.

https://github.com/sntran/ecto_sandbox_bug

When I run mix test under the umbrella folder, I got the same error as his.

I am running on Elixir 1.4. The difference I had is to have app B calling a function of app A, in which it fetches the database.

[foucist]

Just a note to others with a similar error.

If you make an umbrella app and use mix phx.new.web --no-ecto the generated test/support/conn_case.ex will be missing the :ok = Ecto.Adapters.SQL.Sandbox.checkout(AppA.Repo) under the setup block.