Ecto: "(RuntimeError) cannot find ownership process" error for channel tests in Phoenix (phoenix_ecto: 3.0.0-beta.2)

[pdawczak]

Environment

• Elixir version (elixir -v): Elixir 1.2.1
• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): psql (PostgreSQL) 9.4.4
• Ecto version (mix deps):

➜  fire_slack git:(pd-channels) ✗ mix deps
* connection 1.0.2 (Hex package) (mix)
  locked at 1.0.2 (connection)
  ok
* fs 0.9.1 (Hex package) (rebar)
  locked at 0.9.2 (fs)
  ok
* gettext 0.10.0 (Hex package) (mix)
  locked at 0.10.0 (gettext)
  ok
* ranch 1.2.1 (Hex package) (rebar)
  locked at 1.2.1 (ranch)
  ok
* poolboy 1.5.1 (Hex package) (rebar)
  locked at 1.5.1 (poolboy)
  ok
* decimal 1.1.1 (Hex package) (mix)
  locked at 1.1.1 (decimal)
  ok
* poison 2.1.0 (Hex package) (mix)
  locked at 2.1.0 (poison)
  ok
* db_connection 0.2.4 (Hex package) (mix)
  locked at 0.2.4 (db_connection)
  ok
* cowlib 1.0.2 (Hex package) (rebar)
  locked at 1.0.2 (cowlib)
  ok
* cowboy 1.0.4 (Hex package) (rebar)
  locked at 1.0.4 (cowboy)
  ok
* plug 1.1.2 (Hex package) (mix)
  locked at 1.1.2 (plug)
  ok
* phoenix_html 2.5.1 (Hex package) (mix)
  locked at 2.5.1 (phoenix_html)
  ok
* phoenix 1.1.4 (Hex package) (mix)
  locked at 1.1.4 (phoenix)
  ok
* phoenix_live_reload 1.0.3 (Hex package) (mix)
  locked at 1.0.3 (phoenix_live_reload)
  ok
* postgrex 0.11.1 (Hex package) (mix)
  locked at 0.11.1 (postgrex)
  ok
* ecto 2.0.0-beta.2 (Hex package) (mix)
  locked at 2.0.0-beta.2 (ecto)
  ok
* phoenix_ecto 3.0.0-beta.2 (Hex package) (mix)
  locked at 3.0.0-beta.2 (phoenix_ecto)
  ok

• Operating system: MacOS X 10.10.5
  

  Description

I've taken the steps from section Using Ecto 2.0 in Phoenix projects to try out the new version of Ecto in Phoenix application.

When running the tests they fail for _channels_ (please note, the tests were automatically generated for mix phoenix.gen.channel):

➜  fire_slack git:(pd-channels) ✗ mix test
....

  1) test shout broadcasts to channel:lobby (FS.ChannelChannelTest)
     test/channels/channel_channel_test.exs:19
     ** (EXIT from #PID<0.294.0>) an exception was raised:
         ** (RuntimeError) cannot find ownership process for #PID<0.296.0>.

     When using ownership, you must manage connections in one
     of the three ways:

       * By explicitly checking out a connection
       * By explicitly allowing a spawned process
       * By running the pool in shared mode

     The first two options require every new process to explicitly
     check a connection out or be allowed.

     If you are reading this error, it means you have not done one
     of the steps above or that the owner process has crashed.

             (db_connection) lib/db_connection.ex:817: DBConnection.checkout/2
             (db_connection) lib/db_connection.ex:717: DBConnection.run/3
             (db_connection) lib/db_connection.ex:535: DBConnection.prepare_execute/4
             (ecto) lib/ecto/adapters/sql.ex:369: Ecto.Adapters.SQL.execute_and_cache/7
             (ecto) lib/ecto/repo/queryable.ex:127: Ecto.Repo.Queryable.execute/5
             (ecto) lib/ecto/repo/queryable.ex:40: Ecto.Repo.Queryable.all/4
             (fire_slack) web/channels/channel_channel.ex:35: FS.ChannelChannel.handle_info/2
             (phoenix) lib/phoenix/channel/server.ex:242: Phoenix.Channel.Server.handle_info/2
             (stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
             (stdlib) gen_server.erl:681: :gen_server.handle_msg/5
             (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3



  2) test ping replies with status ok (FS.ChannelChannelTest)
     test/channels/channel_channel_test.exs:14
     ** (EXIT from #PID<0.297.0>) an exception was raised:
         ** (RuntimeError) cannot find ownership process for #PID<0.299.0>.

     When using ownership, you must manage connections in one
     of the three ways:

       * By explicitly checking out a connection
       * By explicitly allowing a spawned process
       * By running the pool in shared mode

     The first two options require every new process to explicitly
     check a connection out or be allowed.

     If you are reading this error, it means you have not done one
     of the steps above or that the owner process has crashed.

             (db_connection) lib/db_connection.ex:817: DBConnection.checkout/2
             (db_connection) lib/db_connection.ex:717: DBConnection.run/3
             (db_connection) lib/db_connection.ex:535: DBConnection.prepare_execute/4
             (ecto) lib/ecto/adapters/sql.ex:369: Ecto.Adapters.SQL.execute_and_cache/7
             (ecto) lib/ecto/repo/queryable.ex:127: Ecto.Repo.Queryable.execute/5
             (ecto) lib/ecto/repo/queryable.ex:40: Ecto.Repo.Queryable.all/4
             (fire_slack) web/channels/channel_channel.ex:35: FS.ChannelChannel.handle_info/2
             (phoenix) lib/phoenix/channel/server.ex:242: Phoenix.Channel.Server.handle_info/2
             (stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
             (stdlib) gen_server.erl:681: :gen_server.handle_msg/5
             (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3



  3) test broadcasts are pushed to the client (FS.ChannelChannelTest)
     test/channels/channel_channel_test.exs:24
     ** (EXIT from #PID<0.300.0>) an exception was raised:
         ** (RuntimeError) cannot find ownership process for #PID<0.302.0>.

     When using ownership, you must manage connections in one
     of the three ways:

       * By explicitly checking out a connection
       * By explicitly allowing a spawned process
       * By running the pool in shared mode

     The first two options require every new process to explicitly
     check a connection out or be allowed.

     If you are reading this error, it means you have not done one
     of the steps above or that the owner process has crashed.

             (db_connection) lib/db_connection.ex:817: DBConnection.checkout/2
             (db_connection) lib/db_connection.ex:717: DBConnection.run/3
             (db_connection) lib/db_connection.ex:535: DBConnection.prepare_execute/4
             (ecto) lib/ecto/adapters/sql.ex:369: Ecto.Adapters.SQL.execute_and_cache/7
             (ecto) lib/ecto/repo/queryable.ex:127: Ecto.Repo.Queryable.execute/5
             (ecto) lib/ecto/repo/queryable.ex:40: Ecto.Repo.Queryable.all/4
             (fire_slack) web/channels/channel_channel.ex:35: FS.ChannelChannel.handle_info/2
             (phoenix) lib/phoenix/channel/server.ex:242: Phoenix.Channel.Server.handle_info/2
             (stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
             (stdlib) gen_server.erl:681: :gen_server.handle_msg/5
             (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

..16:49:47.112 [error] GenServer #PID<0.296.0> terminating
** (RuntimeError) cannot find ownership process for #PID<0.296.0>.

When using ownership, you must manage connections in one
of the three ways:

  * By explicitly checking out a connection
  * By explicitly allowing a spawned process
  * By running the pool in shared mode

The first two options require every new process to explicitly
check a connection out or be allowed.

If you are reading this error, it means you have not done one
of the steps above or that the owner process has crashed.

    (db_connection) lib/db_connection.ex:817: DBConnection.checkout/2
    (db_connection) lib/db_connection.ex:717: DBConnection.run/3
    (db_connection) lib/db_connection.ex:535: DBConnection.prepare_execute/4
    (ecto) lib/ecto/adapters/sql.ex:369: Ecto.Adapters.SQL.execute_and_cache/7
    (ecto) lib/ecto/repo/queryable.ex:127: Ecto.Repo.Queryable.execute/5
    (ecto) lib/ecto/repo/queryable.ex:40: Ecto.Repo.Queryable.all/4
    (fire_slack) web/channels/channel_channel.ex:35: FS.ChannelChannel.handle_info/2
    (phoenix) lib/phoenix/channel/server.ex:242: Phoenix.Channel.Server.handle_info/2
    (stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:681: :gen_server.handle_msg/5
    (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
16:49:47.113 [error] GenServer #PID<0.299.0> terminating
** (RuntimeError) cannot find ownership process for #PID<0.299.0>.

When using ownership, you must manage connections in one
of the three ways:

  * By explicitly checking out a connection
  * By explicitly allowing a spawned process
  * By running the pool in shared mode

The first two options require every new process to explicitly
check a connection out or be allowed.

If you are reading this error, it means you have not done one
of the steps above or that the owner process has crashed.

    (db_connection) lib/db_connection.ex:817: DBConnection.checkout/2
    (db_connection) lib/db_connection.ex:717: DBConnection.run/3
    (db_connection) lib/db_connection.ex:535: DBConnection.prepare_execute/4
    (ecto) lib/ecto/adapters/sql.ex:369: Ecto.Adapters.SQL.execute_and_cache/7
    (ecto) lib/ecto/repo/queryable.ex:127: Ecto.Repo.Queryable.execute/5
    (ecto) lib/ecto/repo/queryable.ex:40: Ecto.Repo.Queryable.all/4
    (fire_slack) web/channels/channel_channel.ex:35: FS.ChannelChannel.handle_info/2
    (phoenix) lib/phoenix/channel/server.ex:242: Phoenix.Channel.Server.handle_info/2
    (stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:681: :gen_server.handle_msg/5
    (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
16:49:47.113 [error] GenServer #PID<0.302.0> terminating
** (RuntimeError) cannot find ownership process for #PID<0.302.0>.

When using ownership, you must manage connections in one
of the three ways:

  * By explicitly checking out a connection
  * By explicitly allowing a spawned process
  * By running the pool in shared mode

The first two options require every new process to explicitly
check a connection out or be allowed.

If you are reading this error, it means you have not done one
of the steps above or that the owner process has crashed.

    (db_connection) lib/db_connection.ex:817: DBConnection.checkout/2
    (db_connection) lib/db_connection.ex:717: DBConnection.run/3
    (db_connection) lib/db_connection.ex:535: DBConnection.prepare_execute/4
    (ecto) lib/ecto/adapters/sql.ex:369: Ecto.Adapters.SQL.execute_and_cache/7
    (ecto) lib/ecto/repo/queryable.ex:127: Ecto.Repo.Queryable.execute/5
    (ecto) lib/ecto/repo/queryable.ex:40: Ecto.Repo.Queryable.all/4
    (fire_slack) web/channels/channel_channel.ex:35: FS.ChannelChannel.handle_info/2
    (phoenix) lib/phoenix/channel/server.ex:242: Phoenix.Channel.Server.handle_info/2
    (stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:681: :gen_server.handle_msg/5
    (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
........

Finished in 0.4 seconds (0.3s on load, 0.1s on tests)
17 tests, 3 failures

Randomized with seed 46676

Files changed after updating Ecto:

test/test_helper.exs

ExUnit.start

Mix.Task.run "ecto.create", ~w(-r FS.Repo --quiet)
Mix.Task.run "ecto.migrate", ~w(-r FS.Repo --quiet)
Ecto.Adapters.SQL.Sandbox.mode(FS.Repo, :manual)

test/support/channel_case.exs

defmodule FS.ChannelCase do
  @moduledoc """
  This module defines the test case to be used by
  channel tests.

  Such tests rely on `Phoenix.ChannelTest` and also
  imports other functionality to make it easier
  to build and query models.

  Finally, if the test case interacts with the database,
  it cannot be async. For this reason, every test runs
  inside a transaction which is reset at the beginning
  of the test unless the test case is marked as async.
  """

  use ExUnit.CaseTemplate

  using do
    quote do
      # Import conveniences for testing with channels
      use Phoenix.ChannelTest

      alias FS.Repo
      import Ecto
      import Ecto.Changeset
      import Ecto.Query, only: [from: 1, from: 2]


      # The default endpoint for testing
      @endpoint FS.Endpoint
    end
  end

  setup tags do
    :ok = Ecto.Adapters.SQL.Sandbox.checkout(FS.Repo)
  end
end

All the rest of the tests (controllers/models) seem to be fine. Please, let me know if I can provide any more information.

Thanks!

[josevalim]

You need to explicitly allow the channel process. We will need to find a way to integrate this with Phoenix. For now do this: Ecto.Adapters.SQL.Sandbox.mode(FS.Repo, {:shared, self()}) after checkout and you will be good to go.

[pdawczak]

Hey @josevalim! Thank you very much for your super quick answer!

This simple thing helped me with my problem.

I'm afraid I'm not in position to help implementing proper solution for the problem, but would that help if I opened a pull request with a note regarding this very issue? Please, take a look at the change: here.

Hope this might be useful!

Again - thank you!

[josevalim]

Thanks for the info. I don't think we should talk about Phoenix channels in Ecto docs but no worries, we will figure it out before Ecto 2.0 is released and adapt Phoenix accordingly.

[pdawczak]

Ok. Thank you for your help!

[hubertlepicki]

I just wanted to let you know that this is not only issue with channels. This looks like it is an issue with any process in an app other than the default ones created by Phoenix.

For example, if you have a background job worker, that is a supervised GenServer (for example) and it tries to perform some operations on database, the above error will appear.

So possibly would be good to mention the above configuration with shared sandbox mode, in cases people have more complicated set ups than default Phoenix scaffold...

[josevalim]

Yes, that's how ownership works. Check out the docs for the SQL Sandbox
pool, it covers such scenarios and how to solve them.

On Friday, May 6, 2016, Hubert Łępicki [email protected] wrote:

I just wanted to let you know that this is not only issue with channels.
This looks like it is an issue with any process in an app other than the
default ones created by Phoenix.

For example, if you have a background job worker, that is a supervised
GenServer (for example) and it tries to perform some operations on
database, the above error will appear.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/elixir-lang/ecto/issues/1319#issuecomment-217368018

_José Valimwww.plataformatec.com.br
http://www.plataformatec.com.br/Founder and Director of R&D_

[chrismcg]

@josevalim Is manual checkout going to be the default for newly generated apps? So far every time I've tried to use it I've had to switch to shared mode because I'm using another process to do some work that hits the DB.

[michalmuskala]

Ecto itself has no default - you need to explicitly choose it.

I believe phoenix defaults to manual mode for async tests and shared mode for non-async tests.

[chrismcg]

I realise now my confusion was because I ran into this problem writing my first acceptance test. The AcceptanceCase in the README for Wallaby doesn't have a switch similar to Phoenix for async/sync so I've created a PR to add it.

[josevalim]

The AcceptanceCase in the README for Wallaby doesn't have a switch similar to Phoenix for async/sync so I've created a PR to add it.

:heart: :green_heart: :blue_heart: :yellow_heart: :purple_heart:

[lessless]

The problem still persists in the setup_all callback even after following https://gist.github.com/chrismccord/29100e16d3990469c47f851e3142f766#tests instructions

It's caused by Repo.insert(changeset)

[josevalim]

@lessless why are you using setup_all? You need to setup one connection per test and not one connection for all tests in a case. :)

[lessless]

We are testing ES integration and prior to ES interaction entity's model should exist in the PostgreSQL database, so initially we thought about inserting it in the setup_all, once for all tests.
(though I'm not sure about Phoenix/Ecto cleanup strategy, probably transactions, and later on we moved insertion to the setup hook).

[josevalim]

Then you need to checkout a connection in setup_all and then once per setup, as they run in different processes!

[nicooga]

I get this error after changing a portion of my code to run asynchronously using Task.async/1 (this code uses Ecto). I've already have t his in my spec_helper.exs:

~~~elixir
ESpec.configure fn(config) ->
config.before fn(_tags) ->
:ok = Ecto.Adapters.SQL.Sandbox.checkout(YaVende.Repo)
Ecto.Adapters.SQL.Sandbox.mode(Some.Repo, {:shared, self()})
end

config.finally fn(_shared) ->
Ecto.Adapters.SQL.Sandbox.checkin(Some.Repo, [])
end
end
~~~

I think I need to better understand how the connection strategies work.

[EduardoBorsa]

I will to describe how I manage to solve this problem and related ones that I had:
First of, I am using Hound 1.0.3 and PhantomJS.

1) If you are in Ubuntu, don't use apt-get PhantomJS, get it from the website.

2) If yout are using Ubuntu 17 you will need to install some libraries by hand. (Just get the error, google for it and go to go)

3) On test/test_helper.exs put:
"{:ok, _} = Application.ensure_all_started(:hound)"
before "ExUnit.start()"

4) on config/test.exs change to:
config :name_of_your_app, APPNAMEWeb.Endpoint,
http: [port: 4001],
server: true

5) in config/test.exs
config :name_of_your_app, sql_sandbox: true
config :hound, drive: "phantomjs"

6) On lib/appName_web/endpoint.ex and before the plug APPName.Router add:
if Application.get_env(:app_name, :sql_sandbox) do
plug(Phoenix.Ecto.SQL.Sandbox)
end

And finally
7) In each specific test add: (or create a template)
defmodule APPNAMEWeb.PageNameForTest do
use ExUnit.Case
use Hound.Helpers

setup do
  :ok = Ecto.Adapters.SQL.Sandbox.checkout(AppName.Repo)
  metadata = Phoenix.Ecto.SQL.Sandbox.metadata_for(AppName.Repo, self())
  Hound.start_session(metadata: metadata)
  alias AppName.Repo
  alias AppName.Module.NameOfStruct
  Repo.insert(%NameOfStruct{name: "Tomato", price: 50, is_seasonal: false, category: "vegetables"})
  Repo.insert(%NameOfStruct{name: "Apple", price: 100, is_seasonal: true, category: "fruits"})
  :ok
end

test "Name of the Test" do
  navigate_to("/")

  page_title =
    find_element(:css, ".element-to-Search")
    |> visible_text()

  assert page_title =~ "Element Name"
end

end