Elixir: Issue with loading existing atoms from other modules

[tallakt]

Environment

• Elixir version (elixir -v):
  1.3.0-rc.0 (but also on older Elixir versions)
• Operating system:
  Ubuntu 3.1.16-0-73-generic (on virtualbox VM)
  

  Current behavior

A demonstrator for the bug may be found at https://github.com/tallakt/test_for_atom_elixir_bug

Running mix test fails ~50% on my machine.

The issue being that the atom table of the module does not seem to get loaded before a function on that module has been called. This leads to subtle, seemingly random failures, for instance if pattern matching is being used in that module. To catch this error you need to setup the project folder accurately (hence the git repo), but a simplified version of the code is:

defmodule A do
  def foo(:atom_used_only_in_a), do: ok
  def say_ok: do: :ok
end

defmodule B do
  def wrapper(s), do: A.foo(String.to_existing_atom(s))
  def say_ok, do: A.say_ok
end


# this would fail
B.wrapper "atom_only_used_in_a"

# this would succeed
B.say_ok
B.wrapper "atom_only_used_in_a"

Expected behavior

mix test should always run without errors.

[josevalim]

@tallakt everything seems to work as expected.

1. Your tests run in random order so they will fail from time to time depending on the seed value
2. "The issue being that the atom table of the module does not seem to get loaded before a function on that module has been called." - this is always true, the atom table for a module only gets loaded when that module is loaded (and calling a function always loads it). Therefore, you need to make sure the module has been loaded before calling to_existing_atom. One way to do so is by adding:

@on_load :load_atoms

def load_atoms() do
  Enum.each [Foo, Bar], &Code.ensure_loaded?/1
  :ok
end

[tallakt]

Thanks for clearing that out, also with a solution :)

[tim2CF]

@josevalim but if I have for example big amount of modules, and I don't want to think which modules I should preload, can I use code like this on application start?

I can sacrifice few seconds on application start to do it, it's not a problem. But do you know, are there some other disadvantages of doing things like this?

    :code.get_path
    |> Enum.each(fn(dir) ->
      dir
      |> File.ls!
      |> Stream.filter(&(String.ends_with?(&1, ".beam")))
      |> Stream.map(&(Regex.replace(~r/(\.beam)$/, &1, fn(_,_) -> "" end) |> String.to_atom))
      |> Enum.each(fn(module) ->
        module
        |> Code.ensure_loaded?
        |> case do
          true ->
            :ok
          false ->
            {:module, ^module} =
              "#{dir}/#{module}"
              |> String.to_charlist
              |> :code.load_abs
        end
      end)
    end)

[OvermindDL1]

@tim2CF Generally questions like this should be asked at the Elixir Forums. :-)

However, if you really want to preload all modules on start instead of on demand, this is precisely what embedded mode is for (elixir defaults to interactive mode). Everything listed in the boot script is then fully loaded.

However, the better question is 'why' are you needing to do this? It is usually the wrong thing to do and usually something like a plugin system is better. :-)

Ask on the forums though with the actual thing you are trying to accomplish. :-)

[timCF]

The issue is the same - if module is compiled, but not loaded yet, String.to_existing_atom will not work, :erlang.apply will not work, :erlang.function_exported will not work, and some other functions will not work as well :)

Jose suggested solution like

@on_load :load_atoms

def load_atoms() do
  Enum.each [Foo, Bar], &Code.ensure_loaded?/1
  :ok
end

But here you have to specify all modules explicitly, I'm looking for more generic solution. Maybe build_embedded is thing I'm looking for

[OvermindDL1]

For a generic solution I use a plugin style system, anything that needs to register atoms I just have them load themselves into the running system via their own applications.

Embedded mode does preload all modules.

But still, this is how the BEAM works and is not something Elixir can change, so this should be asked on the forums. :-)
https://elixirforum.com/

[timCF]

thanks, sure!