Ejabberd: Issues compiling 17.03 as a dependency of Phoenix project

[jglover]

What version of ejabberd are you using?

17.03

What operating system (version) are you using?

Mac OS Sierra 10.12.2

How did you install ejabberd (source, package, distribution)?
Hex.pm package
Added dependency to mix.exs specifying GitHub url (using 17.3 causes SemVer issue).

{:ejabberd, ">= 17.03.0", github: "processone/ejabberd"}
mix deps.get works fine, mix deps.compile fails with the log output in the next section.

What did not work as expected? Are there error messages in the log? What
was the unexpected behavior? What was the expected result?

I encountered issues compiling due to apparent source issues which seems wrong because 17.03 is a released (stable) version. I feel like I'm missing something obvious?

==> ejabberd
Compiling 1 file (.asn1)
Compiling 1 file (.yrl)
Compiling 242 files (.erl)
src/mod_pres_counter.erl:28: Warning: behaviour gen_mod undefined
src/mod_ping.erl:32: Warning: behaviour gen_mod undefined
src/ejabberd_access_permissions.erl:32: Warning: behaviour ejabberd_config undefined
src/mod_client_state.erl:31: Warning: behaviour gen_mod undefined
src/mod_muc_room.erl:3691: field jid undefined in record muc_subscribe
src/mod_muc_room.erl:3696: variable 'SubJid' is unbound
src/mod_muc_room.erl:3698: field jid undefined in record muc_subscribe
src/mod_muc_room.erl:3735: field jid undefined in record muc_unsubscribe
src/mod_muc_room.erl:3740: variable 'UnsubJid' is unbound
src/mod_muc_room.erl:3742: field jid undefined in record muc_unsubscribe
src/randoms.erl:53: Warning: crypto:rand_bytes/1 is deprecated and will be removed in a future release; use crypto:strong_rand_bytes/1
src/mod_http_upload.erl:627: undefined macro 'NS_HTTP_UPLOAD_0'
src/mod_http_upload.erl:688: undefined macro 'NS_HTTP_UPLOAD_0'
src/mod_http_upload.erl:473: function iq_disco_info/4 undefined
src/mod_http_upload.erl:482: record upload_request_0 undefined
src/mod_http_upload.erl:487: variable 'CType' is unbound
src/mod_http_upload.erl:487: variable 'File' is unbound
src/mod_http_upload.erl:487: variable 'Size' is unbound
src/mod_http_upload.erl:487: variable 'XMLNS' is unbound
src/mod_http_upload.erl:510: function mk_slot/3 undefined
src/mod_http_upload.erl:513: function mk_slot/3 undefined
src/mod_http_upload.erl:620: spec for undefined function mk_slot/3
src/mod_http_upload.erl:670: spec for undefined function iq_disco_info/4
src/mod_muc_mnesia.erl:28: Warning: behaviour mod_muc_room undefined
src/mod_muc_sql.erl:30: Warning: behaviour mod_muc_room undefined
src/mod_muc_riak.erl:28: Warning: behaviour mod_muc_room undefined
could not compile dependency :ejabberd, "mix compile" failed. You can recompile this dependency with "mix deps.compile ejabberd", update it with "mix deps.update ejabberd" or clean it with "mix deps.clean ejabberd"
==> 
** (Mix) Encountered compilation errors

Any help much appreciated!

[cromain]

you need some dependencies in the path to compile ejabberd. see ejabberd's mix.exs for erlc_options
you'll need your project to add similar calls to deps_include, and define that function to produce path like deps/${dep}/include

[jglover]

Thanks @cromain ! Appreciate the help
Forgive my ignorance as I'm fairly new to Mix but I don't understand why it's causing a problem since those dependencies are defined in ejabberd's mix.exs and it is defined as a dependency of my project?
It seems odd to me that during compilation it doesn't read from ejabberd's own mix file and pull and compile it's dependencies when running mix.get and mix.compile?
I'm just trying to understand how it works, thank you :)

[reneklacan]

@cromain I'm having similar issue, everything goes ok and then I get:

could not compile dependency :ejabberd, "mix compile" failed. You can recompile this dependency with "mix deps.compile ejabberd", update it with "mix deps.update ejabberd" or clean it with "mix deps.clean ejabberd"
==> fetchat
** (Mix) Expected :version to be a SemVer version, got: "17.03.0"

Seems like mix is enforcing correct semver and ejabberd is using invalid version (17.03.0 vs 17.3.0)

See https://github.com/elixir-lang/elixir/blob/c655ac091a1b760ea99abe4f0036accb012877de/lib/elixir/lib/version.ex#L256-L267

iex(4)> Version.parse("17.03.0")
:error
iex(5)> Version.parse("17.3.0")
{:ok, #Version<17.3.0>}

Any chance this can be fixed?

[reneklacan]

you need some dependencies in the path to compile ejabberd. see ejabberd's mix.exs for erlc_options

@cromain If that's true then you should probably update your documentation https://docs.ejabberd.im/developer/extending-ejabberd/elixir/#embedding-ejabberd-in-an-elixir-application because nothing like that is mentioned

[reneklacan]

@jglover to successfully compile phoenix project with ejabberd you can currently do following changes to mix.exs:

1. Add erlc_options and deps_include methods:

  defp erlc_options do
    # Use includes from ejabberd dependencies
    includes = deps_include(["fast_xml", "xmpp", "p1_utils"])
    [:debug_info, {:d, :ELIXIR_ENABLED}] ++ Enum.map(includes, fn(path) -> {:i, path} end)
  end

  defp deps_include(deps) do
    base = case Mix.Project.deps_paths()[:ejabberd] do
      nil -> "deps"
      _ -> ".."
    end
    Enum.map(deps, fn dep -> base<>"/#{dep}/include" end)
  end

1. Use erlc_options method in project list:

  def project do
    [
      ...
      erlc_options: erlc_options(),
    ]
  end

1. Add fork to your deps (fork only fixes version and mix warnings):

  defp deps do
    [
      ...
      {:ejabberd, "17.3.0", github: "reneklacan/ejabberd", branch: "working-17-3"},
    ]
  end

1. Run mix deps.get and mix compile and you should be good to go

[jglover]

Thank you very much @reneklacan , that worked perfectly! I figured it would be along those lines from @cromain 's comments but I didn't get around to trying it.
I'm guessing you forked and fixed the SemVer issue by changing the version number in mix.exs your branch? I did it locally for now but please make a PR for it, it's going to cause so many issues with Mix.

I thought it was odd that when doing deps.compile it couldn't use all of the config from the dependencies' mix file, but after doing some reading on Mix (RTFM), normal behaviour in the project function is to return a keyword list of configuration anyway, the Phoenix one by default calls plenty of functions. To my knowledge there doesn't seem like there's a way for dependencies to extend on that though which is a shame, copying erlc_options function feels like a step that shouldn't be needed., hopefully Mix will have that in the future.

Thanks guys, much appreciate your help :)

[reneklacan]

I'm guessing you forked and fixed the SemVer issue by changing the version number in mix.exs your branch? I did it locally for now but please make a PR for it, it's going to cause so many issues with Mix.

Yes, exactly. I already created PR for it https://github.com/processone/ejabberd/pull/1759

I thought it was odd that when doing deps.compile ... copying erlc_options function feels like a step that shouldn't be needed., hopefully Mix will have that in the future.

100% agree

[jglover]

@reneklacan I spoke too soon, now it isn't compiling for me. Despite having those changes in my mix.exs (deps_include, erlc_options functions & added erlc_options to project list), 17.03 -> 17.3 change was merged to master a few minutes ago.

[reneklacan]

@jglover Use my branch, processone's master is currently not working because it's relying on some changes in xmpp library that hasn't been released yet (eg this constant https://github.com/processone/xmpp/commit/06d4d5720c2609c5b44ca59400fddcf944d6f3fa#diff-374df80684a5cc6e2e1080f0b65ddbd0R159)

[jglover]

Ah I didn't notice that, thanks!

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.