Elixir: Support --warnings-as-errors on mix test

[TheFirstAvenger]

--warnings-as-errors on mix test would allow automated build systems to catch warnings in test files. Currently, since test files don't get compiled with mix compile, and --warnings-as-errors isn't recognized on mix test, there is no way to root out these warnings in test files via automated builds or linting tools.

[josevalim]

@perc-MikeBinns I think this one should be solved with:

$ MIX_ENV=test mix do compile --warning-as-errors, test

It would be really hard to support all compiler options through mix test.

[nathany]

I tried running my tests with that command but I still get warnings

MIX_ENV=test mix do compile --warning-as-errors, test --exclude remote

example.ex:23: warning: variable x is unused

For now I'm running the following on CI

mix compile --warnings-as-errors
mix test --exclude remote

[nathany]

Sorry, my fault. A warning-as-errors vs. warnings-as-errors typo.

UPDATE: the typo is also in https://github.com/elixir-lang/elixir/issues/3223#issuecomment-87724948

[harlantwood]

$ MIX_ENV=test mix do compile --warning-as-errors, test

This did not halt tests on warnings (which are emitted during test, not during compile) in my case.

However, what did work was adding this to my mix project:

elixirc_options: [warnings_as_errors: true]

[ghost]

This is not working for me, I see the warning that is in the test itself as it runs the test but it just shows everything passed.

[AdrianSchneider]

With

elixirc_options: [warnings_as_errors: true]

The test compilation fails as expected for me. But, subsequent runs will print the warning but not fail. Given it's not being compiled the second time (without the file changing) it makes sense, but is a little confusing.

[dylan-chong]

The problem with elixirc_options: [warnings_as_errors: true] is that you can't say have this enabled on the CI where you want to fail on warnings, but in the local environment it would be very annoying if you had to remove unused aliases and unused variables every time you wanted to run the test

[whatyouhide]

@dylan-chong you can do elixirc_options: [warnings_as_errors: System.get_env("CI") == "true"] or whatever mechanism you use to detect if you're on CI :)

[dylan-chong]

@whatyouhide You are a god :D

[wojtekmach]

My favourite option is changing CI config, instead of a single mix test invocation I have three:

MIX_ENV=test mix deps.compile
MIX_ENV=test mix compile --warnings-as-errors
mix test

the advantage is when I need to debug the output tests, I don't have a wall of compilation output in front (most CIs neatly group command output per command)

[dylan-chong]

@wojtekmach Doesnt that mean you dont get errors for warnings in test files themselves as mix compile only works on src, not test

[wojtekmach]

Right, mix compile doesn’t work on test/ so there’s no difference between a single mix test and mix compile —warnings-as-errors && mix test.

On 27 Apr 2020, at 21:47, Dylan Chong [e-mail] wrote:

@wojtekmach https://github.com/wojtekmach Doesnt that mean you dont get errors for warnings in test files themselves as mix compile only works on src, not test

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/elixir-lang/elixir/issues/3223#issuecomment-620195960, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAASSJ57UXI6N6V446WBYNLROXON5ANCNFSM4A64RHBQ.

[dylan-chong]

With the elixirc_options: [warnings_as_errors: System.get_env("CI") == "true"] method, has anyone had issues where errors are raised as warnings the first time you run CI=true mix test but then rerunning that command does not produce the error (even though the unused variable warning is printed) - something to do with caching?