elixirc does not compile modules from files

[MaximusWhite]

I have a folder with sub-folders containing .ex/.exs files with Elixir modules. All I need to do is go through all the folders and simply compile the modules.
I have a small PowerShell script to do that:

$files = get-childitem . -Recurse -Filter *.ex*
foreach ($file in $files)
{   
    elixirc $file.Fullname          # tried $file.Name as well, doesn't work either
}

For some reason that does not do the job.
Passing neither $file.Fullname nor $file.Name works.
Very confusing, because it's as simplistic of a task as it can get.

Environment

• Elixir & Erlang/OTP versions (elixir --version): Erlang/OTP 21 [erts-10.3] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1]
• Operating system: Windows 10, v. 10.0.18363 Build 18363

Current behavior

For each {filename_here}.ex/.exs file matched, I get a message:
No files matched pattern(s) {filename_here}.ex

Expected behavior

For each file, a module should be compiled.

[josevalim]

If you call it as elixirc path/to/file.ex directly, does it work? If so, it means it is most likely a bug in the script and how it calls elixirc, rather than Elixir.

[MaximusWhite]

It does not work, it seems. Still returns the same error.
This is something I noticed some time ago actually, elixirc seems not to work with absolute or relative paths (at least for me). It only works if I manually go to the directory where file.ex resides and call elixirc on it from there. Otherwise, it refuses to see the file.

[josevalim]

So we have tests that run full paths and they work. Does anything change if you use elixirc.bat path/to/file? Also, are you using elixirc path/to/file or elixirc path\to\file? Does it have any difference in behaviour?

[MaximusWhite]

Hmmm... I used elixirc path\to\file format (since that's Windows formatting) and it didn't work.
Somehow changing it to elixirc path/to/file seems to be working.
Is that a Powershell issue? Or is it Elixir issue because it's expecting non-Windows path format on Windows?

[josevalim]

It would be an Elixir issue then. Thanks for checking!

[MaximusWhite]

Ah, I see. Thanks for quick reply!

For those having the same issue: a workaround for me is to manually transform Windows-style path to Unix-style path inside my script,

[josevalim]

So I was able to track the change of behaviour to this commit: https://github.com/erlang/otp/commit/36dc96339e2b2b692e6dfe6de43db3a2348732bd

All of our command line options that accept patterns from the command line does not work with backslashes at the moment. We could rewrite backslashes to forward slashes but then it will introduce the same bug that the Erlang commit above wanted to prevent. For example, if you write this: "foo\bar*.ex" on Windows, does it mean in Unix terms that you want to lookup for "foo/bar*.ex" or "foo/bar/*.ex"?

@bjorng apologies for pinging you but if you have any insights on how to move forward, it would be very appreciated. Thanks!

[bjorng]

@josevalim I suggest using filename:join([Pattern]) to normalize the pattern before calling filelib:wilcard/1,2. That will convert backslashes to forward slashes on Windows but not on Unix. That means that backslashes cannot be used on Windows to protect special characters, but that should not matter as those special characters (*, {, and so on) are not allowed characters in filenames on Windows (as far as I remember).

[josevalim]

@bjorng good idea, thank you.

If that’s the case, couldn’t we change Erlang to only escape \ on Unix systems? Your commit message talks about consistency across OSes but the file API is already “inconsistent” (I mean it as a positive thing). For example, file:open will accept paths with \ on Windows but not on Unix.

On the other hand, it seems { and } are still valid on Windows paths (? And * aren’t), which means the ambiguity would still exist in those cases.

[bjorng]

@josevalim The idea for the change of filelib:wildcard/1,2 was that a hard-coded pattern in the source code should match the same files on both Windows and Unix. We didn't consider patterns originating from the user.

I don't want to change the semantics of wildcard again, and I would want to avoid adding options for different behavior.

Yes, you are right. The characters that are special in Windows wildcards are not allowed in file names. { and } are not special in Windows wildcards, so they are allowed in file names.

[josevalim]

I don't want to change the semantics of wildcard again, and I would want to avoid adding options for different behavior.

Thanks, I understand. Another option is to introduce Path.external_wildcard or similar, but given this is a breaking change in Elixir, I think I will do an initial pass on the path when running on Windows before calling Erlang.