Quarkus: Unable to copy .env file in dev mode

[Legion2]

Describe the bug
The .env file for environment variables at project root can't be used on Windows 10.
When creating this file and running mvnw quarkus:dev it produces a warning in the log and the environment variables are not used by quarkus.
Expected behavior
I can define environment variables in .env file and use it in application.properties like this:
greeting.message = ${MESSAGE}

Actual behavior
A warning and exception is printed to the console:

Listening for transport dt_socket at address: 5005
__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
2020-03-31 12:12:54,098 WARN  [io.qua.dev.DevModeMain] (main) Unable to copy .env file
2020-03-31 12:12:57,080 ERROR [io.qua.application] (main) Failed to start application: java.util.NoSuchElementException: Property MESSAGE not found
        at io.smallrye.config.SmallRyeConfig.propertyNotFound(SmallRyeConfig.java:209)
        at io.smallrye.config.SmallRyeConfig.getValue(SmallRyeConfig.java:96)
        at io.smallrye.config.SmallRyeConfig.getValue(SmallRyeConfig.java:80)
        at io.quarkus.runtime.configuration.ConfigExpander.accept(ConfigExpander.java:55)
        at io.quarkus.runtime.configuration.ConfigExpander.accept(ConfigExpander.java:15)
        at org.wildfly.common.expression.ExpressionNode.emit(ExpressionNode.java:42)
        at org.wildfly.common.expression.Expression.evaluateException(Expression.java:75)
        at org.wildfly.common.expression.Expression.evaluate(Expression.java:89)
        at io.quarkus.runtime.configuration.ExpandingConfigSource.expandValue(ExpandingConfigSource.java:101)
        at io.quarkus.runtime.configuration.ExpandingConfigSource.expand(ExpandingConfigSource.java:69)   
        at io.quarkus.runtime.configuration.ExpandingConfigSource.getValue(ExpandingConfigSource.java:47) 
        at io.quarkus.runtime.configuration.DeploymentProfileConfigSource.getValue(DeploymentProfileConfigSource.java:80)
        at io.smallrye.config.SmallRyeConfig.getRawValue(SmallRyeConfig.java:121)
        at io.smallrye.config.SmallRyeConfig.getValue(SmallRyeConfig.java:84)
        at io.smallrye.config.SmallRyeConfig.getOptionalValue(SmallRyeConfig.java:131)
        at io.quarkus.arc.runtime.ConfigRecorder.validateConfigProperties(ConfigRecorder.java:35)
        at io.quarkus.deployment.steps.ConfigBuildStep$validateConfigProperties24.deploy_0(ConfigBuildStep$validateConfigProperties24.zig:120)
        at io.quarkus.deployment.steps.ConfigBuildStep$validateConfigProperties24.deploy(ConfigBuildStep$validateConfigProperties24.zig:36)
        at io.quarkus.runner.ApplicationImpl.doStart(ApplicationImpl.zig:154)
        at io.quarkus.runtime.Application.start(Application.java:90)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at io.quarkus.runner.bootstrap.StartupActionImpl.run(StartupActionImpl.java:99)
        at io.quarkus.dev.IsolatedDevModeMain.firstStart(IsolatedDevModeMain.java:60)
        at io.quarkus.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:236)
        at io.quarkus.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:39)
        at io.quarkus.bootstrap.app.CuratedApplication.runInCl(CuratedApplication.java:131)
        at io.quarkus.bootstrap.app.CuratedApplication.runInAugmentClassLoader(CuratedApplication.java:84)
        at io.quarkus.dev.DevModeMain.start(DevModeMain.java:113)
        at io.quarkus.dev.DevModeMain.main(DevModeMain.java:54)

2020-03-31 12:12:57,111 ERROR [io.qua.dev.DevModeMain] (main) Failed to start Quarkus: java.lang.RuntimeException: Failed to start quarkus
        at io.quarkus.runner.ApplicationImpl.doStart(ApplicationImpl.zig:232)
        at io.quarkus.runtime.Application.start(Application.java:90)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at io.quarkus.runner.bootstrap.StartupActionImpl.run(StartupActionImpl.java:99)
        at io.quarkus.dev.IsolatedDevModeMain.firstStart(IsolatedDevModeMain.java:60)
        at io.quarkus.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:236)
        at io.quarkus.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:39)
        at io.quarkus.bootstrap.app.CuratedApplication.runInCl(CuratedApplication.java:131)
        at io.quarkus.bootstrap.app.CuratedApplication.runInAugmentClassLoader(CuratedApplication.java:84)
        at io.quarkus.dev.DevModeMain.start(DevModeMain.java:113)
        at io.quarkus.dev.DevModeMain.main(DevModeMain.java:54)
Caused by: java.util.NoSuchElementException: Property MESSAGE not found
        at io.smallrye.config.SmallRyeConfig.propertyNotFound(SmallRyeConfig.java:209)
        at io.smallrye.config.SmallRyeConfig.getValue(SmallRyeConfig.java:96)
        at io.smallrye.config.SmallRyeConfig.getValue(SmallRyeConfig.java:80)
        at io.quarkus.runtime.configuration.ConfigExpander.accept(ConfigExpander.java:55)
        at io.quarkus.runtime.configuration.ConfigExpander.accept(ConfigExpander.java:15)
        at org.wildfly.common.expression.ExpressionNode.emit(ExpressionNode.java:42)
        at org.wildfly.common.expression.Expression.evaluateException(Expression.java:75)
        at org.wildfly.common.expression.Expression.evaluate(Expression.java:89)
        at io.quarkus.runtime.configuration.ExpandingConfigSource.expandValue(ExpandingConfigSource.java:101)
        at io.quarkus.runtime.configuration.ExpandingConfigSource.expand(ExpandingConfigSource.java:69)   
        at io.quarkus.runtime.configuration.ExpandingConfigSource.getValue(ExpandingConfigSource.java:47) 
        at io.quarkus.runtime.configuration.DeploymentProfileConfigSource.getValue(DeploymentProfileConfigSource.java:80)
        at io.smallrye.config.SmallRyeConfig.getRawValue(SmallRyeConfig.java:121)
        at io.smallrye.config.SmallRyeConfig.getValue(SmallRyeConfig.java:84)
        at io.smallrye.config.SmallRyeConfig.getOptionalValue(SmallRyeConfig.java:131)
        at io.quarkus.arc.runtime.ConfigRecorder.validateConfigProperties(ConfigRecorder.java:35)
        at io.quarkus.deployment.steps.ConfigBuildStep$validateConfigProperties24.deploy_0(ConfigBuildStep$validateConfigProperties24.zig:120)
        at io.quarkus.deployment.steps.ConfigBuildStep$validateConfigProperties24.deploy(ConfigBuildStep$validateConfigProperties24.zig:36)
        at io.quarkus.runner.ApplicationImpl.doStart(ApplicationImpl.zig:154)
        ... 13 more

To Reproduce
Steps to reproduce the behavior:

1. git clone https://github.com/quarkusio/quarkus-quickstarts.git (tag: 1.3.1.Final)
2. go to config-quickstart directory
3. add a .env file in the config-quickstart directory, see below for the content
4. modify application.properties like below
5. run mvnw quarkus:dev

Configuration
application.properties:

# Your configuration properties
greeting.message = ${MESSAGE}
greeting.name = quarkus

.env:

MESSAGE=hello

Screenshots
(If applicable, add screenshots to help explain your problem.)

Environment (please complete the following information):

• Output of uname -a or ver: Microsoft Windows [Version 10.0.18363.720]
• Output of java -version:
  openjdk version "1.8.0_232" OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_232-b09) OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.232-b09, mixed mode)
• GraalVM version (if different from Java): not installed
• Quarkus version or git rev: 1.3.1.Final
• Build tool (ie. output of mvnw --version or gradlew --version):
  Apache Maven 3.6.2 (40f52333136460af0dc0d7232c0dc0bcf0d9e117; 2019-08-27T17:06:16+02:00) Maven home: C:\Users\Leon\.m2\wrapper\dists\apache-maven-3.6.2-bin\795eh28tki48bv3l67maojf0ra\apache-maven-3.6.2 Java version: 1.8.0_232, vendor: AdoptOpenJDK, runtime: C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre Default locale: de_DE, platform encoding: Cp1252 OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

Additional context
Feature was added in #7519

[geoand]

This is weird since the feature is being tested in Windows as well. We would need someone with a Windows machine to take a look and find the root cause of this.
@jaikiran would you be able to take a look?

[jaikiran]

I borrowed a Windows setup and gave this a quick try and I haven't been able to reproduce this. I even tried hot reload and a few other things and even that worked fine.
I've opened https://github.com/quarkusio/quarkus/pull/8319 so that we atleast know why it's failing on @Legion2's setup

[geoand]

Thanks a lot @jaikiran!!!

[jaikiran]

@Legion2, the linked PR has been merged to upstream master branch. Would it be possible for you to build the latest Quarkus master branch and then use the 999-SNAPSHOT version of Quarkus in your application and get us the exception stacktrace that you would see when the .env file copying fails?

[Legion2]

I didn't get the exception printed in english, but here is the corresponding translation: "A required privilege is not held by the client".

2020-04-03 10:16:24,686 WARN  [io.qua.dev.DevModeMain] (main) Unable to copy .env file: java.nio.file.FileSystemException: C:\Users\Leon\git\TIK\viplab\quarkus-quickstarts\config-quickstart\target\.env: Dem Client fehlt ein erforderliches Recht.

        at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
        at sun.nio.fs.WindowsFileSystemProvider.createSymbolicLink(WindowsFileSystemProvider.java:585)
        at java.nio.file.Files.createSymbolicLink(Files.java:1043)
        at io.quarkus.dev.DevModeMain.copyDotEnvFile(DevModeMain.java:137)
        at io.quarkus.dev.DevModeMain.start(DevModeMain.java:107)
        at io.quarkus.dev.DevModeMain.main(DevModeMain.java:54)

But I can create the symlink by hand from the same terminal I started quarkus dev:

C:\Users\Leon\git\TIK\viplab\quarkus-quickstarts\config-quickstart\target>mklink .env ..\.env
symbolische Verknüpfung erstellt für .env <<===>> ..\.env

[arvidvillen]

I have the same issue, is there any known workarounds?

[geoand]

Not unless we can figure out what exactly is causing the symlink creation failure on Windows (it works on Windows CI)

[Legion2]

Does the windows CI use administrator rights?

[haraldatbmw]

I have the same warning on my Windows10 machine if I call mvn quarkus:dev in a command-prompt without administrator rights.
If I open a command-prompt with admin rights Quarkus starts without warnings.

--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
2020-04-23 12:09:16,529 WARN  [io.qua.dev.DevModeMain] (main) Unable to copy .env file: java.nio.file.FileSystemException: C:\git\Back2Code.CampusStudents\feedylight\target\.env: Dem Client fehlt ein erforderliches Recht.

        at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
        at java.base/sun.nio.fs.WindowsFileSystemProvider.createSymbolicLink(WindowsFileSystemProvider.java:585)
        at java.base/java.nio.file.Files.createSymbolicLink(Files.java:1058)
        at io.quarkus.dev.DevModeMain.copyDotEnvFile(DevModeMain.java:137)
        at io.quarkus.dev.DevModeMain.start(DevModeMain.java:107)
        at io.quarkus.dev.DevModeMain.main(DevModeMain.java:54)

@geoand Can you please retest without admin rights?

[haraldatbmw]

Per default my windows user has not the permission to create symbolic links.
To overcome this restriction i had to follow these steps:

• Run secpol.msc
• Go to Security Settings|Local Policies|User Rights Assignment|Create symbolic links
• Add your user name
• Restart your windows session

But I don't think this is a good way to go. Preventing the usage of symbolic links would be better!

[Legion2]

@haraldatbmw You could also just enable Windows Developer Mode https://blogs.windows.com/windowsdeveloper/2016/12/02/symlinks-windows-10/

[haraldatbmw]

@Legion2 Windows Developer Mode did not work for me.

[manuelbl]

Please don't use symbolic links. They usually require elevated privileges on Windows 10 and therefore just cause pain.

[geoand]

What is the proper Windows alternative?

[manuelbl]

There isn't any really. You will have to copy the file.

But I guess a symbolic link would be preferable as the .env file is likely to contain credentials and they shouldn't be copied unnecessarily.

[geoand]

The reason why we add a symbolic is to be able to track changes to and hot deploy Quarkus when that happens.

[jayakrishnankarolilnlsn]

Created a new ticket for a similar issue - #9752. gradlew quarkusDev does not recognize .env file and it is not copied to /build directory.

[jayakrishnankarolilnlsn]

One more issue found. The property definition in .env file is case sensitive.

Say for example

application.properties:

greeting.message = ${MESSAGE}
greeting.name = ${name}

.env:

MESSAGE=hello  # -> succeeds
name=quarkus   # -> fails

Run command mvn quarkus:dev

[geoand]

Please open a new issue for that find.

[famod]

@geoand

What is the proper Windows alternative?

I just checked that on my private machine with Windows 10 Home (please don't ask why it is not a Pro) and with Home you don't even have secpol.msc so you cant even grant yourself the right to create symlinks. Again, please don't ask. 😉

But I was able to create a link via:

mklink.exe /H <linkname> <target>

This will create a _hard_ link, though. But AFAICS this would be a sufficient workaround.
The code could fall back to mklink.exe on Windows if createSymbolicLink() fails (or we figure out how to check it up-front).

mklink should be available for all relevant Windows versions.

Btw, because of this issue DevMojoIT.testThatTheApplicationIsReloadedOnDotEnvConfigChange always fails on my private machine.

PS: Without any elevated rights you can also create a "directory junction" with mklink but I don't see how that can be of real use here.

[geoand]

That is excellent information @famod , thanks a lot! Would you like to submit a PR proposing such a fix?

[famod]

I can try, yes. It is rather unlikely that this would end up in 1.6.x, isn't it?

[geoand]

If it's just a small bug fix, then we can probably backport it. If it's more involved, then it would probably end up in 1.7

[famod]

10316 should fix this. Btw, turns out mklink is not a "standalone" .exe (cmd.exe /C mklink ... has to be used).