Terraform: Windows ${path.module} doesn't handle relative paths

[jodastephen]

Terraform Version

v0.9.6

Behavior

We have the following structure of modules:

- accounts
  - dev
    - main.tf  // the root
- modules
  - application
    - main.tf  // the module file shown below
  - core
    - application-json.template  // a non-terraform file

This is the modules\application\main.tf file:

resource "aws_api_gateway_integration" "calculate-post-integration" {
  ...
  request_templates {
    "application/json" = "${file("${path.module}/../core/application-json.template")}"
  }
}

This works on Mac, but not on Windows, so I'm pretty sure there is a bug here.

On Mac, the ${path.module}/../core/application-json.template correctly resolves the file in the sibling modules\application\core folder. On Windows, the same path is resolved relative to the root instead, trying and failing to find the file in accounts\core.

The problem is caused by the relative "up a level" double dot path. Using ${path.module} with a file in the same folder works fine.

References

This may be a duplicate of, or related to, #14798.

Thanks!

[apparentlymart]

Hi @jodastephen! Sorry for this weird behavior.

To understand what's going on here, it would be helpful to know exactly what path.module is expanding to. If you are able, it would be useful to if you could share the output you see when you run terraform console and then enter the following expression at the prompt:

"${path.module}/../core/application-json.template"

To exit the console, you can type exit at the prompt.

[jodastephen]

When terraform console is run from accounts\dev, I get this:

// broken
E:\dev\myapp\terraform\accounts\dev\../core/application-json.template

When run from modules\application I get:

// broken
E:\dev\myapp\terraform\modules\application\../core/application-json.template

Terraform console for "${path.module}/application-json.template" output:

// working
E:\dev\myapp\terraform\accounts\dev/application-json.template

---

For info, this structure works (where the core folder is beneath application):

- accounts
  - dev
    - main.tf  // the root
- modules
  - application
    - main.tf  // the module file shown below
    - core
      - application-json.template  // a non-terraform file

Giving this Terraform console "${path.module}/core/application-json.template" output:

// working
E:\dev\myapp\terraform\modules\application/core/application-json.template

All on Windows 7 Professional

[apparentlymart]

Thanks for sharing those examples, @jodastephen!

I guess this must be a quirk in how mixtures of back and forward slashes are handled on Windows. :thinking:

I wonder if we could address it by making the file function normalize the path using filepath.Clean would work around this... it would hopefully change your first example to E:\dev\myapp\terraform\accounts\core\application-json.template .

Unfortunately my Windows dev VM has expired so I need to download a fresh one before I can test this, which I probably won't have time to do today. If anyone else is working on Windows and has the time and motivation to give this a try, I'd love to review a PR! The code for this function is in config/interpolation_funcs.go. Otherwise, I'll take a look when I have some time to get a dev environment spun up again.

[aleybovich]

@apparentlymart Have you had a chance to look into fixing this issue?

[apparentlymart]

Unfortunately my attention has been on different aspects of the configuration language for a while now, so I've not been able to look into this one. This function will need to be refactored (along with all the others) as part of our current project to improve the configuration language in general, so we may be able to take care of this along the way, but I won't be able to look at this myself right now.

[azeez-ct-wonga]

Is this fixed?

[JackOneil]

got this error too

[wkrick]

Is there a workaround for this issue? Windows boxes handle forward slashes in paths perfectly fine so we need a way to make the code that populates ${path.module} always use forward slashes.

[apparentlymart]

The problem seems to arise when both types of slashes are used together in the same path, which suggests the following workarounds:

• Use backslashes for the rest of the path when a configuration is expected to be applied on Windows. This approach is only suitable if you know a particular configuration will never be used on a non-Windows machine.
• Use replace(path.module, "\\", "/") to force the path to always use forward slashes, even on Windows. This should create a configuration that will work on both Windows and non-Windows machines.

For the latter workaround, it could help to do it in a named local value so that the complex expression is only in one place:

locals {
  module_path = "${replace(path.module, "\\", "/")}"
}

resource "aws_api_gateway_integration" "calculate-post-integration" {
  ...
  request_templates {
    "application/json" = "${file("${local.module_path}/../core/application-json.template")}"
  }
}

Making the path values always use forward slashes, regardless of platform, is one way this could be resolved. That could be a breaking change for anyone relying on the first workaround above though, so we'll need to try some different options here and see what works best. My original idea of making the file function do the normalization itself would solve the original problem as reported, but would not address other situations where paths are accepted, such as resource type arguments that accept paths directly.

[wkrick]

Replace almost does the trick but we're still having an issue because we're using the path with "local-exec" to execute a script in bash on a windows box. The script is in a folder under the module. I think bash is having trouble with the path starting with "C:" even if we use replace to convert the slashes.

We have some developers on windows and some on non-windows machines. We're trying to configure the modules to be platform agnostic if possible.

For example, this snippet of code from the module's main.tf still fails...

locals {
  module_path = "${replace(path.module, "\\", "/")}"
}

provisioner "local-exec" {
  command = "${local.module_path}/scripts/ourscriptname ${self.id} ${var.aws["region"]}"
  interpreter = ["bash", "-c"]
}

[apparentlymart]

Indeed local-exec presents some further challenges here because you also need to contend with how other software on the system is configured.

Depending on how that bash was built for Windows, it might be linked to a C library that is expecting "POSIX-like" file paths. For example, Cygwin maintains its own virtual filesystem where Windows-style drive letters can in principle appear _anywhere_, while I believe MSYS uses some mechanical convention like replacing the leading C:\ with /c/, though I must admit I'm rusty on that and may be misremembering.

Unfortunately that's a much harder problem for Terraform to address, since it would need awareness of how the external program is built in order to know which of the many filesystem path formats are required here. If you need to use local-exec as part of your configuration, it would probably make sense to wrap Terraform in something that can ensure a consistent runtime environment for those external programs, such as a docker image with all of the necessary software installed.

If that isn't possible, you could consider using a more cross-platform language runtime to write the script, where the actions required can be expressed directly in that language rather than by executing other programs on the host system. For example, interpreter could be set to a Python interpreter if the action you wish to take is possible to express in pure python and you can guarantee that Python is installed on any machine where Terraform will run.

If anyone has some other thoughts on ways we might address that particular problem I'd encourage opening a separate issue for it, since I think it's a much larger problem and I don't want addressing that to distract from the more tractable problem of paths that Terraform consumes directly itself.

[apparentlymart]

For the forthcoming v0.12.0 release we're going to move to using forward slashes consistently on all platforms for path.module, path.root, and path.cwd. This means that this will always combine properly with additional path elements using forward slashes, like "${path.module}/../core/application-json.template", since Windows supports forward slashes as long as they are used consistently throughout the path.

This doesn't address the other more complex issues I described in later comments, such as the different ways drive letters can be handled in different POSIX emulation layers on Windows when interacting with provisioners. Since those were just hypotheticals from me, we're going to hold on those for now and address them as a separate issue if they arise in practice.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.