Terragrunt: Option for less verbose output

[andrewhertog]

Is there an option that will hide a lot of the extra information that terragrunt outputs? While this information is nice to see, it would be nice to have a clean output that would just show us the raw output from terraform.

[brikis98]

Not currently, but there should be. I'd welcome a PR to move Terragrunt to a proper logging system with INFO/DEBUG/etc and a way to set the log level via CLI. We've been meaning to move it over to gruntwork-cli as the base, which has that sort of logging built-in, but haven't had time. A PR for that is very welcome.

In the meantime, a workaround is to note that Terragrunt logs everything to stderr, so you could do terragrunt apply 2>/dev/null to suppress it.

[ezh]

solution for bash only:

terragrunt apply 2> >(grep -v "\[terragrunt]" >&2)

STDOUT and STDERR lines are preserved except [terragrunt] ones

[josegonzalez]

One side-effect of this is that terragrunt output var - where var is a variable defined in your output - has a bunch of intermediate output, so you necessarily need to redirect stderr to ensure you're only getting what you want.

As well, if you're overriding the terraform-source, then there is extra output that is added to stdout, making terragrunt output var useless.

[brikis98]

Terragrunt logs everything to stderr, so terragrunt out <var> should work just fine. If not, please open a separate bug with a way to repro.

[josegonzalez]

Sorry, I meant that the regex above will not catch all lines from terragrunt - specifically, if you have an un-initialized module, some lines will be missing the prefix. Apologies!

I'd assume that the terraform call _also_ outputs to stderr in some instances, so redirecting all stderr wouldn't be a great solution in those cases.

[brikis98]

Sorry, I meant that the regex above will not catch all lines from terragrunt - specifically, if you have an un-initialized module, some lines will be missing the prefix. Apologies!

Ah, I gotcha, thanks for clarifying.

I'd assume that the terraform call also outputs to stderr in some instances, so redirecting all stderr wouldn't be a great solution in those cases.

Yea, Terraform also logs to stderr. However, command where you are explicitly asking something to be printed such as terraform output write the requested info to stdout.

[voiski]

use the scenery for this, it will filter out the terragrunt output and you will also win colors =D

--no-color will remove any colors if you need

terragrunt ... | scenery

[tomaszdudek7]

@voiski

Unfortunately, scenery did not work for me.

It has problems both with terragrunt and terraform 0.12 :(

[scottrbaxter]

To add to the bash solution provided by @ezh, I ran into issues with testing syntax during tf .11 --> .12 upgrades (e.g. include { needs to be updated to include = { for tf12), and terragrunt plan-all 2> >(grep -v "\[terragrunt]" >&2) would not show me the error, since it was being redirected. I had to remove this inverse grep stderr redirect to see anything other than:

Command exited with non-zero status 1

I came up with a slightly more explicit regex that helps prune away some of the excess terragrunt stderr output that we don't generally need to see when running every pipeline, however still shows the output to errors such as the one above.

Solution:
2> >(grep -v "^\[terragrunt]\ \[" >&2)

Basically, this grep inversely redirects any lines that start with [terragrunt] [, such as:

[terragrunt] [</path/to/terragrunt.hcl>] 2020/01/20 17:58:40 Running command: terraform --version

We'll still see errors, and other output, such as
1st example:

[terragrunt] 2020/01/20 17:44:53 Error processing module at '/path/to/terragrunt.hcl'. How this module was found: Terragrunt config file found in a subdirectory of path/to. Underlying error: /path/to/terragrunt.hcl:33,15-16: Missing key/value separator; Expected an equals sign ("=") to mark the beginning of the attribute value.

2nd example:

AccessDenied: User: arn:aws:iam::<account-no>:user/<aws-user> is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::<account-no>:role/<aws-role>
    status code: 403, request id: <request-id>
[terragrunt] 2020/01/20 18:22:21 Unable to determine underlying exit code, so Terragrunt will exit with error code 1
Command exited with non-zero status 1

I do strongly feel that debugging levels would be very beneficial with terragrunt, and would love to see them implemented eventually, so long as this tool continues to be useful. I do understand that it's getting less necessary alongside terraform with the recent additions found in .12, however there still seems to be enough use unique cases out there that it feels worth the while to implement a way to manage levels of output verbosity - especially considering this stderr redirect (work-around) requires calling bash, when we often use alpine based containers that don't have bash natively added.

[yanivillouz]

Is there an option that will hide a lot of the extra information that terragrunt outputs? While this information is nice to see, it would be nice to have a clean output that would just show us the raw output from terraform.

redirect stderr to /dev/null and you will have only the terraform outputs in a clean format i.e terragrunt apply-all 2> /dev/null

[scottrbaxter]

redirect stderr to /dev/null

@yanivillouz but what if you're troubleshooting an error? also, with the recursive nature of terragrunt *-all, the output is not consistently in order, and often difficult to determine what output belongs to which path/folder. i think some work on better output for testing/troubleshooting (especially developing new patterns) would be hugely useful, here.

[yanivillouz]

the original question was to get terraform outputs only, this seems to
solves the request.
you are right of course that this is not the approach for debugging issues

On Mon, Feb 24, 2020 at 5:13 PM Scott Baxter [e-mail]
wrote:

redirect stderr to /dev/null

but what if you're troubleshooting an error? also, with the recursive
nature of terragrunt *-all, the output is not consistently in order, and
often difficult to determine what output belongs to which path/folder. i
think some work on better output for testing/troubleshooting (especially
developing new patterns) would be hugely useful, here.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/gruntwork-io/terragrunt/issues/432?email_source=notifications&email_token=ABSAGEE5JEB3HMJEJEXNZFTREPPZJA5CNFSM4EUDU2PKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMYFSMA#issuecomment-590371120,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABSAGEBUMXIFMQDBGJNJVIDREPPZJANCNFSM4EUDU2PA
.

[scottrbaxter]

@yanivillouz Your "solution" is not different from the first very response in this issue. Redirecting all stderr for terraform will definitely omit a lot of important output from terraform itself.

Please help avoid excess noise by reading through the entire conversation, if you feel like contributing to issues.

[dmattia]

On our CI servers, we wanted to display just the plan output when terragrunt ran successfully, but wanted to display all logs if anything went wrong. Here's a gist for a quick node wrapper that achieves our goal for plan calls: https://gist.github.com/dmattia/0d17696bad1dffd90ec7c899e0343955

[plathrop-sfdc]

What I'd mostly like to see is deduplicating path components in the output. What I mean is, lines like this:

[terragrunt] [/Users/USER/Source/observability/cortex/v2/live/development/cortex] 2020/05/04 11:27:06 Downloading Terraform configurations from file:///Users/USER/Source/observability/cortex/v2/modules/cortex into /Users/USER/Source/observability/cortex/v2/live/development/cortex/.terragrunt-cache/7W8Zynt-Dn0PhoK3U4XzB9LjtcY/zVzA93tT2UNDH-crbKmeNMrR8Sw

Could be made significantly less verbose, something like:

[terragrunt] [In /Users/USER/Source/observability/cortex/v2] [.../live/development/cortex] 2020/05/04 11:27:06 Downloading Terraform configurations from file://.../modules/cortex into .../live/development/cortex/.terragrunt-cache/7W8Zynt-Dn0PhoK3U4XzB9LjtcY/zVzA93tT2UNDH-crbKmeNMrR8Sw

[angeloskaltsikis]

I have pushed an update based on @dmattia .
Posting it here in case it helps anyone.