Terraform: `terraform state show ADDR` outputs ANSI sequences

[abitrolly]

Terraform Version

Terraform v0.12.2
+ provider.aws v2.12.0
+ provider.null v2.1.2
+ provider.template v2.1.2

Actual Behavior

✗ cat testtf.py     
#!/usr/bin/env python3

from subprocess import check_output

out = check_output('terraform state show aws_security_group.allow_ssh', shell=True)
print(out)

✗ ./testtf.py 
b'# aws_security_group.allow_ssh: \nresource "aws_security_group" "allow_ssh" {\n    \x1b[1m\x1b[0marn\x1b[0m\x1b[0m                    = "arn:aws:ec2:us-east-1:71897290:security-group/sg-09570458c5"\n    \x1b[1m\x1b[0mdescription\x1b[0m\x1b[0m            = "Allow SSH via 22 (standart)"\n    \x1b[1m\x1b[0megress\x1b[0m\x1b[0m                 = []\n    \x1b[1m\x1b[0mid\x1b[0m\x1b[0m                     = "sg-09570458c5"\n    \x1b[1m\x1b[0mingress\x1b[0m\x1b[0m                = [\n        {\n            cidr_blocks      = [\n                "0.0.0.0/0",\n            ]\n            description      = ""\n            from_port        = 22\n            ipv6_cidr_blocks = []\n            prefix_list_ids  = []\n            protocol         = "tcp"\n            security_groups  = []\n            self             = false\n            to_port          = 22\n        },\n    ]\n    \x1b[1m\x1b[0mname\x1b[0m\x1b[0m                   = "allow_ssh"\n    \x1b[1m\x1b[0mowner_id\x1b[0m\x1b[0m               = "71897290"\n    \x1b[1m\x1b[0mrevoke_rules_on_delete\x1b[0m\x1b[0m = false\n    \x1b[1m\x1b[0mtags\x1b[0m\x1b[0m                   = {}\n    \x1b[1m\x1b[0mvpc_id\x1b[0m\x1b[0m                 = "vpc-d99a3"\n\n    timeouts {}\n}\n\n\x1b[0m\x1b[0m\n'



md5-0ce8f68f910b8e4ce2123eb08712ba9f



# aws_security_group.allow_ssh: 
resource "aws_security_group" "allow_ssh" {
    arn                    = "arn:aws:ec2:us-east-1:...:security-group/sg-..."
    description            = "Allow SSH via 22 (standart)"
    egress                 = []
    id                     = "sg-..."
    ingress                = [
        {
            cidr_blocks      = [
                "0.0.0.0/0",
            ]
            description      = ""
            from_port        = 22
            ipv6_cidr_blocks = []
            prefix_list_ids  = []
            protocol         = "tcp"
            security_groups  = []
            self             = false
            to_port          = 22
        },
    ]
    name                   = "allow_ssh"
    owner_id               = "..."
    revoke_rules_on_delete = false
    tags                   = {}
    vpc_id                 = "vpc-..."

    timeouts {}
}

Additional Context

I wrote parser for terraform state show to generate Ansible inventory, but I didn't expect to deal with ANSI sequences.

[apparentlymart]

Thanks for reporting this, @abitrolly.

The terraform state show output is intended for human consumption via a terminal rather than for machine consumption, so it's subject to change at any time to improve Terraform's UI. With that said, if it's going to include escape sequences in its output then it should support the same -no-color option we support on other commands that generate escape sequences, so I think this issue will represent making sure that option works for this command so that escape sequences can be disabled in scenarios where they aren't appropriate, such as capturing Terraform output into log systems that don't support terminal sequences.

Terraform 0.12 introduced terraform show -json as a machine-oriented way to look at the state in a JSON format. If possible I'd recommend using that command to obtain state data for further analysis; although its structure is still subject to change for the moment as we gather feedback on it, we intend to be very conservative about changes and eventually make compatibility promises about it.

[abitrolly]

@apparentlymart when I execute state show ADDR from my terminal, I don't see any colors. Why ANSI colors appear when I run the command through the script?

I took a second look at terraform show -json format. It is less friendly for humans and I need to pass it through jq . to make sense of it, but it looks like it is much easier to parse. I will try it.

[007]

@abitrolly The ANSI escapes that it's outputting are all "resets", where they set the state back to default / nocolor / nohighlight / etc. So the effect on a terminal that has no existing state is a no-op. To illustrate:

tput setaf 2;echo "this is green";echo "this is still green?"

shows two green lines, versus

tput setaf 2;echo "this is green";terraform state show $SOMETHING;echo "this is still green?"

Which shows one green line, one green line of the resource, and then resets to terminal-default for the rest of the lines and the green?.

@apparentlymart In 0.12 is there any way to reproduce terraform state show in a way that is machine-consumable? Or a way to get -json as an option for state show instead of just show?

EDIT: I just read the docs, and they directly contradict your statement (emphasis added):

The attributes are listed in alphabetical order (with the except of "id" which is always at the top). They are outputted in a way that is easy to parse on the command-line.

[ausfestivus]

Ive put this issue on the radar of our SE to see if we can get this issue under the nose of the product manager for consideration in 0.12.7. Hopefully thats the correct way to get this issue some visibility.

Ive got some chunky import work in the pipe and this issue is going to bite the customers plans.

[dohoangkhiem]

+1 for terraform state show -json, it would be much convenient.

[tdmalone]

Further to @007's comment above on the docs for state show, the help shown for terraform state -h also states:

The structure and output of the commands is specifically tailored to work
well with the common Unix utilities such as grep, awk, etc. We recommend
using those tools to perform more advanced state tasks.

We have a script that has been doing exactly that, and has broken after the upgrade to 0.12. While changes in output are not unexpected, debugging our greps to find out they were failing due to invisible ANSI colour codes _was_ unexpected 😅

If these commands are no longer designed to be used programmatically, the help and docs should be updated to note that. But, a -no-color option would be excellent too!

(And yes - we will eventually rewrite to use terraform show -json and parse appropriately, was just hoping to at least make some slight changes to existing scripts first).

[tdmalone]

Workaround:

terraform state show "your_resource_address" | sed -Ee 's/\x1B\[[0-1]m//g'

(disclaimer: YMMV, depending on your version of sed)

[abitrolly]

It would be more fun if it could be sponsored by my employee (and I don't have any), but it looks like I could probably write a patch for it.

https://github.com/hashicorp/terraform/blob/master/command/state_show.go#L131

There might be a bug that even if the Color is disabled for state show, it still outputs the invisible sequences. Or just explicitly hardcode color disability here.