Azure-pipelines-tasks: Secret passed in to Bash task as argument only works with $(MYSECRET) not $MYSECRET

[GrahamDSmith]

In Azure Pipelines I can pass normal variables to a v3 Bash script (Shell Script task) as an argument using $MYVARIABLE. However, secrets (which can only be passed in as arguments) do not get passed in using the $MYSECRET form but they do get passed in using the $(MYSECRET) form. I've not found this documented anywhere. Is this a bug?

[bryanmacfarlane]

You have to map secrets into the env block of the step. @ericsciple can point to example

[GrahamDSmith]

Thanks @bryanmacfarlane. Just to clarify, I have this working fine, however the issue is that I've found secrets need to be passed to a script with parentheses round the word, ie $(MYSECRET), as opposed to anything that isn't a secret which can be passed without parentheses around the word ie $MYPLAINVARIABLE. I haven't seen this need for the parentheses for secrets documented anywhere and am wondering if it's a bug because I'm sure it will trip some people up.

[ericsciple]

That is intentional. Secrets are not automatically added into the environment block. They must be explicitly mapped in. Either into the env block, or into one of the task inputs. Examples are here.

[GrahamDSmith]

Thanks @ericsciple , but that's not the issue. Please see previous comments.

[ericsciple]

@GrahamDSmith what's the issue? The behavior you are describing aligns with the documentation I linked.

[GrahamDSmith]

The problem @ericsciple is one of consistency. If I can pass non-secrets to a script using $MYPLAINVARIABLE then for consistency I should be able to pass secrets using $MYSECRET. This lack of consistency will trip some people up which is what I'm trying to address. So the fix is one of these:

• Allow secrets to be passed in as $MYSECRET to be consistent with $MYPLAINVARIABLE
• Force all arguments to be passed in as $(MYPLAINVARIABLE) to be consistent with $(MYSECRET)
• Make the documentation very clear that secrets have to be passed in to a script in a very specific way, ie $(MYSECRET) works but $MYSECRET doesn't.

The second option is obviously a breaking change so one or three would be preferable.

Cheers!

[dylan-smith]

Are you sure it's doing what you think. When you pass in a non-secret using the $MYPLAINVARIABLE i suspect that it's just getting passed to your task as a string (AzDO is not converting it to the value), then in the script $MYPLAINVARIABLE resolves to the environment variable so things work.

I wouldn't expect AzDO to process anything that doesn't have the $() syntax.

[GrahamDSmith]

Hi @dylan-smith

Thanks for the reply. I've tested this as follows:

• New variable created as Name = Graham, Value = Smith
• the Graham variable is passed to the bash script as a second parameter as $GRAHAM
• In the script I have echo "TEST is $2"
• In the log output I have TEST is Smith

I'm fairly new to Bash scripts and passing parameters to them in Azure DevOps but this does feel like the $MYPLAINVARIABLE syntax is (unexpectedly!) working or have I missed something?

Cheers - Graham

[dylan-smith]

I think bash is just being too smart. Here's what I see:

I created a build with 2 variables $username and $foo (values of dylan and blah). I have a bash task that looks like this:

test.sh is:

echo "USERNAME is $1"
echo "FOO is $2"

It prints out:

USERNAME is dylan
FOO is blah

Which is the same behavior you are seeing, but if you look at the build log more closely you will see this:

Generating script.
[command]"C:\Program Files\Git\bin\bash.exe" --noprofile --norc -c pwd
/d/a/1/s
Formatted command: . '/d/a/1/s/test.sh' $USERNAME blah

so it really is passing in "$USERNAME" as a literal string, and bash is just doing something "smart" with it

[GrahamDSmith]

What happens if test.sh is:

echo "TEST is $1"

ie USERNAME is not used in the script?

Whatever is happening though, the net effect for end users is the same, ie a lack in consistency which will trip some people up.

[ericsciple]

This section contains a table which describes where the azure-pipelines macro syntax $( ) can be used, and describes alternate syntaxes from accessing environment variables within a script (batch, powershell, and bash).

And this section talks about working with secret variables which intentionally must by mapped-in using macro syntax. You are correct it's inconsistent, however we intentionally chose that behavior to help prevent accidental leaking of secrets (crash dumps, etc).

I think the confusion is, the filename + arguments that you specify are written to a temporary script. And bash is used to run that command. So when you are using bash variable-syntax (e.g. $FOO) in the arguments input, those arguments are getting interpreted by bash. Many customers desire this behavior and it makes the task more consistent with what they type on the command line.

If you have any specific feedback for the docs, I'm happy to pass it along.

[GrahamDSmith]

Not sure if this is specific feedback, but I think the whole scenario of passing secrets to Bash scripts via arguments needs to be explained explicitly and clearly, and be easy to stumble upon.

[ericsciple]

Thanks, makes sense. Personally I agree, as I've seen it be a very common stumbling point for customers in the past.

I'll relay the feedback and follow up with appropriate folks tomorrow.

[ericsciple]

@vtbassmatt fyi

[ericsciple]

@andyjlewis fyi

[lawrencegripper]

To highlight this one I also got confused, to add to the confusion the UI seems to suggest that I can pass my secret as a normal bash variable and gives me an example... following the example fails.

[vtbassmatt]

That's an unfortunate bit of UI - thanks for reporting. I'll see if we can disable that "helpful" text when it's a secret.

@wnjenkin I think that's in your area.

[collinbarrett]

I am having issues getting secrets passed into my bash tasks here and here. For some reason, they don't seem to be injected properly. I can use the script task and map the variables right in from YAML successfully, but I read that the more secure approach is mapping them into the shell's environment variables. Anybody spot what I'm doing incorrectly?

[collinbarrett]

That last commit worked. Not sure exactly what solved it. Sorry to hijack this Issue. Maybe it'll help somebody out. It'd be great to have a bash sample alongside the powershell sample here.

[vtbassmatt]

@collinbarrett when you use env, we don't do any case-munging or replace . with _. So when you said:

- script: ...
  env:
    GitHubName: $(GitHub.Name)

then we were injecting exactly $GitHubName into your environment. I see that you've fixed it so that you're using uppercase everywhere.