Azure-cli: "az vm run-command invoke" cannot pass parameters to a Linux machine via a script file

[Carlovo]

Describe the bug
"az vm run-command invoke" cannot pass parameters to a Linux machine via a script file.

To Reproduce
Make a file called script.sh containing:

#!/bin/bash

cd /home && echo "foo" >> test.txt && echo $1 >> test.txt

Execute the script above on a VM with Azure CLI with a command that looks like this:
az vm run-command invoke --resource-group MyRG --name MyVM --command-id RunShellScript --scripts '@script.sh' --parameters bar

If I log into the VM in Azure and cat /home/test.txt it gives me:

Where is "bar"?

Expected behavior
Now I directly run the following Azure CLI command:
az vm run-command invoke --resource-group MyRG --name MyVM --command-id RunShellScript --scripts 'cd /home && echo "foo" >> test.txt && echo $1 >> test.txt' --parameters bar

If I now log into the VM in Azure and cat /home/test.txt it gives me:

bar

This is how I expect it to be.

Furthermore, this bug does not happen when executing a PowerShell script on Windows...

Additional context
I tries this (and ton of other stuff). Nothing worked to get the parameters into the shell script.

[Carlovo]

Well, it turns out this actually works if you omit the:

#!/bin/bash

I find this behavior strange still and it's not documented, but I feel it's not really a bug.

[qwordy]

I tried and I got the same confusing result. It's quite interesting.
text.txt has a empty line after foo. So echo $1 is executed. But $1 is empty.
script.sh is successfully uploaded to /var/lib/waagent/run-command/download in my experiment.
I can get correct result when I manually run it with parameter.

[qwordy]

I find that the request body is {"commandId": "RunShellScript", "script": ["#!/bin/bash\r\ncd /home && echo \"foo\" > test.txt && echo $1 >> test.txt"], "parameters": [{"name": "arg1", "value": "bar"}]}
So CLI expands the file to string, then send it to server.

[Carlovo]

Ah nice to see that!

Probably the Az command uses a different bash than the default #!/bin/bash and then loses it's environment variables when switching. If this is so, that's not wrong, but it should at least be documented somewhere. Starting bash scripts with a Shebang is so natural, it should be mentioned somewhere if this should not be done with the Azure CLI.

[adewaleo]

@Carlovo The CLI does not interpret the script, it sends the contents of the script to the run-command service which interprets and executes what was sent by the CLI on the VM.

According to az vm run-command show --command-id runshellscript -l westus. The run-command service expects the script to be a list of lines. However when the CLI reads in the script using @{file} it passes a single line with new lines.

Given the vm is a linux vm, It is possible that it doesn't run the script properly because the script contains carriage returns -\r\n - instead of regular new lines \n. I won't be surprised if regular unix-style new lines behave differently.

I have two suggestions / questions:

What happens if you run:

az vm run-command invoke --resource-group MyRG --name MyVM --command-id RunShellScript --scripts '#!/bin/bash' 'cd /home && echo "foo" >> test.txt && echo $1 >> test.txt' --parameters bar

Also, what happens if you edit the content of the scripts to use unix-style new lines \n instead of windows-style new lines \r\n.

[Carlovo]

Sorry for the long delay. Anyway, I tried.

Running the command you state works. Probably because it gets executed as two separate scripts.
Running the script with Unix line endings doesn't work.

[adewaleo]

I doubt it gets run as separate scripts.

az vm run-command show --command-id runshellscript -l westus

{
  "description": "Custom multiline shell script should be defined in script property. Optional parameters can be set in parameters property.",
  "id": "RunShellScript",
  "label": "Executes a Linux shell script",
  "osType": "Linux",
  "parameters": [
    {
      "defaultValue": null,
      "name": "arg1",
      "required": false,
      "type": "string"
    },
    {
      "defaultValue": null,
      "name": "arg2",
      "required": false,
      "type": "string"
    }
  ],
  "schema": "http://schema.management.azure.com/schemas/2016-11-17/runcommands.json",
  "script": [
    "echo This is a sample script",
    "echo Optional parameters: $0 $1"
  ]
}

Emphasis on
"script": [
"echo This is a sample script",
"echo Optional parameters: $0 $1"
]

It shows how to use the run-command endpoint. You pass the lines of your script as a list of lines.

What payload is sent on to Azure (ARM) via the CLI when you run each of the two scenarios above with --debug?

[qwordy]

Can anyone in VM service team help with this issue?
In this command, Azure CLI will read the file content and send it to server.

{"commandId": "RunShellScript", "script": ["#!/bin/bash\r\ncd /home && echo \"foo\" > test.txt && echo $1 >> test.txt"], "parameters": [{"name": "arg1", "value": "bar"}]}

Should I split lines?
I tried and I got the same confusing result as the author's result. It's quite interesting.
text.txt has a empty line after foo. So echo $1 is executed. But $1 is empty.
The script is successfully saved to /var/lib/waagent/run-command/download in my experiment.
I can get correct result when I manually run it with parameter.
How does the VM invoke the script?

[yonzhan]

@qwordy you can @ someone from VM service team for help.

[koralski]

Please refer to the Linux example provided by the command:
az vm run-command invoke -h

az vm run-command invoke -g MyResourceGroup -n MyVm --command-id RunShellScript --scripts
'echo $0 $1' --parameters "arg1=hello" "arg2=world"

You are supposed to pass named parameters even if you don't intend to use the names.
These parameters are parsed and added at the top of your script in the format:

export arg1=hello arg2=world
set -- hello world

You can SSH to your VM and see how you script is modified before execution.

[heoelri]

I spent some time during the last days with problem. We're trying to execute a Script and handing over parameters that are needed to run the script.

We're calling az vm run-command invoke from a AzureCLI@1 task. The inline script looks like this:

az vm run-command invoke -n customer-${{ parameters.hostname }}-vm-$(stage) \
--command-id RunShellScript \
--resource-group customer-${{ parameters.resourcegroup }}-$(stage) \
--debug \
--scripts @shared-services/buildagent/prepare_buildagent.sh --parameters "AZP_URL=${{ parameters.azpurl }}" "AZP_TOKEN=$(patAdoBuildagentsToken)" "AZP_AGENT_NAME=customer-${{ parameters.hostname }}-vm-$(stage)" "AZP_POOL=${{ parameters.azppool }}" "AZP_WORK='_work'"

All the parameters contain values. That something we've doublechecked using a bash task to export their values.

Within the script itself we tried a whole bunch of options already to access the parameters. We started with $1, $2, .. and later we switched to named parameters as mentioned by @koralski above.

Currently we're trying it with the following code block:

echo $@

for ARGUMENT in "$@"
do

    KEY=$(echo $ARGUMENT | cut -f1 -d=)
    VALUE=$(echo $ARGUMENT | cut -f2 -d=)   

    case "$KEY" in
            AZP_URL)        export AZP_URL=${VALUE} ;;
            AZP_TOKEN)      export AZP_TOKEN=${VALUE} ;;     
            AZP_AGENT_NAME) export AZP_AGENT_NAME=${VALUE} ;;
            AZP_POOL)       export AZP_POOL=${VALUE} ;;
            AZP_WORK)       export AZP_WORK=${VALUE} ;;
            *)   
    esac    
done

But it does not really work. Not even $@ contains something. Has one of you guys already successfully used az vm run-command invoke successfully with a script and parameters for a Linux vm?

[heoelri]

Based on @Carlovo 's comment we've removed #!/bin/bash from our script file and now it works.

[qwordy]

Comment copied from #9596
The request body:

{"commandId": "RunShellScript", "script": ["#!/bin/bash\r\nprintenv"], "parameters": [{"name": "arg1", "value": "Param1"}]}

No "param1" in ENV.

If I use
Comment copied from #9596

az vm run-command invoke -g rg -n vm --command-id RunShellScript --scripts '#!/bin/bash' 'printenv' --parameters 'Param1'

request body is

{"commandId": "RunShellScript", "script": ["#!/bin/bash", "printenv"], "parameters": [{"name": "arg1", "value": "Param1"}]}

I can get the correct result. ENV has "param1".

[mayanknayar]

@qwordy, per the last comment from heolri, the issue seems to have been resolved by removing #!/bin/bash from the script. Is there further action needed here?

[Drewm3]

Closing this issue as it sounds like it has been resolved.

[jnoller]

@Drewm3 this is not resolved - the string interpolation that's occuring isn't correct - or not documented. Specifically, omitting the #!/bin/... line is a super weird thing and non intuitive for linux users.

[mayanknayar]

@koralski can you comment on the string interpolation? And do we require #!/bin/bash to be removed?

[koralski]

Parameter enumeration start from 0 instead of 1. The difference is that parameters definition is embedded in the script instead of passed outside in a command line (where $0 is equals to bash).
If you see the example we start from $0:
echo $0 $1

About the other issue where script starts with #!/bin/bash
I discovered a bug in the parameter inserting logic where we insert the parameter definition after the first line if starts with #!
The problem is the whole script is in one string and all parameter definition ends up inserted after the script. We will fix that and new release is expected to deploy by the end of April 2020.

[koralski]

About the other issue where script starts with #!/bin/bash

This issue was fixed end of April 2020.

[msftbot[bot]]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @Drewm3, @avirishuv, @axayjo, @vaibhav-agar.