Cli: force:apex:execute does not exit with non-zero exit code when script does not compile

[OscarScholten]

Hi,

Today I noticed that force:apex:execute does not exit with a non-zero exit code when the script that is supplied does not compile. I did not experiment further, but it would be good if it also exits with non-zero in case of execution exceptions, etc. The command does exit with a non-zero exit code when the file does not exist.

Cheers, Oscar

Steps To Reproduce:

Open a terminal/dos box and execute the following commands

C:UsersOscartmp>echo "i do not compile" > foo.apex

C:UsersOscartmp>sfdx force:apex:execute -u dx -f foo.apex
ERROR: Compilation failed.
! ERROR: Line: 1, Column: 1
! Unrecognized symbol '"', which is not a valid Apex identifier.
ERROR: Execution failed.

C:UsersOscartmp>echo %ERRORLEVEL%
0

Expected result

The process should have exited with a non-zero exit code.

Actual result

The process exits with a 0 exit code, which is inconsistent with general accepted behavior of command line tools.

Additional information

I'm running this on Windows 10 Pro

Version info:

C:UsersOscartmp>sfdx -v
sfdx-cli/7.1.4-79f97a7df8 win32-x64 node-v10.15.3

[clairebianchi]

@ntotten, this looks like an error on the apex:execute command. Can you take a look

[ntotten]

Yup, this looks like a bug. I added this to our backlog.

[pawel-id]

it seems that current version sfdx-cli/7.75.0 darwin-x64 node-v12.18.3 still has the issue (tested on mac os and linux)

This complicates CI/CD implementations where it is expected that pipeline should fail in case of error... Does anyone has maybe a workaround for that?

[jamesweakley]

A workaround is to dump to json file and then parse the result flag (not the status code which is also always 0 in the json).

Then if the result is not true you can also output the compilation error or the full logs.

Example, if you have a bunch of apex files that are doing setup/integration tests/whatever, in the CI/CD pipeline you can do:

#!/bin/bash
set -e

cd integration_tests

for f in ./*.apex; do
    echo "Running test $f"
    # substitute environment variables in the apex code
    envsubst < "$f" > "$f.tmp" && mv "$f.tmp" "$f"
    # execute the apex code, capture the output to file
    sfdx force:apex:execute -f "$f" --json > out.json
    # check for apex compilation success
    COMPILE_SUCCESS=`cat out.json | jq -r .result.compiled`
    if [ $COMPILE_SUCCESS != "true" ]
    then
        # The result json contains a terse error description without too much noise
        cat out.json | jq -r .result
        exit 1
    fi
    # check for apex execution success
    EXECUTION_SUCCESS=`cat out.json | jq -r .result.success`
    if [ $EXECUTION_SUCCESS != "true" ]
    then
        # Returning just the logs value will mean it renders correctly in terminal without escaped characters
        cat out.json | jq -r .result.logs
        exit 1
    fi

    rm -f out.json
done

Requires jq to parse the json, the envsubst thing is not a requirement but useful in CI/CD environments

[NatalyaMurphyCS]

If you don't have jq installed, here's an adaptation of @jamesweakley's script above that uses the built-in grep command instead. You could get fancier with this and grep just the first 2 lines of the output to ensure you're really getting the actual status, but the chances of the exact lines "Compiled successfully." and "Executed successfully." being elsewhere in the file are pretty slim.

#!/bin/bash
set -e

cd integration_tests

for f in ./*.apex; do
    echo "Running test $f"
    # substitute environment variables in the apex code
    envsubst < "$f" > "$f.tmp" && mv "$f.tmp" "$f"
    # execute the apex code, capture the output to file
    sfdx force:apex:execute -f "$f"  > out.txt

    # check for apex compilation success - should return "1" if the line was found
    COMPILE_CHECK=`grep -c -x 'Compiled successfully.' out.txt`

    if [ $COMPILE_CHECK -eq 0 ]
    then
        cat out.txt
        echo "******FAILED to compile apex script $f. ***********"
        exit 1
    fi
    # check for apex execution success - should return "1" if the line was found
    EXECUTION_CHECK=`grep -c -x 'Executed successfully.' out.txt`

    if [ $EXECUTION_CHECK -eq 0 ]
    then
        cat out.txt | grep 'Exception'
        echo "*****************Failed while running apex script $f.  See errors above*************"
        exit 1
    fi

    rm -f out.txt
done