Berry: [Bug] SyntaxError when running a "bin" file that isn't authored in JS

Created on 2 Feb 2020 · 11Comments · Source: yarnpkg/berry

[ ] I'd be willing to implement a fix

Describe the bug

Yarn 2.0 fails with a SyntaxError when running a bin file that isn't authored in JS. For example, the bin file may be an executable file with a shebang #!/usr/bin/env sh.

Example bin file:

#!/usr/bin/env sh
echo 'Hello world!'

To Reproduce

See sample package (@cameronhunter/berry-bin-bug) that defines a bin that is an executable file with a shebang.

```js repro
const { promises: { writeFile, chmod } } = require('fs');

await writeFile('./hello-world', '#!/usr/bin/env sh\necho "Hello world!"');
await chmod('./hello-world', 0o765);

await packageJsonAndInstall({
bin: { 'hello-world': './hello-world' }
});

await expect(yarn('hello-world')).resolves.toContain('Hello world!');


Example of failure:

```sh
$ yarn berry-bin-bug
/Users/chunter/workspace/github/cameronhunter/berry-bin-bug/bin/berry-bin-bug:3
echo "Hello world!"
     ^^^^^^^^^^^^^^

SyntaxError: Unexpected string
    at Module._compile (internal/modules/cjs/loader.js:895:18)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
    at Module.load (internal/modules/cjs/loader.js:815:32)
    at Function.module_1.Module._load (/Users/chunter/workspace/github/cameronhunter/berry-bin-bug/.pnp.js:13519:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1047:10)
    at internal/main/run_main_module.js:17:11

Environment if relevant (please complete the following information):

OS: MacOS Mojave (10.14.5)
Node version: 12.14.0
Yarn version: 2.0.0-rc.28

bug reproducible

Source

cameronhunter

👍4

Most helpful comment

Thanks for your response, @arcanis. Cross-platform isn't a goal for many of our (internal) tools so it's a shame that we have to spin up node to execute a shebang executable. I can certainly work around this using your suggestion though.

Perhaps this ticket could be changed to focus on improving the error message so that other users don't fall into the same confusion.

Side-question: Is cross-platform portability a goal of berry or is it more the implementation detail of not being able to run them from zip archives?

cameronhunter on 3 Feb 2020

👍2

All 11 comments

The reproduction case in your issue seems broken (ie it neither pass nor fail due to throwing an unmanaged exception):

Error: Command failed: /usr/bin/node /github/workspace/scripts/actions/../run-yarn.js berry-bin-bug
/tmp/tmp-27BzdWjpD360Na/.yarn/cache/@cameronhunter-berry-bin-bug-npm-0.0.0-553518ee28-2.zip/node_modules/@cameronhunter/berry-bin-bug/bin/berry-bin-bug:3
echo "Hello world!"
     ^^^^^^^^^^^^^^

SyntaxError: Unexpected string
    at Module._compile (internal/modules/cjs/loader.js:895:18)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
    at Module.load (internal/modules/cjs/loader.js:815:32)
    at Function.module_1.Module._load (/tmp/tmp-27BzdWjpD360Na/.pnp.js:13587:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1047:10)
    at internal/main/run_main_module.js:17:11


    at ChildProcess.exithandler (child_process.js:295:12)
    at ChildProcess.emit (events.js:210:5)
    at maybeClose (internal/child_process.js:1021:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:283:5)

Remember: any non-Jest exceptions will cause the test to be reported as broken. If you expect something to pass without throwing, you must wrap it into something like await expect(...).resolves.toBeTruthy(). If you instead expect something to throw, you need to wrap it into await expect(...).rejects.toThrow().

yarnbot on 2 Feb 2020

The reproduction case in your issue seems broken (ie it neither pass nor fail due to throwing an unmanaged exception):

Error: Command failed: /usr/bin/node /github/workspace/scripts/actions/../run-yarn.js berry-bin-bug
/tmp/tmp-28bSX4haV7o6PU/.yarn/cache/@cameronhunter-berry-bin-bug-npm-0.0.0-553518ee28-2.zip/node_modules/@cameronhunter/berry-bin-bug/bin/berry-bin-bug:3
echo "Hello world!"
     ^^^^^^^^^^^^^^

SyntaxError: Unexpected string
    at Module._compile (internal/modules/cjs/loader.js:895:18)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
    at Module.load (internal/modules/cjs/loader.js:815:32)
    at Function.module_1.Module._load (/tmp/tmp-28bSX4haV7o6PU/.pnp.js:13587:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1047:10)
    at internal/main/run_main_module.js:17:11


    at ChildProcess.exithandler (child_process.js:295:12)
    at ChildProcess.emit (events.js:210:5)
    at maybeClose (internal/child_process.js:1021:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:283:5)

yarnbot on 2 Feb 2020

The reproduction case in your issue seems broken (ie it neither pass nor fail due to throwing an unmanaged exception):

Error: Command failed: /usr/bin/node /github/workspace/scripts/actions/../run-yarn.js berry-bin-bug
/tmp/tmp-27N1R4P7BV4pz1/.yarn/cache/@cameronhunter-berry-bin-bug-npm-0.0.0-553518ee28-2.zip/node_modules/@cameronhunter/berry-bin-bug/bin/berry-bin-bug:3
echo "Hello world!"
     ^^^^^^^^^^^^^^

SyntaxError: Unexpected string
    at Module._compile (internal/modules/cjs/loader.js:895:18)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
    at Module.load (internal/modules/cjs/loader.js:815:32)
    at Function.module_1.Module._load (/tmp/tmp-27N1R4P7BV4pz1/.pnp.js:13587:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1047:10)
    at internal/main/run_main_module.js:17:11


    at ChildProcess.exithandler (child_process.js:295:12)
    at ChildProcess.emit (events.js:210:5)
    at maybeClose (internal/child_process.js:1021:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:283:5)

yarnbot on 2 Feb 2020

We couldn't reproduce your issue (all the assertions passed on master).

yarnbot on 2 Feb 2020

This issue reproduces on master:

Error: expect(received).resolves.toContain()

Received promise rejected instead of resolved
Rejected to value: [Error: Command failed: /usr/bin/node /github/workspace/scripts/actions/../run-yarn.js berry-bin-bug
/tmp/tmp-27MSa69izmI67r/.yarn/cache/@cameronhunter-berry-bin-bug-npm-0.0.0-553518ee28-2.zip/node_modules/@cameronhunter/berry-bin-bug/bin/berry-bin-bug:3
echo "Hello world!"
     ^^^^^^^^^^^^^^

SyntaxError: Unexpected string
    at Module._compile (internal/modules/cjs/loader.js:895:18)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
    at Module.load (internal/modules/cjs/loader.js:815:32)
    at Function.module_1.Module._load (/tmp/tmp-27MSa69izmI67r/.pnp.js:13587:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1047:10)
    at internal/main/run_main_module.js:17:11

]
    at expect (/github/workspace/.yarn/cache/expect-npm-24.8.0-8c7640c562-2.zip/node_modules/expect/build/index.js:138:15)
    at module.exports (evalmachine.<anonymous>:6:7)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13
    at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16)
    at async Object.executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12)
    at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:25:38)
    at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-2.zip/node_modules/clipanion/lib/advanced/Command.js:161:26)
    at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-2.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24)
    at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-2.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

yarnbot on 2 Feb 2020

This issue reproduces on master:

Error: expect(received).resolves.toContain()

Received promise rejected instead of resolved
Rejected to value: [Error: Command failed: /usr/bin/node /github/workspace/scripts/actions/../run-yarn.js berry-bin-bug
/tmp/tmp-27dTJ81SxRH10H/.yarn/cache/@cameronhunter-berry-bin-bug-npm-0.0.0-553518ee28-2.zip/node_modules/@cameronhunter/berry-bin-bug/bin/berry-bin-bug:3
echo "Hello world!"
     ^^^^^^^^^^^^^^

SyntaxError: Unexpected string
    at Module._compile (internal/modules/cjs/loader.js:895:18)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
    at Module.load (internal/modules/cjs/loader.js:815:32)
    at Function.module_1.Module._load (/tmp/tmp-27dTJ81SxRH10H/.pnp.js:13587:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1047:10)
    at internal/main/run_main_module.js:17:11

]
    at expect (/github/workspace/.yarn/cache/expect-npm-24.8.0-8c7640c562-2.zip/node_modules/expect/build/index.js:138:15)
    at module.exports (evalmachine.<anonymous>:6:7)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13
    at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16)
    at async Object.executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12)
    at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:25:38)
    at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-2.zip/node_modules/clipanion/lib/advanced/Command.js:161:26)
    at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-2.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24)
    at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-2.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

yarnbot on 2 Feb 2020

This issue reproduces on master:

Error: expect(received).resolves.toContain()

Received promise rejected instead of resolved
Rejected to value: [Error: Command failed: /usr/bin/node /github/workspace/scripts/actions/../run-yarn.js berry-bin-bug
/tmp/tmp-277pN3smpUUGTq/.yarn/cache/@cameronhunter-berry-bin-bug-npm-0.0.0-553518ee28-2.zip/node_modules/@cameronhunter/berry-bin-bug/bin/berry-bin-bug:3
echo "Hello world!"
     ^^^^^^^^^^^^^^

SyntaxError: Unexpected string
    at Module._compile (internal/modules/cjs/loader.js:895:18)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
    at Module.load (internal/modules/cjs/loader.js:815:32)
    at Function.module_1.Module._load (/tmp/tmp-277pN3smpUUGTq/.pnp.js:13587:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1047:10)
    at internal/main/run_main_module.js:17:11

]
    at expect (/github/workspace/.yarn/cache/expect-npm-24.8.0-8c7640c562-2.zip/node_modules/expect/build/index.js:138:15)
    at module.exports (evalmachine.<anonymous>:6:7)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13
    at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16)
    at async Object.executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12)
    at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:25:38)
    at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-2.zip/node_modules/clipanion/lib/advanced/Command.js:161:26)
    at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-2.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24)
    at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-2.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

yarnbot on 2 Feb 2020

This issue reproduces on master:

Error: expect(received).resolves.toContain()

Received promise rejected instead of resolved
Rejected to value: [Error: Command failed: /usr/bin/node /github/workspace/scripts/actions/../run-yarn.js hello-world
/tmp/tmp-2722xAfw0xGBGh/hello-world:2
echo "Hello world!"
     ^^^^^^^^^^^^^^

SyntaxError: Unexpected string
    at Module._compile (internal/modules/cjs/loader.js:895:18)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
    at Module.load (internal/modules/cjs/loader.js:815:32)
    at Function.module_1.Module._load (/tmp/tmp-2722xAfw0xGBGh/.pnp.js:13576:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1047:10)
    at internal/main/run_main_module.js:17:11

]
    at expect (/github/workspace/.yarn/cache/expect-npm-24.8.0-8c7640c562-2.zip/node_modules/expect/build/index.js:138:15)
    at module.exports (evalmachine.<anonymous>:11:7)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13
    at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16)
    at async Object.executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12)
    at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.38-d4f5e2dbf3-2.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:25:38)
    at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-2.zip/node_modules/clipanion/lib/advanced/Command.js:161:26)
    at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-2.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24)
    at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-2.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

yarnbot on 2 Feb 2020

That would be a wontfix as third-party binaries aren't portable and cannot be run from zip archives. At worst, you can make a JS script that uses execFileSync.

arcanis on 2 Feb 2020

Perhaps this ticket could be changed to focus on improving the error message so that other users don't fall into the same confusion.

Side-question: Is cross-platform portability a goal of berry or is it more the implementation detail of not being able to run them from zip archives?

cameronhunter on 3 Feb 2020

👍2

Side-question: Is cross-platform portability a goal of berry or is it more the implementation detail of not being able to run them from zip archives?

A bit of both - I'd say the main reason is that we'd like to encourage applications to avoid using shell scripts as it puts the burden on users (installs take more time, and zero installs don't work). So if you want to absolutely do it, better do it explicitly.

arcanis on 3 Feb 2020

Was this page helpful?

0 / 5 - 0 ratings