Shellcheck: Different SC results for the same sed script when quoted

Created on 28 May 2020  Β·  6Comments  Β·  Source: koalaman/shellcheck

For bugs

  • Rule Id (if any, e.g. SC1000): SC2039
  • My shellcheck version (shellcheck --version or "online"): 0.7.0
  • [ ] The rule's wiki page does not already cover this (e.g. https://shellcheck.net/wiki/SC2086)
  • [X] I tried on shellcheck.net and verified that this is still a problem on the latest commit

For new checks and feature suggestions

Here's a snippet or screenshot that shows the problem:

#!/bin/sh

# shell is dash

echo "0123456789ABCDEF" | sed s/[^0-9]/\ /g

echo "0123456789ABCDEF" | sed 's/[^0-9]/\ /g'

Here's what shellcheck currently says:

In ./sc-test.sh line 3:
echo "0123456789ABCDEF" | sed s/[^0-9]/\ /g
                                ^----^ SC2039: In POSIX sh, ^ in place of ! in glob bracket expressions is undefined.

Here's what I wanted or expected to see:

I do not understand why the quoted sed script does not trigger the same warning.

All 6 comments

Because the meta-characters lose their special meanings if they are quoted.

The meta-characters are β€œ!”, β€œ*”, β€œ?”, and β€œ[” in dash. A left bracket (β€œ[”) introduces a character class. The character class may be complemented by making an exclamation point ("!") the first character of the character class.

In bash you can use "^" in place of "!" to complement the character class.

See "Shell Patterns" section at https://www.man7.org/linux/man-pages/man1/dash.1.html and "Pattern Matching" section at https://www.man7.org/linux/man-pages/man1/bash.1.html for reference documentation.

Keep in mind that, if no matching filenames are found, and the shell option nullglob is not enabled, the word is left unchanged.

That is why unquoted s/[^0-9]/\ /g often works: because no matching filenames are found.

In bash you can use "^" in place of "!" to complement the character class.

An unexpected bashism .. Thanks for the tip!

After careful consideration, sed specifically uses regular-expressions not shell pattern matching, so, does that make this example a false positive ?

Before sending the string you are in shell land, at that point in time it is a shell pattern matching - the outcome of that is unknown. When you take the string and send it to sed you have left shell land and entered sed land, at that point in time it is a regular expression string.
By adding the quotes you will instruct the shell that it is a literal string and not a shell pattern.

By adding the quotes you will instruct the shell that it is a literal string and not a shell pattern.

Thanks for clarifying that point to me.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

bje- picture bje-  Β·  3Comments

hugovk picture hugovk  Β·  4Comments

quchen picture quchen  Β·  3Comments

ghost picture ghost  Β·  4Comments

koalaman picture koalaman  Β·  4Comments