Dietpi: Coding | Performance tests

[MichaIng]

As I step over different coding solutions for the same result, I thought it might be worth doing some benchmarks:

Testsystem: DietPi_VirtualBox_x86_64-Stretch v6.2

grep -q vs. cat | grep -ci -m1

root@DietPi:~# time for ((i=0;i<10000;i++)); do if grep -q 'DEV_GITBRANCH=testing' /DietPi/dietpi.txt; then ((i++)); fi; done
real    0m5.607s
user    0m0.392s
sys     0m0.856s
root@DietPi:~# time for ((i=0;i<10000;i++)); do if (( $(cat /DietPi/dietpi.txt | grep -ci -m1 'DEV_GITBRANCH=testing') )); then ((i++)); fi; done
real    0m14.784s
user    0m0.632s
sys     0m0.740s

• grep -q is clear winner here, where I think the reason is, that it does not necessarily read (and write) the whole file content (like cat, although the match was at the end of the file for this test) first

root@DietPi:~# time for ((i=0;i<1000;i++)); do if (( $(dpkg --get-selections | grep -ci -m1 'tzdata') )); then ((i++)); fi; done
real    0m6.019s
user    0m3.216s
sys     0m0.956s
root@DietPi:~# time for ((i=0;i<1000;i++)); do if (( $(dpkg --get-selections | grep -ci -m1 'tzdata') )); then ((i++)); fi; done
real    0m5.958s
user    0m3.124s
sys     0m1.044s
root@DietPi:~# time for ((i=0;i<1000;i++)); do if dpkg --get-selections | grep -q 'tzdata'; then ((i++)); fi; done
real    0m5.563s
user    0m3.104s
sys     0m1.132s
root@DietPi:~# time for ((i=0;i<1000;i++)); do if dpkg --get-selections | grep -q 'tzdata'; then ((i++)); fi; done
real    0m5.621s
user    0m3.064s
sys     0m1.176s
root@DietPi:~# time for ((i=0;i<1000;i++)); do if (( $(dpkg --get-selections | grep -q 'tzdata') )); then ((i++)); fi; done
real    0m12.008s
user    0m6.168s
sys     0m2.148s
root@DietPi:~# time for ((i=0;i<1000;i++)); do if [ dpkg --get-selections | grep -q 'tzdata'; then ((i++)) ]; fi; done
-bash: syntax error near unexpected token `]'
root@DietPi:~# time for ((i=0;i<1000;i++)); do if [[ dpkg --get-selections | grep -q 'tzdata'; then ((i++)) ]]; fi; done
-bash: conditional binary operator expected
-bash: syntax error near `--get-selections'
root@DietPi:~# time for ((i=0;i<1000;i++)); do if ( dpkg --get-selections | grep -q 'tzdata'; then ((i++)) ); fi; done
-bash: syntax error near unexpected token `then'

• For non-file checks, the difference is marginal.
• But you cannot use grep -q inside any kind of brackets/parenthesis, it also does not give any valid numeric result: (( grep -q )) results in skipping ((i++)), thus ~double time.

---

...

[Fourdee]

@MichaIng

Great writeup 👍

root@DietPi:~# time for ((i=0;i<1000;i++)); do if dpkg --get-selections | grep -q 'tzdata'; then ((i++)); fi; done
real 0m5.621s
user 0m3.064s
sys 0m1.176s
root@DietPi:~# time for ((i=0;i<1000;i++)); do if (( $(dpkg --get-selections | grep -q 'tzdata') )); then ((i++)); fi; done
real 0m12.008s
user 0m6.168s
sys 0m2.148s

Interesting. cat with grep is useless, as it spawns an additional pipe where grep can read the file directly (which I did not know at the start of DietPi, and, quickly became a bad habit). You'll probably notice i've been removing cat for our greps, every time I see one during coding :)

I'll do some RPi tests, should be interesting.

[MichaIng]

@Fourdee
Jep, I will also adjust this then in our code, where I find it.

There were also some other examples, where I thought, that benchmarks could be interesting, but I can't remember any more for now, which was also the reason I opened this issue for collecting.

[Fourdee]

@MichaIng

RPi 3:

cat << _EOF_ > test1
#!/bin/bash
TEST=0
for ((i=0;i<1000;i++))
do

    if (( \$(grep -ci -m1 'DEV_GITBRANCH=testing' /DietPi/dietpi.txt ) )); then

        ((TEST++))

    fi

done
echo \$TEST
_EOF_
chmod +x test1
time ./test1

cat << _EOF_ > test2
#!/bin/bash
TEST=0
for ((i=0;i<1000;i++))
do

    if grep -q 'DEV_GITBRANCH=testing' /DietPi/dietpi.txt; then

        ((TEST++))

    fi

done
echo \$TEST
_EOF_
chmod +x test2
time ./test2

---

Results -ci -m1:

root@DietPi:~# time ./test1
1000

real    0m8.026s
user    0m0.410s
sys     0m0.330s

🈯️ Results -q:

root@DietPi:~# time ./test2
1000

real    0m3.894s
user    0m0.140s
sys     0m0.390s

😲 grep -q it is!

[MichaIng]

@Fourdee
Wow clearer winner, than I tought. I just don't like too much, that there seems to be no brackets/parenthesis allow around grep -q... Not tested { } though. It makes the code look a bid "unstructured" or something, don't find the right words 😆. Love to have the "question" clearly separated, also if doing something with (( test )) && do // [ test ] && do...

[Fourdee]

@MichaIng

I just don't like too much, that there seems to be no brackets/parenthesis allow around grep -q... Not tested { } though. It makes the code look a bid "unstructured" or something,

I 100% agree, feel same, raw entry without () or [] feels wrong, python, "naughty" 😃

We'll have to play around and see if we can make it more "standardized" to fit our coding expectations.

[MichaIng]

@Fourdee
Add empty files/assure existence

• Not that it takes much time anyway, but yeah still interesting:

root@VM-Stretch:~# time for ((i=0;i<50000;i++)); do touch test; done

real    0m36.639s
user    0m2.476s
sys     0m6.300s
root@VM-Stretch:~# time for ((i=0;i<50000;i++)); do echo '' > test; done

real    0m7.646s
user    0m5.452s
sys     0m1.424s
root@VM-Stretch:~# time for ((i=0;i<50000;i++)); do echo '' > test; done

real    0m7.691s
user    0m5.328s
sys     0m1.608s
root@VM-Stretch:~# time for ((i=0;i<50000;i++)); do echo "" > test; done

real    0m7.567s
user    0m5.564s
sys     0m1.248s
root@VM-Stretch:~# time for ((i=0;i<50000;i++)); do echo "" > test; done

real    0m7.600s
user    0m5.532s
sys     0m1.276s
root@VM-Stretch:~# time for ((i=0;i<50000;i++)); do echo -e "" > test; done

real    0m7.646s
user    0m5.808s
sys      @0m0.936s
root@VM-Stretch:~# time for ((i=0;i<50000;i++)); do echo -n '' > test; done

real    0m1.323s
user    0m0.724s
sys     0m0.592s
root@VM-Stretch:~# time for ((i=0;i<50000;i++)); do echo -n '' > test; done

real    0m1.308s
user    0m0.708s
sys     0m0.592s
root@VM-Stretch:~# time for ((i=0;i<50000;i++)); do echo -n > test; done

real    0m1.240s
user    0m0.612s
sys     0m0.624s
root@VM-Stretch:~# time for ((i=0;i<50000;i++)); do echo -n > test; done

real    0m1.234s
user    0m0.676s
sys     0m0.548s
root@VM-Stretch:~# time for ((i=0;i<50000;i++)); do > test; done

real    0m1.067s
user    0m0.548s
sys     0m0.516s
root@VM-Stretch:~# time for ((i=0;i<50000;i++)); do > test; done

real    0m1.074s
user    0m0.484s
sys     0m0.584s

• I thought, touch should be the fastest command to create an empty file, as it does not "change" the file. But it is extremely the opposite way. I found a good explanation for this: https://unix.stackexchange.com/a/123850 => echo is bash builtin!
• echo [-e] ''/"" all are the same. For consistency I would prefer single quotes, but:
• It adds a newline to the file as well, echo -n '' is several times faster again.
• It is not even necessary to quote the empty string, which saves again a bid.
• Best was to just use > file instead of using echo at all 😆.

[MichaIng]

root@VM-Stretch:~# time for ((i=0;i<1000;i++)); do if (( $(dpkg --get-selections | grep -ci -m1 'hwinfo') )); then > /dev/null; fi done

real    0m8.730s
user    0m6.756s
sys     0m1.922s
root@VM-Stretch:~# time for ((i=0;i<1000;i++)); do if (( $(dpkg --get-selections | grep -ci -m1 'hwinfo') )); then > /dev/null; fi done

real    0m8.730s
user    0m6.888s
sys     0m1.790s
root@VM-Stretch:~# time for ((i=0;i<1000;i++)); do if dpkg --get-selections | grep -q 'hwinfo'; then > /dev/null; fi done

real    0m8.024s
user    0m6.091s
sys     0m1.893s
root@VM-Stretch:~# time for ((i=0;i<1000;i++)); do if dpkg --get-selections | grep -q 'hwinfo'; then > /dev/null; fi done

real    0m7.988s
user    0m6.172s
sys     0m1.776s

• Minimal, but there is a difference 😄.

[MichaIng]

grep file vs variable

grep here string:

root@DietPi:~# cat test
#!/bin/bash
list="$(dpkg --get-selections | awk '{print $1}')"

time for ((i=0;i<10000;i++)); do

        if grep -q '^dropbear' <<< "$list"; then

                > /dev/null

        fi

done
root@DietPi:~# ./test

real    0m11.648s
user    0m0.432s
sys     0m1.076s
root@DietPi:~# ./test

real    0m11.736s
user    0m0.300s
sys     0m1.184s

grep RAM disk file:

root@DietPi:~# cat test
#!/bin/bash
#list="$(dpkg --get-selections | awk '{print $1}')"
dpkg --get-selections | awk '{print $1}' > /tmp/testing

time for ((i=0;i<10000;i++)); do

        if grep -q '^dropbear' /tmp/testing; then

                > /dev/null

        fi

done

rm /tmp/testing
root@DietPi:~# ./test

real    0m9.295s
user    0m0.632s
sys     0m0.824s
root@DietPi:~# ./test

real    0m9.355s
user    0m0.620s
sys     0m0.880s

echo grep

root@DietPi:~# cat test
#!/bin/bash
list="$(dpkg --get-selections | awk '{print $1}')"
#dpkg --get-selections | awk '{print $1}' > /tmp/testing

time for ((i=0;i<10000;i++)); do

        if echo "$list" | grep -q '^dropbear'; then

                > /dev/null

        fi

done

#rm /tmp/testing
root@DietPi:~# ./test

real    0m15.618s
user    0m0.700s
sys     0m1.588s
root@DietPi:~# ./test

real    0m15.703s
user    0m0.652s
sys     0m1.600s

Interesting, if searching a long string, it is more efficient to grep it from RAM (!!) disk file, instead of echo or here string, I guess because in the latter cases the whole string must be handled/written again every loop, where the file only needs to be written one time, just read afterwards. I tested with wput as well, to see the influence of where grep -q stops reading, but the results were exactly the same!

Test with shorter string:

root@DietPi:~# cat test
#!/bin/bash
list="$(df)"
#dpkg --get-selections | awk '{print $1}' > /tmp/testing

time for ((i=0;i<10000;i++)); do

        if grep -q 'DietPi' <<< "$list"; then

                > /dev/null

        fi

done

#rm /tmp/testing
root@DietPi:~# ./test

real    0m10.569s
user    0m0.364s
sys     0m1.084s
root@DietPi:~# ./test

real    0m10.700s
user    0m0.548s
sys     0m0.976s
root@DietPi:~# nano test
root@DietPi:~# cat test
#!/bin/bash
#list="$(df)"
df > /tmp/testing

time for ((i=0;i<10000;i++)); do

        if grep -q 'DietPi' /tmp/testing; then

                > /dev/null

        fi

done

#rm /tmp/testing
root@DietPi:~# ./test

real    0m9.277s
user    0m0.620s
sys     0m0.848s
root@DietPi:~# ./test

real    0m9.291s
user    0m0.600s
sys     0m0.872s

The length of the string has minor influence.

Influence of writing/removing the file:

root@DietPi:~# cat test
#!/bin/bash
#list="$(df)"
df > /tmp/testing

for ((i=0;i<10;i++)); do

        if grep -q 'DietPi' /tmp/testing; then

                > /dev/null

        fi

done

#rm /tmp/testing
root@DietPi:~# time for ((i=0;i<1000;i++)); do ./test; done

real    0m11.965s
user    0m0.000s
sys     0m0.228s
root@DietPi:~# time for ((i=0;i<1000;i++)); do ./test; done

real    0m12.504s
user    0m0.096s
sys     0m0.244s
root@DietPi:~# time for ((i=0;i<1000;i++)); do ./test; done

real    0m11.955s
user    0m0.060s
sys     0m0.172s
root@DietPi:~# nano test
root@DietPi:~# cat test
#!/bin/bash
list="$(df)"
#df > /tmp/testing

for ((i=0;i<10;i++)); do

        if grep -q 'DietPi' <<< "$list"; then

                > /dev/null

        fi

done

#rm /tmp/testing
root@DietPi:~# time for ((i=0;i<1000;i++)); do ./test; done

real    0m13.370s
user    0m0.060s
sys     0m0.172s
root@DietPi:~# time for ((i=0;i<1000;i++)); do ./test; done

real    0m13.417s
user    0m0.072s
sys     0m0.160s
root@DietPi:~# nano test
root@DietPi:~# cat test
#!/bin/bash
#list="$(df)"
df > /tmp/testing

for ((i=0;i<10;i++)); do

        if grep -q 'DietPi' /tmp/testing; then

                > /dev/null

        fi

done

rm /tmp/testing
root@DietPi:~# time for ((i=0;i<1000;i++)); do ./test; done

real    0m13.063s
user    0m0.124s
sys     0m0.232s
root@DietPi:~# time for ((i=0;i<1000;i++)); do ./test; done

real    0m12.680s
user    0m0.072s
sys     0m0.168s
root@DietPi:~# time for ((i=0;i<1000;i++)); do ./test; done

real    0m12.696s
user    0m0.072s
sys     0m0.164s

Final results:

• Very interesting! Writing a file to ramdisk (/tmp) seems to be a quite efficient alternative to creating a variable, even that the file needs to be created and removed again.
• At least if doing grep on it 10 times or more, the faster grep on file compared to string outweighs the effort of creating and removing the tmp file.
• If grep is done just one time, it should be preferred to directly grep the output of a command and skip creating a variable/file completely.
• If grep less often then 10 times, it seems to be a question of taste and beauty of code, if tmp file or variable is used.
• If a variable is chosen, then use "here string" instead of echo+grep: grep -q "$pattern" <<< "$var" Ref: https://en.wikipedia.org/wiki/Here_document#Here_strings
• €: Uhh, grep should be totally skipped when just comparing/checking variables:

root@DietPi:~# echo $TEST
abc
root@DietPi:~# grep -q 'abc' <<< "$TEST" && time for ((i=0;i<1000;i++)); do grep -q 'abc' <<< "$TEST" && :; done

real    0m4.798s
user    0m1.296s
sys     0m2.020s
root@DietPi:~# grep -q 'abc' <<< "$TEST" && time for ((i=0;i<1000;i++)); do grep -q 'abc' <<< "$TEST" && :; done

real    0m4.360s
user    0m0.748s
sys     0m0.972s
root@DietPi:~# [[ $TEST =~ abc ]] && time for ((i=0;i<1000;i++)); do [[ $TEST =~ abc ]] && :; done

real    0m0.113s
user    0m0.116s
sys     0m0.000s
root@DietPi:~# [[ $TEST =~ abc ]] && time for ((i=0;i<1000;i++)); do [[ $TEST =~ abc ]] && :; done

real    0m0.124s
user    0m0.124s
sys     0m0.000s

• Even piping command outputs into grep is multiple times slower than compare it within double brackets:

root@DietPi:~# echo 'abc' | grep -q 'abc' && time for ((i=0;i<1000;i++)); do echo 'abc' | grep -q 'abc' && :; done

real    0m6.113s
user    0m0.376s
sys     0m1.028s
root@DietPi:~# echo 'abc' | grep -q 'abc' && time for ((i=0;i<1000;i++)); do echo 'abc' | grep -q 'abc' && :; done

real    0m6.418s
user    0m0.388s
sys     0m1.160s
root@DietPi:~# [[ $(echo 'abc') =~ abc ]] && time for ((i=0;i<1000;i++)); do [[ $(echo 'abc') =~ abc ]] && :; done

real    0m2.452s
user    0m0.244s
sys     0m0.500s
root@DietPi:~# [[ $(echo 'abc') =~ abc ]] && time for ((i=0;i<1000;i++)); do [[ $(echo 'abc') =~ abc ]] && :; done

real    0m2.384s
user    0m0.292s
sys     0m0.432s

• Same for scanning files:

root@DietPi:~# grep -q 'abc' TEST && time for ((i=0;i<10000;i++)); do grep -q 'abc' TEST && :; done

real    0m42.166s
user    0m2.576s
sys     0m5.280s
root@DietPi:~# [[ $(<TEST) =~ abc ]] && time for ((i=0;i<10000;i++)); do [[ $(<TEST) =~ abc ]] && :; done

real    0m21.589s
user    0m2.336s
sys     0m4.508s

• Test with larger files:

root@DietPi:~# grep -q 'restrict ::1' /etc/ntp.conf && time for ((i=0;i<1000;i++)); do grep -q 'restrict ::1' /etc/ntp.conf && :; done

real    0m4.104s
user    0m0.208s
sys     0m0.532s
root@DietPi:~# grep -q 'restrict ::1' /etc/ntp.conf && time for ((i=0;i<1000;i++)); do grep -q 'restrict ::1' /etc/ntp.conf && :; done

real    0m4.268s
user    0m0.364s
sys     0m0.464s
root@DietPi:~# [[ $(</etc/ntp.conf) =~ 'restrict ::1' ]] && time for ((i=0;i<1000;i++)); do [[ $(</etc/ntp.conf) =~ 'restrict ::1' ]] && :; done

real    0m3.093s
user    0m0.596s
sys     0m0.700s
root@DietPi:~# [[ $(</etc/ntp.conf) =~ 'restrict ::1' ]] && time for ((i=0;i<1000;i++)); do [[ $(</etc/ntp.conf) =~ 'restrict ::1' ]] && :; done

real    0m2.876s
user    0m0.612s
sys     0m0.580s

=> Less difference, as the file reading itself seems to take most of the time, but the final string comparison seems to be still faster than grep -q.

• I tested and compared single brackets [ ] as well, and = / =~. Generally if it's about comparing strings, double brackets [[ ]] seem to be marginal faster, double quoting " " of variables is not needed, same as within arithmetic (( )), it clearly defines a string "environment", and it allows all the extended regex features. I even found [ -n/-z "$var"] to be slower:

root@DietPi:~# [ -n "$TEST" ] && time for ((i=0;i<10000;i++)); do [ -n "$TEST" ] && :; done

real    0m0.889s
user    0m0.888s
sys     0m0.000s
root@DietPi:~# [ -n "$TEST" ] && time for ((i=0;i<10000;i++)); do [ -n "$TEST" ] && :; done

real    0m0.903s
user    0m0.904s
sys     0m0.000s
root@DietPi:~# [[ $TEST != '' ]] && time for ((i=0;i<10000;i++)); do [[ $TEST != '' ]] && :; done

real    0m0.678s
user    0m0.676s
sys     0m0.004s
root@DietPi:~# [[ $TEST != '' ]] && time for ((i=0;i<10000;i++)); do [[ $TEST != '' ]] && :; done

real    0m0.667s
user    0m0.668s
sys     0m0.000s

• Finally if it's about string comparison, no matter if coming from variable or command output or file, [[ ]] should be used in every case! = for equality and =~ for containing. One has to take care, that wildcards are handled differently here:

root@DietPi:~# [[ abc = ab? ]] && echo yes
yes
root@DietPi:~# [[ abc = abc? ]] && echo yes
root@DietPi:~# [[ abc =~ abc? ]] && echo yes
yes
root@DietPi:~# [[ abc =~ abcc? ]] && echo yes
yes
root@DietPi:~# [[ abc =~ abccc? ]] && echo yes
root@DietPi:~# [[ abc = abc? ]] && echo yes
root@DietPi:~# ^C
root@DietPi:~# [[ abc = ab ]] && echo yes
root@DietPi:~# [[ abc =~ ab ]] && echo yes
yes
root@DietPi:~# [[ abc == ab? ]] && echo yes
yes
root@DietPi:~# [[ abc == abc? ]] && echo yes

=> With = the wildcards stand for additional characters, where with =~ they are related to the character they're following. Same with *... Thus =~ ab. equals = ab? while =~ ab? equals = a? and =~ ab.* equals = ab* etc.

[MichaIng]

@Fourdee

return value + function return check vs. echo value + arithmetic result check

root@DietPi:~# cat testscript
#!/bin/bash

returnfunction(){ return 0; }

echofunction(){ echo 0; }

time for ((i=0;i<10000;i++));
do
        returnfunction && echo 0 &> /dev/null
done

time for ((i=0;i<10000;i++));
do
        (( $(echofunction) == 0 )) && echo 0 &> /dev/null
done
root@DietPi:~# ./testscript

real    0m0.339s
user    0m0.236s
sys     0m0.100s

real    0m5.196s
user    0m0.732s
sys     0m1.012s
root@DietPi:~# ./testscript

real    0m0.335s
user    0m0.248s
sys     0m0.088s

real    0m5.252s
user    0m0.724s
sys     0m1.048s

• Clear winner is using return for checks, e.g. G_CHECK_FREESPACE/G_CHECK_VALIDINT, also it doesn't matter is echo/return 1 or 0 and check for for yes/no, I verified, that both functions do the && echo 0 😉.
• Again with both negative results, thus no && echo 0

root@DietPi:~# cat testscript
#!/bin/bash

returnfunction(){ return 1; }

echofunction(){ echo 1; }

time for ((i=0;i<10000;i++));
do
        returnfunction && echo 0 &> /dev/null
done

time for ((i=0;i<10000;i++));
do
        (( $(echofunction) == 0 )) && echo 0 &> /dev/null
done
root@DietPi:~# ./testscript

real    0m0.144s
user    0m0.144s
sys     0m0.000s

real    0m4.785s
user    0m0.548s
sys     0m0.836s

• Looks like the arithmetic check itself takes so much more time than the return check, as missing second echo 0 has no large relative effect on the second method.

[Fourdee]

@MichaIng

Clear winner is using return for checks

Nice one 👍. I'll try to keep that in mind 😃

[MichaIng]

Float comparison

Ugly manual solution vs bc

Let's see how efficient bc compares float values:

root@VM-Stretch:~# echo "$G_WHIP_RETURNED_VALUE"
0.5
root@VM-Stretch:~# time for ((i=0;i<1000;i++)); do (( $(bc -l <<< "$G_WHIP_RETURNED_VALUE <= 50") )) && :; done

real    0m6.629s
user    0m1.388s
sys     0m1.904s
root@VM-Stretch:~# time for ((i=0;i<1000;i++)); do (( $(bc -l <<< "$G_WHIP_RETURNED_VALUE <= 50") )) && :; done

real    0m6.999s
user    0m1.692s
sys     0m2.376s
root@VM-Stretch:~# time for ((i=0;i<1000;i++)); do (( $(bc -l <<< "$G_WHIP_RETURNED_VALUE <= 50") )) && :; done

real    0m7.259s
user    0m1.916s
sys     0m2.336s
root@VM-Stretch:~# time for ((i=0;i<1000;i++)); do (( $(bc -l <<< "$G_WHIP_RETURNED_VALUE <= 50") )) && :; done

real    0m6.131s
user    0m1.120s
sys     0m1.336s
root@VM-Stretch:~# time for ((i=0;i<1000;i++)); do [ "$G_WHIP_RETURNED_VALUE" = '50' ] || (( $(sed 's/\..*$//' <<< "$G_WHIP_RETURNED_VALUE") < 50 )) && :; done

real    0m7.104s
user    0m1.464s
sys     0m2.328s
root@VM-Stretch:~# time for ((i=0;i<1000;i++)); do [ "$G_WHIP_RETURNED_VALUE" = '50' ] || (( $(sed 's/\..*$//' <<< "$G_WHIP_RETURNED_VALUE") < 50 )) && :; done

real    0m7.380s
user    0m1.628s
sys     0m2.724s
root@VM-Stretch:~# time for ((i=0;i<1000;i++)); do [ "$G_WHIP_RETURNED_VALUE" = '50' ] || (( $(sed 's/\..*$//' <<< "$G_WHIP_RETURNED_VALUE") < 50 )) && :; done

real    0m7.093s
user    0m1.388s
sys     0m2.468s
root@VM-Stretch:~# time for ((i=0;i<1000;i++)); do [ "$G_WHIP_RETURNED_VALUE" = '50' ] || (( $(sed 's/\..*$//' <<< "$G_WHIP_RETURNED_VALUE") < 50 )) && :; done

real    0m7.297s
user    0m1.708s
sys     0m2.712s

=> Jep, bc does it quite efficient, I like this calculator. Comparison with integers:

root@VM-Stretch:~# echo $G_WHIP_RETURNED_VALUE
1
root@VM-Stretch:~# time for ((i=0;i<1000;i++)); do (( $G_WHIP_RETURNED_VALUE <= 50 )) && :; done

real    0m0.069s
user    0m0.068s
sys     0m0.000s
root@VM-Stretch:~# time for ((i=0;i<1000;i++)); do (( $(bc -l <<< "$G_WHIP_RETURNED_VALUE <= 50") )) && :; done

real    0m7.046s
user    0m1.656s
sys     0m2.092s

=> Uii with integers, bash internals are wayyyyy more effective: If we need to do much calculation, we should try to stay with integers, is what I get from here. If floats are needed, bc does good clean job.
Sadly bc cannot be used to combine comparison with valid float check:

root@VM-Stretch:~# bc -l <<< 'jkshfk > 10'
0
root@VM-Stretch:~# bc -l <<< 'jkshfk < 10'
1
root@VM-Stretch:~# bc -l <<< 'jkshfk == 0'
1

An invalid float will be just converted to zero 😢.

[MichaIng]

pgrep vs. ps aux | grep -q

• Sadly there is no quiet check option for pgrep, anyway it's faster:

2018-04-30 13:15:47 root@micha:/var/log# time for ((i=0;i<100;i++)); do pgrep 'apache' &> /dev/null; done

real    0m2.286s
user    0m0.890s
sys     0m1.427s
2018-04-30 13:16:01 root@micha:/var/log# time for ((i=0;i<100;i++)); do pgrep 'apache' &> /dev/null; done

real    0m2.278s
user    0m0.744s
sys     0m1.563s
2018-04-30 13:16:04 root@micha:/var/log# time for ((i=0;i<100;i++)); do pgrep 'apache' &> /dev/null; done

real    0m2.288s
user    0m0.628s
sys     0m1.689s
2018-04-30 13:16:07 root@micha:/var/log# time for ((i=0;i<100;i++)); do ps aux | grep -q 'apache'; done

real    0m4.188s
user    0m1.885s
sys     0m3.060s
2018-04-30 13:16:26 root@micha:/var/log# time for ((i=0;i<100;i++)); do ps aux | grep -q 'apache'; done

real    0m4.185s
user    0m2.010s
sys     0m2.937s
2018-04-30 13:16:31 root@micha:/var/log# time for ((i=0;i<100;i++)); do ps aux | grep -q 'apache'; done

real    0m4.194s
user    0m2.009s
sys     0m2.941s

[Fourdee]

@MichaIng

pgrep vs. ps aux | grep -q

Very nice, basically x2 performance 👍

[MichaIng]

@Fourdee
🤣 Found grep -q to be slower in any case compared to [[ ]], where we finally just want to check/compare strings. Should be used only, if it's about isolating certain lines from string/command/file:
NB: Finally the overall difference of using this or that of course will be in most cases unnoticed by end user, so no need to actively change. But very interesting to keep in mind.
€: Okay, [[ $VAR =~ ... ]] handles everything as single line, also "" does not help there, as it does for e.g. echo. Thus this is only useful if we do not need to check for ^ or $.

root@DietPi:~# echo $TEST
abc
root@DietPi:~# grep -q 'abc' <<< "$TEST" && time for ((i=0;i<1000;i++)); do grep -q 'abc' <<< "$TEST" && :; done

real    0m4.798s
user    0m1.296s
sys     0m2.020s
root@DietPi:~# grep -q 'abc' <<< "$TEST" && time for ((i=0;i<1000;i++)); do grep -q 'abc' <<< "$TEST" && :; done

real    0m4.360s
user    0m0.748s
sys     0m0.972s
root@DietPi:~# [[ $TEST =~ abc ]] && time for ((i=0;i<1000;i++)); do [[ $TEST =~ abc ]] && :; done

real    0m0.113s
user    0m0.116s
sys     0m0.000s
root@DietPi:~# [[ $TEST =~ abc ]] && time for ((i=0;i<1000;i++)); do [[ $TEST =~ abc ]] && :; done

real    0m0.124s
user    0m0.124s
sys     0m0.000s

• Even piping command outputs into grep is multiple times slower than compare it within double brackets:

root@DietPi:~# echo 'abc' | grep -q 'abc' && time for ((i=0;i<1000;i++)); do echo 'abc' | grep -q 'abc' && :; done

real    0m6.113s
user    0m0.376s
sys     0m1.028s
root@DietPi:~# echo 'abc' | grep -q 'abc' && time for ((i=0;i<1000;i++)); do echo 'abc' | grep -q 'abc' && :; done

real    0m6.418s
user    0m0.388s
sys     0m1.160s
root@DietPi:~# [[ $(echo 'abc') =~ abc ]] && time for ((i=0;i<1000;i++)); do [[ $(echo 'abc') =~ abc ]] && :; done

real    0m2.452s
user    0m0.244s
sys     0m0.500s
root@DietPi:~# [[ $(echo 'abc') =~ abc ]] && time for ((i=0;i<1000;i++)); do [[ $(echo 'abc') =~ abc ]] && :; done

real    0m2.384s
user    0m0.292s
sys     0m0.432s

• Same for scanning files:
  €: $(<file) leads to single line string, also double quotes do not help, thus line start end ending cannot be used. => not useful to scan files! Back to grep here!

root@DietPi:~# grep -q 'abc' TEST && time for ((i=0;i<10000;i++)); do grep -q 'abc' TEST && :; done

real    0m42.166s
user    0m2.576s
sys     0m5.280s
root@DietPi:~# [[ $(<TEST) =~ abc ]] && time for ((i=0;i<10000;i++)); do [[ $(<TEST) =~ abc ]] && :; done

real    0m21.589s
user    0m2.336s
sys     0m4.508s

• Test with larger files:

root@DietPi:~# grep -q 'restrict ::1' /etc/ntp.conf && time for ((i=0;i<1000;i++)); do grep -q 'restrict ::1' /etc/ntp.conf && :; done

real    0m4.104s
user    0m0.208s
sys     0m0.532s
root@DietPi:~# grep -q 'restrict ::1' /etc/ntp.conf && time for ((i=0;i<1000;i++)); do grep -q 'restrict ::1' /etc/ntp.conf && :; done

real    0m4.268s
user    0m0.364s
sys     0m0.464s
root@DietPi:~# [[ $(</etc/ntp.conf) =~ 'restrict ::1' ]] && time for ((i=0;i<1000;i++)); do [[ $(</etc/ntp.conf) =~ 'restrict ::1' ]] && :; done

real    0m3.093s
user    0m0.596s
sys     0m0.700s
root@DietPi:~# [[ $(</etc/ntp.conf) =~ 'restrict ::1' ]] && time for ((i=0;i<1000;i++)); do [[ $(</etc/ntp.conf) =~ 'restrict ::1' ]] && :; done

real    0m2.876s
user    0m0.612s
sys     0m0.580s

=> Less difference, as the file reading itself seems to take most of the time, but the final string comparison seems to be still faster than grep -q.

• I tested and compared single brackets [ ] as well, and = / =~. Generally if it's about comparing strings, double brackets [[ ]] seem to be marginal faster, double quoting " " of variables is not needed, same as within arithmetic (( )), it clearly defines a string "environment", and it allows all the extended regex features. I even found [ -n/-z "$var"] to be slower:

root@DietPi:~# [ -n "$TEST" ] && time for ((i=0;i<10000;i++)); do [ -n "$TEST" ] && :; done

real    0m0.889s
user    0m0.888s
sys     0m0.000s
root@DietPi:~# [ -n "$TEST" ] && time for ((i=0;i<10000;i++)); do [ -n "$TEST" ] && :; done

real    0m0.903s
user    0m0.904s
sys     0m0.000s
root@DietPi:~# [[ $TEST != '' ]] && time for ((i=0;i<10000;i++)); do [[ $TEST != '' ]] && :; done

real    0m0.678s
user    0m0.676s
sys     0m0.004s
root@DietPi:~# [[ $TEST != '' ]] && time for ((i=0;i<10000;i++)); do [[ $TEST != '' ]] && :; done

real    0m0.667s
user    0m0.668s
sys     0m0.000s

• Finally if it's about string comparison, no matter if coming from variable or command output or file, [[ ]] should be used in every case! == for equality (or =, which behaves exactly the same, but double seems bash consistent) and =~ for containing. One has to take care, that wildcards are handled differently here:

root@DietPi:~# [[ abc = ab? ]] && echo yes
yes
root@DietPi:~# [[ abc = abc? ]] && echo yes
root@DietPi:~# [[ abc =~ abc? ]] && echo yes
yes
root@DietPi:~# [[ abc =~ abcc? ]] && echo yes
yes
root@DietPi:~# [[ abc =~ abccc? ]] && echo yes
root@DietPi:~# [[ abc = abc? ]] && echo yes
root@DietPi:~# ^C
root@DietPi:~# [[ abc = ab ]] && echo yes
root@DietPi:~# [[ abc =~ ab ]] && echo yes
yes
root@DietPi:~# [[ abc == ab? ]] && echo yes
yes
root@DietPi:~# [[ abc == abc? ]] && echo yes

=> With == the wildcards stand for additional characters, where with =~ they are related to the character they're following. Same with *... Thus =~ ab. equals == ab? while =~ ab? equals == a? and =~ ab.* equals == ab* etc.

• Ref. about the different brackets: https://serverfault.com/questions/52034/what-is-the-difference-between-double-and-single-square-brackets-in-bash?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa
  => As we anyway rely on bash and don't need portability and as the above advantages of clear string environment, we should stick to double brackets.

€: At least pgrep is still preferred to check for active processes 😉:

root@DietPi:~# pgrep 'dropbear' &> /dev/null && time for ((i=0;i<100;i++)); do pgrep 'dropbear' &> /dev/null && :; done

real    0m1.606s
user    0m0.400s
sys     0m0.904s
root@DietPi:~# pgrep 'dropbear' &> /dev/null && time for ((i=0;i<100;i++)); do pgrep 'dropbear' &> /dev/null && :; done

real    0m1.595s
user    0m0.524s
sys     0m0.772s
root@DietPi:~# [[ $(ps aux) =~ dropbear ]] && time for ((i=0;i<100;i++)); do [[ $(ps aux) =~ dropbear ]] && :; done

real    0m2.661s
user    0m1.032s
sys     0m1.536s
root@DietPi:~# [[ $(ps aux) =~ dropbear ]] && time for ((i=0;i<100;i++)); do [[ $(ps aux) =~ dropbear ]] && :; done

real    0m2.470s
user    0m1.000s
sys     0m1.384s

[MichaIng]

[[ ]] vs. [ ]

root@DietPi:~# [ -n "$(<TEST)" ] && time for ((i=1;i<1000;i++)); do [ -n "$(<TEST)" ] && :; done

real    0m2.123s
user    0m0.332s
sys     0m0.364s
root@DietPi:~# [ -n "$(<TEST)" ] && time for ((i=1;i<1000;i++)); do [ -n "$(<TEST)" ] && :; done

real    0m2.358s
user    0m0.332s
sys     0m0.452s
root@DietPi:~# [[ -n $(<TEST) ]] && time for ((i=1;i<1000;i++)); do [[ -n $(<TEST) ]] && :; done

real    0m2.060s
user    0m0.204s
sys     0m0.384s
root@DietPi:~# [[ -n $(<TEST) ]] && time for ((i=1;i<1000;i++)); do [[ -n $(<TEST) ]] && :; done

real    0m2.077s
user    0m0.208s
sys     0m0.384s
root@DietPi:~# [ -s TEST ] && time for ((i=1;i<1000;i++)); do [ -s TEST ] && :; done

real    0m0.087s
user    0m0.080s
sys     0m0.008s
root@DietPi:~# [ -s TEST ] && time for ((i=1;i<1000;i++)); do [ -s TEST ] && :; done

real    0m0.092s
user    0m0.092s
sys     0m0.000s
root@DietPi:~# [[ -s TEST ]] && time for ((i=1;i<1000;i++)); do [[ -s TEST ]] && :; done

real    0m0.064s
user    0m0.056s
sys     0m0.008s
root@DietPi:~# [[ -s TEST ]] && time for ((i=1;i<1000;i++)); do [[ -s TEST ]] && :; done

real    0m0.063s
user    0m0.064s
sys     0m0.000s

• As also shown above with string comparisons, double quotes seem to be generally handled slightly faster in bash. Also the advantage that ext. regex are allowed, && || within the brackets leads to that it might be worth thinking to strictly use double brackets within DietPi / bash-only scripts. Only for backwards compatibility of scripts, that users might want to use in other non bash environments/shells, single brackets have advantages.

[Fourdee]

@MichaIng

Also the advantage that ext. regex are allowed, && || within the brackets leads to that it might be worth thinking to strictly use double brackets within DietPi / bash-only scripts

Great find, yep, lets go with double brackets from now on? 👍
Might take me a while to adjust, got so used to [ ] lol 😃

[Fourdee]

$x is also faster than "$x":

root@DietPi:~# TEST='/DietPi/dietpi/login'; time for ((i=1;i<10000;i++)); do [[ -f "$TEST" ]] && :; done

real    0m0.436s
user    0m0.440s
sys     0m0.000s

root@DietPi:~# TEST='/DietPi/dietpi/login'; time for ((i=1;i<10000;i++)); do [[ -f $TEST ]] && :; done

real    0m0.383s
user    0m0.340s
sys     0m0.050s

Maybe we could $x for filepaths/strings that we know do not contain culprit chars?

[MichaIng]

@Fourdee
As said, within double brackets, double quotes are not needed, as string splitting is no problem here. It should always behave exactly the same with and without.
Interesting, that the difference is quite measurable. I think there really seems to be one additional (not needed or doubled) transformation done with "".

[MichaIng]

Reading file content to variable

$(cat /path/to/file) vs $(

2018-06-07 13:17:56 root@micha:~# time for ((i=0;i<100;i++)); do TEST=$(cat /etc/network/interfaces); done

real    0m0.668s
user    0m0.315s
sys     0m0.389s
2018-06-07 13:18:15 root@micha:~# time for ((i=0;i<100;i++)); do TEST=$(cat /etc/network/interfaces); done

real    0m0.671s
user    0m0.316s
sys     0m0.389s
2018-06-07 13:18:17 root@micha:~# echo "$TEST"
#/etc/network/interfaces
#Please use DietPi-Config to modify network settings.

# Local
auto lo
iface lo inet loopback

# Ethernet
allow-hotplug eth0
iface eth0 inet static
address 192.168.178.20
netmask 255.255.255.0
gateway 192.168.178.1
dns-nameservers 192.168.178.1

# Wifi
#allow-hotplug wlan0
iface wlan0 inet dhcp
address 192.168.0.100
netmask 255.255.255.0
gateway 192.168.0.1
wireless-power off
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
#dns-nameservers 8.8.8.8 8.8.4.4
2018-06-07 13:18:25 root@micha:~# time for ((i=0;i<100;i++)); do TEST=$(</etc/network/interfaces); done

real    0m0.445s
user    0m0.314s
sys     0m0.176s
2018-06-07 13:18:34 root@micha:~# time for ((i=0;i<100;i++)); do TEST=$(</etc/network/interfaces); done

real    0m0.449s
user    0m0.291s
sys     0m0.201s
2018-06-07 13:18:35 root@micha:~# echo "$TEST"
#/etc/network/interfaces
#Please use DietPi-Config to modify network settings.

# Local
auto lo
iface lo inet loopback

# Ethernet
allow-hotplug eth0
iface eth0 inet static
address 192.168.178.20
netmask 255.255.255.0
gateway 192.168.178.1
dns-nameservers 192.168.178.1

# Wifi
#allow-hotplug wlan0
iface wlan0 inet dhcp
address 192.168.0.100
netmask 255.255.255.0
gateway 192.168.0.1
wireless-power off
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
#dns-nameservers 8.8.8.8 8.8.4.4

Scanning file for string (one liner or no line start/ending check)

grep -q vs [[ $(

2018-06-07 13:22:29 root@micha:~# [[ $(</etc/network/interfaces) =~ 'wireless' ]] && time for ((i=0;i<100;i++)); do [[ $(</etc/network/interfaces) =~ 'wireless' ]] && :; done

real    0m0.500s
user    0m0.316s
sys     0m0.276s
2018-06-07 13:22:48 root@micha:~# [[ $(</etc/network/interfaces) =~ 'wireless' ]] && time for ((i=0;i<100;i++)); do [[ $(</etc/network/interfaces) =~ 'wireless' ]] && :; done

real    0m0.503s
user    0m0.324s
sys     0m0.272s
2018-06-07 13:22:49 root@micha:~# grep -q 'wireless' /etc/network/interfaces && time for ((i=0;i<100;i++)); do grep -q 'wireless' /etc/network/interfaces && :; done

real    0m0.734s
user    0m0.409s
sys     0m0.376s
2018-06-07 13:23:23 root@micha:~# grep -q 'wireless' /etc/network/interfaces && time for ((i=0;i<100;i++)); do grep -q 'wireless' /etc/network/interfaces && :; done

real    0m0.736s
user    0m0.403s
sys     0m0.383s

• Double brackets are faster than grep, already tested before actually, but double brackets cannot check for line start and ending! Just the very first line and end of file can be found via ^/$.

2018-06-07 13:23:24 root@micha:~# [[ $(</etc/network/interfaces) =~ '^wireless' ]] && time for ((i=0;i<100;i++)); do [[ $(</etc/network/interfaces) =~ '^wireless' ]] && :; done
2018-06-07 13:23:40 root@micha:~# [[ $(</etc/network/interfaces) =~ '\nwireless' ]] && time for ((i=0;i<100;i++)); do [[ $(</etc/network/interfaces) =~ '\nwireless' ]] && :; done
2018-06-07 13:23:54 root@micha:~# [[ $(</etc/network/interfaces) =~ \nwireless ]] && time for ((i=0;i<100;i++)); do [[ $(</etc/network/interfaces) =~ \nwireless ]] && :; done
2018-06-07 13:24:09 root@micha:~# [[ $(</etc/network/interfaces) =~ ^wireless ]] && time for ((i=0;i<100;i++)); do [[ $(</etc/network/interfaces) =~ ^wireless ]] && :; done
2018-06-07 13:24:37 root@micha:~# [[ "$(</etc/network/interfaces)" =~ ^wireless ]] && time for ((i=0;i<100;i++)); do [[ "$(</etc/network/interfaces)" =~ ^wireless ]] && :; done
2018-06-07 13:24:53 root@micha:~# [[ "$(</etc/network/interfaces)" =~ "^wireless" ]] && time for ((i=0;i<100;i++)); do [[ "$(</etc/network/interfaces)" =~ "^wireless" ]] && :; done
2018-06-07 13:25:06 root@micha:~# [[ "$(</etc/network/interfaces)" =~ "\nwireless" ]] && time for ((i=0;i<100;i++)); do [[ "$(</etc/network/interfaces)" =~ "\nwireless" ]] && :; done

[MichaIng]

Alternative to sed for cutting oneline strings: https://stackoverflow.com/a/19482947

• http://wiki.bash-hackers.org/syntax/pe

root@VM-Stretch:~# INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP ') && echo $INDEX
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000\ link/ether 08:00:27:25:10:84 brd ff:ff:ff:ff:ff:ff
root@VM-Stretch:~# INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UPN/A ') && echo $INDEX
root@VM-Stretch:~# INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP ' | sed -e 's/^.*eth\([0-9]\):.*$/\1/') && echo $INDEX
0
root@VM-Stretch:~# INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UPN/A ' | sed -e 's/^.*eth\([0-9]\):.*$/\1/') && echo $INDEX

root@VM-Stretch:~#

Just to verify:

• Variables can be set within conditional statements.
• grep -m1 correctly returns true and false based on match.
• sed does not, breaks condition check of grep, thus need to be used afterwards.
• But bash direct variable manipulation need to be done after conditional check as well.

root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=$(sed -e 's/^.*eth\([0-9]\):.*$/\1/' <<< $INDEX); fi; done && echo $INDEX

real    0m3.173s
user    0m1.052s
sys     0m0.936s
0
root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=$(sed -e 's/^.*eth\([0-9]\):.*$/\1/' <<< $INDEX); fi; done && echo $INDEX

real    0m2.843s
user    0m0.952s
sys     0m0.668s
0
root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=$(sed -e 's/^.*eth\([0-9]\):.*$/\1/' <<< $INDEX); fi; done && echo $INDEX

real    0m3.072s
user    0m0.948s
sys     0m0.932s
0

root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=${INDEX#* eth} && INDEX=${INDEX%%: *}; fi; done && echo $INDEX

real    0m1.821s
user    0m0.608s
sys     0m0.536s
0
root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=${INDEX#* eth} && INDEX=${INDEX%%: *}; fi; done && echo $INDEX

real    0m1.570s
user    0m0.464s
sys     0m0.404s
0
root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=${INDEX#* eth} && INDEX=${INDEX%%: *}; fi; done && echo $INDEX

real    0m1.593s
user    0m0.532s
sys     0m0.356s
0



md5-ce210b8560f12e56f7caa23c3d5907c2



root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=$(sed 's/^.* eth//' <<< $INDEX | sed 's/: .*$//'); fi; done && echo $INDEX

real    0m3.501s
user    0m1.084s
sys     0m1.088s
0
root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=$(sed 's/^.* eth//' <<< $INDEX | sed 's/: .*$//'); fi; done && echo $INDEX

real    0m3.430s
user    0m1.080s
sys     0m0.972s
0
root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=$(sed 's/^.* eth//' <<< $INDEX | sed 's/: .*$//'); fi; done && echo $INDEX

real    0m3.884s
user    0m1.148s
sys     0m1.384s
0



md5-dac7df4740ce636a25e607cda6ecf8c7



root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=$(awk '{print $2}' <<< $INDEX); fi; done && echo $INDEX

real    0m2.188s
user    0m0.600s
sys     0m0.344s
eth0:
root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=$(awk '{print $2}' <<< $INDEX); fi; done && echo $INDEX

real    0m2.156s
user    0m0.588s
sys     0m0.360s
eth0:
root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=$(awk '{print $2}' <<< $INDEX); fi; done && echo $INDEX

real    0m2.245s
user    0m0.624s
sys     0m0.348s
eth0:
root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=${INDEX/* eth/eth} && INDEX=${INDEX//: */:}; fi; done && echo $INDEX

real    0m1.385s
user    0m0.480s
sys     0m0.232s
eth0:
root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=${INDEX/* eth/eth} && INDEX=${INDEX//: */:}; fi; done && echo $INDEX

real    0m1.410s
user    0m0.476s
sys     0m0.248s
eth0:
root@VM-Stretch:~# time for ((i=0;i<100;i++)); do if INDEX=$(ip -o l | grep -m1 'eth[0-9]: .* state UP '); then INDEX=${INDEX/* eth/eth} && INDEX=${INDEX//: */:}; fi; done && echo $INDEX

real    0m1.394s
user    0m0.448s
sys     0m0.252s
eth0:

• Bash string manipulation is of course inflexible in comparison, as it does not allow special wild cards, fields or 2nd variables, AFAIK.
• Also it handles multiple lines like one, doesn't understand line beginning and ending, this multiple line output. Thus a grepis always needed at first.

[MichaIng]

Avoiding sub shells ( commands... ), as well for checking conditions, where possible, as they take a while to initiate:

2018-07-24 19:39:49 root@DietPi:/tmp# time for ((i=0;i<1000;i++)); do [[ $NOT_EXISTENT ]] || ( : && : ); done

real    0m2.471s
user    0m0.124s
sys     0m0.592s
2018-07-24 19:41:33 root@DietPi:/tmp# time for ((i=0;i<1000;i++)); do [[ $NOT_EXISTENT ]] || ( : && : ); done

real    0m2.502s
user    0m0.120s
sys     0m0.588s
2018-07-24 19:42:08 root@DietPi:/tmp# time for ((i=0;i<1000;i++)); do if [[ $NOT_EXISTENT ]]; then :; else : && :; fi done

real    0m0.099s
user    0m0.100s
sys     0m0.000s
2018-07-24 19:42:22 root@DietPi:/tmp# time for ((i=0;i<1000;i++)); do if [[ $NOT_EXISTENT ]]; then :; else : && :; fi done

real    0m0.084s
user    0m0.084s
sys     0m0.000s
2018-07-24 19:43:07 root@DietPi:/tmp# time for ((i=0;i<1000;i++)); do if [[ $NOT_EXISTENT ]]; then :; elif ( : && : ); then :; fi done

real    0m2.503s
user    0m0.064s
sys     0m0.676s
2018-07-24 19:43:12 root@DietPi:/tmp# time for ((i=0;i<1000;i++)); do if [[ $NOT_EXISTENT ]]; then :; elif ( : && : ); then :; fi done

real    0m2.522s
user    0m0.072s
sys     0m0.668s

[MichaIng]

Using or skipping single quotes

2018-07-29 15:57:35 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test=1; done

real    0m5.319s
user    0m5.320s
sys     0m0.000s
2018-07-29 16:15:54 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test=1; done

real    0m5.360s
user    0m5.364s
sys     0m0.000s
2018-07-29 16:16:01 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test='1'; done

real    0m5.669s
user    0m5.664s
sys     0m0.004s
2018-07-29 16:16:11 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test='1'; done

real    0m5.985s
user    0m5.976s
sys     0m0.008s
2018-07-29 16:16:19 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test='1'; done

real    0m5.890s
user    0m5.888s
sys     0m0.000s
2018-07-29 16:16:26 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test=1; done

real    0m5.463s
user    0m5.464s
sys     0m0.000s

Multiple chars

2018-07-29 16:18:26 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test='abcdefg'; done

real    0m5.938s
user    0m5.940s
sys     0m0.000s
2018-07-29 16:18:34 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test='abcdefg'; done

real    0m5.845s
user    0m5.844s
sys     0m0.000s
2018-07-29 16:18:41 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test='abcdefg'; done

real    0m6.437s
user    0m6.416s
sys     0m0.020s
2018-07-29 16:18:48 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test='abcdefg'; done

real    0m6.271s
user    0m6.268s
sys     0m0.004s
2018-07-29 16:18:56 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test='abcdefg'; done

real    0m5.748s
user    0m5.748s
sys     0m0.000s
2018-07-29 16:19:04 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test=abcdefg; done

real    0m5.681s
user    0m5.668s
sys     0m0.012s
2018-07-29 16:19:14 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test=abcdefg; done

real    0m5.856s
user    0m5.844s
sys     0m0.012s
2018-07-29 16:19:21 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test=abcdefg; done

real    0m6.095s
user    0m6.096s
sys     0m0.000s
2018-07-29 16:19:28 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test=abcdefg; done

real    0m6.062s
user    0m6.060s
sys     0m0.000s
2018-07-29 16:19:36 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test=abcdefg; done

real    0m5.668s
user    0m5.668s
sys     0m0.004s
2018-07-29 16:19:45 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test='abcdefg'; done

real    0m6.246s
user    0m6.248s
sys     0m0.000s
2018-07-29 16:19:56 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test='abcdefg'; done

real    0m6.040s
user    0m6.036s
sys     0m0.004s
2018-07-29 16:20:03 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test='abcdefg'; done

real    0m6.450s
user    0m6.440s
sys     0m0.008s
2018-07-29 16:20:10 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test='abcdefg'; done

real    0m5.987s
user    0m5.984s
sys     0m0.000s
2018-07-29 16:20:18 root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do test='abcdefg'; done

real    0m6.071s
user    0m6.064s
sys     0m0.008s

Hard to see a difference, at least it is tiny. Theoretically quotes are only needed in case of spaces within string/value, but I think it is good practice to use use them, single quotes, if $ should be expended, single quotes otherwise/to explicitly take everything literally.

[MichaIng]

dpkg --get-selections vs. dpkg-query -s

2018-08-02 22:17:23 root@VM-Stretch:~# time for ((i=0;i<100;i++)); do dpkg --get-selections | grep -q 'unzip'; done

real    0m8.882s
user    0m7.448s
sys     0m1.828s
2018-08-02 22:17:43 root@VM-Stretch:~# time for ((i=0;i<100;i++)); do dpkg --get-selections | grep -q 'unzip'; done

real    0m8.431s
user    0m6.956s
sys     0m1.816s
2018-08-02 22:17:52 root@VM-Stretch:~# time for ((i=0;i<100;i++)); do dpkg --get-selections | grep -q 'unzip'; done

real    0m8.362s
user    0m7.056s
sys     0m1.600s
2018-08-02 22:18:03 root@VM-Stretch:~# time for ((i=0;i<100;i++)); do dpkg --get-selections &> /dev/null; done

real    0m8.077s
user    0m6.724s
sys     0m1.156s
2018-08-02 22:18:32 root@VM-Stretch:~# time for ((i=0;i<100;i++)); do dpkg --get-selections &> /dev/null; done

real    0m8.274s
user    0m6.968s
sys     0m1.108s
2018-08-02 22:18:41 root@VM-Stretch:~# time for ((i=0;i<100;i++)); do dpkg --get-selections &> /dev/null; done

real    0m8.040s
user    0m6.816s
sys     0m1.040s
2018-08-02 22:18:49 root@VM-Stretch:~# time for ((i=0;i<100;i++)); do dpkg-query -s unzip &> /dev/null; done

real    0m7.114s
user    0m5.972s
sys     0m0.944s
2018-08-02 22:19:03 root@VM-Stretch:~# time for ((i=0;i<100;i++)); do dpkg-query -s unzip &> /dev/null; done

real    0m6.658s
user    0m5.552s
sys     0m0.912s
2018-08-02 22:19:11 root@VM-Stretch:~# time for ((i=0;i<100;i++)); do dpkg-query -s unzip &> /dev/null; done

real    0m6.856s
user    0m5.836s
sys     0m0.816s
2018-08-02 22:19:18 root@VM-Stretch:~# time for ((i=0;i<100;i++)); do dpkg --get-selections | grep -q 'unzip'; done

real    0m8.569s
user    0m7.252s
sys     0m1.644s

dpkg-query -s <package> is just slightly faster, but no grep needed to check for installed package. We should replace this.

[MichaIng]

How to scrape a specific line

2018-08-30 02:53:39 root@micha:/tmp# time for ((i=0;i<1000;i++)); do l | sed -n '4p' &> /dev/null; done

real    0m12.624s
user    0m6.676s
sys     0m14.066s
2018-08-30 02:54:01 root@micha:/tmp# time for ((i=0;i<1000;i++)); do l | sed -n '4p' &> /dev/null; done

real    0m12.462s
user    0m6.703s
sys     0m13.797s
2018-08-30 02:54:17 root@micha:/tmp# time for ((i=0;i<1000;i++)); do l | awk 'NR==4' &> /dev/null; done

real    0m14.365s
user    0m8.705s
sys     0m16.489s
2018-08-30 02:54:49 root@micha:/tmp# time for ((i=0;i<1000;i++)); do l | awk 'NR==4' &> /dev/null; done

real    0m14.387s
user    0m8.846s
sys     0m16.455s
2018-08-30 02:55:05 root@micha:/tmp# time for ((i=0;i<1000;i++)); do l | head -n 4 | tail -n 1 &> /dev/null; done

real    0m12.992s
user    0m11.136s
sys     0m14.562s
2018-08-30 02:58:02 root@micha:/tmp# time for ((i=0;i<1000;i++)); do l | head -n 4 | tail -n 1 &> /dev/null; done

real    0m13.094s
user    0m10.808s
sys     0m15.148s
2018-08-30 03:00:54 root@micha:/tmp# time for ((i=0;i<1000;i++)); do l | tail -n +4 | head -n 1 &> /dev/null; done

real    0m13.019s
user    0m11.239s
sys     0m14.615s
2018-08-30 03:01:23 root@micha:/tmp# time for ((i=0;i<1000;i++)); do l | tail -n +4 | head -n 1 &> /dev/null; done

real    0m12.976s
user    0m10.912s
sys     0m14.822s
2018-08-30 03:01:38 root@micha:/tmp# l
total 0
drwxr-xr-x 3 root root 80 Aug 28 12:28 apt
drwxrwxrwt 2 root root 40 Aug 15 00:15 .font-unix
drwxrwxrwt 2 root root 40 Aug 15 00:15 .ICE-unix
drwx------ 3 root root 60 Aug 23 18:40 systemd-private-0df235a06e2242cd9a8075b6ee8614f1-apache2.service-dMmgQL
drwx------ 3 root root 60 Aug 23 18:40 systemd-private-0df235a06e2242cd9a8075b6ee8614f1-redis-server.service-m1SHYb
drwxrwxrwt 2 root root 40 Aug 15 00:15 .Test-unix
drwxrwxrwt 2 root root 40 Aug 15 00:15 .X11-unix
drwxrwxrwt 2 root root 40 Aug 15 00:15 .XIM-unix

• sed -n '[0-9]p', which we already use, is also slightly fastest, awk 'NR==[0-9]' as often slowest, but not much difference overall.... although...

2018-08-30 03:03:27 root@micha:/tmp# time for ((i=0;i<1000;i++)); do l &> /dev/null; done

real    0m11.956s
user    0m3.360s
sys     0m9.101s

... very large difference, since 12s are just the l (my alias for ls -lA) 😄.

[MichaIng]

cat vs echo

2018-09-09 16:52:14 root@micha:/tmp# cat test
#!/bin/bash

test_cat(){
        cat << _EOF_ > /dev/null
test1
test2
test3
_EOF_
}

test_echo(){
        echo -e 'test1\ntest2\ntest3' > /dev/null
}

time for ((i=0;i<100;i++)); do
        test_cat
done

time for ((i=0;i<100;i++)); do
        test_echo
done
2018-09-09 16:52:16 root@micha:/tmp# ./test

real    0m0.706s
user    0m0.386s
sys     0m0.346s

real    0m0.024s
user    0m0.024s
sys     0m0.000s
2018-09-09 16:52:22 root@micha:/tmp# ./test

real    0m0.691s
user    0m0.302s
sys     0m0.418s

real    0m0.027s
user    0m0.021s
sys     0m0.005s

• I guess it's due to echo being a bash builtin, which cat is not:

2018-09-09 17:07:16 root@micha:/tmp# builtin echo

2018-09-09 17:07:24 root@micha:/tmp# builtin cat
bash: builtin: cat: not a shell builtin

• Since echo can be used with \n and as well just with a line break within the code, I don't see any advantage of using cat at all:

2018-09-09 17:14:19 root@micha:/tmp# cat test
#!/bin/bash

echo -e 'First line
second line
third line
...
we do not need cat for this :D!' > testfile
2018-09-09 17:14:21 root@micha:/tmp# ./test
2018-09-09 17:14:24 root@micha:/tmp# cat testfile
First line
second line
third line
...
we do not need cat for this :D!

• Hmm, since when does escaping single quote \' not work at all in echo [-e] '...' 🤔.

[Fourdee]

@MichaIng

Intresting find 👍

For me, I prefer using cat for the simplicity of easy markings/viewing in notepad++

[MichaIng]

@Fourdee
Jep, slight benefit of cat that the output lines between cat << _EOF_ and _EOF_ will be taken one-to-one, where with echo first and last line contain the command/forwarding as well.

Finally since usually cat is used just once each script call to create a config or service file e.g., it should not matter. Within dietpi-drive_manager I thought it makes sense to replace it, as it is called several times in loops and each time one navigates through the menus.

• Different topic, but I also though about reducing the amount of drive init loops, e.g. doing drive init just once on first access (refresh button is there in case) and when editing a certain drive, just refresh this one drive info. Also service handling could be moved to the steps, where it is really needed (format userdata/rootfs/swap drive, userdata/rootfs/swap move and such), instead of start/stop on menu directly. E.g. when user wants to add a new drive to a production server, the downtime could be quite a hurdle.

[Fourdee]

@MichaIng

but I also though about reducing the amount of drive init loops, e.g. doing drive init just once on first access (refresh button is there in case) and when editing a certain drive, just refresh this one drive info. Also service handling could be moved to the steps, where it is really needed (format userdata/rootfs/swap drive, userdata/rootfs/swap move and such), instead of start/stop on menu directly. E.g. when user wants to add a new drive to a production server, the downtime could be quite a hurdle.

Agree, could do with some improvements 👍

[MichaIng]

for loop c-style vs. bash-style

c-style: for ((i=a;i<=b;i=i+c))

bash-style: for i in {a..b..c} / for i in {a..b}, default c=1

root@VM-Stretch:~# time for i in {0..100000}; do :; done

real    0m0.874s
user    0m0.868s
sys     0m0.004s
root@VM-Stretch:~# time for i in {0..100000}; do :; done

real    0m0.855s
user    0m0.856s
sys     0m0.000s
root@VM-Stretch:~# time for i in {0..100000}; do :; done

real    0m0.890s
user    0m0.888s
sys     0m0.000s
root@VM-Stretch:~# time for i in {0..100000..1}; do :; done

real    0m1.387s
user    0m1.312s
sys     0m0.072s
root@VM-Stretch:~# time for i in {0..100000..1}; do :; done

real    0m1.289s
user    0m1.284s
sys     0m0.000s
root@VM-Stretch:~# time for i in {0..100000..1}; do :; done

real    0m1.190s
user    0m1.188s
sys     0m0.000s
root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do :; done

real    0m2.308s
user    0m2.304s
sys     0m0.000s
root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do :; done

real    0m2.310s
user    0m2.308s
sys     0m0.000s
root@VM-Stretch:~# time for ((i=0;i<100000;i++)); do :; done

real    0m2.600s
user    0m2.600s
sys     0m0.000s

@Fourdee
Clearly bash-style is faster, although the loop itself takes nearly no time (100,000 loops!), so totally marginal and personal coding preferences are totally okay.
But you know [[ $ME == 'perfectionist' ]], so I hope it's okay that I implement the bash-style loop into our code when touching code 😉.

[Fourdee]

@MichaIng

Clearly bash-style is faster

Dammit! 🤣

But you know [[ $ME == 'perfectionist' ]], so I hope it's okay that I implement the bash-style loop into our code when touching code 😉.

Yep, lets go for it 👍 Might take me a while to adjust, but i'll try 😄

[Fourdee]

@MichaIng

for i in {0..100000..1}; do :; done

How would you do a <=100000, or is that default?

[MichaIng]

@Fourdee
It's <=100000 by this, ..... So for <100000 it should be 0..99999 or for same loop count 1..100000

Ah yeah the c-style loop above made one loop less 😉.

Might take me a while to adjust, but i'll try 😄

Jep no problem, since the overall time a loop up-count takes is anyway in 10 to 100 nano seconds area, this is nothing urgent 😄.

[MichaIng]

Index loop vs. loop through array/files

# Create array with 10000 entries
root@VM-Stretch:~# for i in {0..9999}; do aTEST[$i]="value$i"; done
# Simple loop through 0 to 9999
root@VM-Stretch:~# time for i in {0..9999}; do :; done

real    0m0.089s
user    0m0.088s
sys     0m0.000s
root@VM-Stretch:~# time for i in {0..9999}; do :; done

real    0m0.097s
user    0m0.096s
sys     0m0.000s
root@VM-Stretch:~# time for i in {0..9999}; do :; done

real    0m0.094s
user    0m0.096s
sys     0m0.000s
# Loop through 10000 array indices
root@VM-Stretch:~# time for i in ${!aTEST[@]}; do :; done

real    0m0.124s
user    0m0.124s
sys     0m0.000s
root@VM-Stretch:~# time for i in ${!aTEST[@]}; do :; done

real    0m0.118s
user    0m0.116s
sys     0m0.000s
root@VM-Stretch:~# time for i in ${!aTEST[@]}; do :; done

real    0m0.125s
user    0m0.124s
sys     0m0.000s
# Loop through 10000 array values
root@VM-Stretch:/tmp/test# time for i in ${aTEST[@]}; do :; done

real    0m0.142s
user    0m0.144s
sys     0m0.000s
root@VM-Stretch:/tmp/test# time for i in ${aTEST[@]}; do :; done

real    0m0.157s
user    0m0.156s
sys     0m0.000s
root@VM-Stretch:/tmp/test# time for i in ${aTEST[@]}; do :; done

real    0m0.153s
user    0m0.152s
sys     0m0.000s
# Create 10000 files (in RAM!)
root@VM-Stretch:/tmp/test# mkdir /tmp/test
root@VM-Stretch:/tmp/test# cd /tmp/test
root@VM-Stretch:/tmp/test# for i in {0..9999}; do > $i; done
# Loop through 10000 files
root@VM-Stretch:/tmp/test# time for i in /tmp/test/*; do :; done

real    0m0.152s
user    0m0.152s
sys     0m0.000s
root@VM-Stretch:/tmp/test# time for i in /tmp/test/*; do :; done

real    0m0.145s
user    0m0.148s
sys     0m0.000s
root@VM-Stretch:/tmp/test# time for i in /tmp/test/*; do :; done

real    0m0.145s
user    0m0.144s
sys     0m0.000s

• Simply counting up the integers is fastest, but not that much faster than looping through an array or through files directly. c-style loop above is even slower than all of them 😄. My guess is the reason is init of arithmetic environment.
• I did this, as we often check for arrays or files via index loop and check/read array value/key or file name based on the index, e.g. dietpi-software array loops, obtain_network_details interface loops.
• Generally it makes more sense to directly loop through existing array entries/files instead. This e.g. marks the need to fully initiate the array obsolete and in case of files as well does not need any max number estimation/declaration. Especially when looping through files, often way less do actually exist than we need to check via index loop, so the amount of loops can be drastically reduced. And of course without a max number, it is even more fail safe, if for some reason e.g. eth10 interface exists.
• In case of looping through files, if non matching file exists, the pattern will be taken as string. Easiest solution is [[ -e $i ]] || break I guess.
• Same intention was with this PR actually: https://github.com/Fourdee/DietPi/pull/2111/files

[MichaIng]

NB: for i in {0..99} does mit take variables for the Intervall, so in most cases c-style loops are still needed ;).

---

https://www.linuxtopia.org/online_books/advanced_bash_scripting_guide/string-manipulation.html
Several string manipulation methods that prevent the use of sed/awk, this should be faster.

[MichaIng]

mawk vs gawk

_Ref: https://github.com/Fourdee/DietPi/issues/2323_

Large files

mawk

root@VM-Stretch:~# time for i in {1..1000}; do mawk '{print $1}' /DietPi/dietpi.txt &> /dev/null; done

real    0m5.043s
user    0m2.000s
sys     0m2.784s
root@VM-Stretch:~# time for i in {1..1000}; do mawk '{print $1}' /DietPi/dietpi.txt &> /dev/null; done

real    0m5.035s
user    0m1.976s
sys     0m2.800s
root@VM-Stretch:~# time for i in {1..1000}; do mawk '{print $1}' /DietPi/dietpi.txt &> /dev/null; done

real    0m5.090s
user    0m2.004s
sys     0m2.784s

gawk

root@VM-Stretch:~# time for i in {1..1000}; do gawk '{print $1}' /DietPi/dietpi.txt &> /dev/null; done

real    0m10.289s
user    0m4.688s
sys     0m4.204s
root@VM-Stretch:~# time for i in {1..1000}; do gawk '{print $1}' /DietPi/dietpi.txt &> /dev/null; done

real    0m10.254s
user    0m4.592s
sys     0m4.260s
root@VM-Stretch:~# time for i in {1..1000}; do gawk '{print $1}' /DietPi/dietpi.txt &> /dev/null; done

real    0m10.303s
user    0m4.632s
sys     0m4.260s

Small files

mawk

root@VM-Stretch:~# time for i in {1..1000}; do mawk '{print $1}' /DietPi/dietpi/.install_stage &> /dev/null; done

real    0m4.293s
user    0m0.156s
sys     0m0.724s
root@VM-Stretch:~# time for i in {1..1000}; do mawk '{print $1}' /DietPi/dietpi/.install_stage &> /dev/null; done

real    0m4.296s
user    0m0.208s
sys     0m0.704s
root@VM-Stretch:~# time for i in {1..1000}; do mawk '{print $1}' /DietPi/dietpi/.install_stage &> /dev/null; done

real    0m4.331s
user    0m0.204s
sys     0m0.736s

gawk

root@VM-Stretch:~# time for i in {1..1000}; do gawk '{print $1}' /DietPi/dietpi/.install_stage &> /dev/null; done

real    0m9.231s
user    0m4.108s
sys     0m4.708s
root@VM-Stretch:~# time for i in {1..1000}; do gawk '{print $1}' /DietPi/dietpi/.install_stage &> /dev/null; done

real    0m9.208s
user    0m4.080s
sys     0m4.752s
root@VM-Stretch:~# time for i in {1..1000}; do gawk '{print $1}' /DietPi/dietpi/.install_stage &> /dev/null; done

real    0m9.230s
user    0m3.988s
sys     0m4.860s

Result

• gawk takes 5 ms more init time that mawk

Large input string

mawk

root@VM-Stretch:~# time for i in {1..1000}; do mawk '{print $1}' <<< "$test" &> /dev/null; done

real    0m11.703s
user    0m6.480s
sys     0m2.368s
root@VM-Stretch:~# time for i in {1..1000}; do mawk '{print $1}' <<< "$test" &> /dev/null; done

real    0m11.702s
user    0m6.460s
sys     0m2.392s
root@VM-Stretch:~# time for i in {1..1000}; do mawk '{print $1}' <<< "$test" &> /dev/null; done

real    0m11.827s
user    0m6.704s
sys     0m2.276s

gawk

root@VM-Stretch:~# time for i in {1..1000}; do gawk '{print $1}' <<< "$test" &> /dev/null; done

real    0m17.230s
user    0m11.200s
sys     0m5.592s
root@VM-Stretch:~# time for i in {1..1000}; do gawk '{print $1}' <<< "$test" &> /dev/null; done

real    0m17.202s
user    0m11.200s
sys     0m5.564s
root@VM-Stretch:~# time for i in {1..1000}; do gawk '{print $1}' <<< "$test" &> /dev/null; done

real    0m17.184s
user    0m11.272s
sys     0m5.472s

Result

• Even larger absolute difference here.

[MichaIng]

Another subshell vs command assembly on sh (dash) vs bash

root@VM-Stretch:/tmp# time sh -c 'i=0; while [ $i -lt 1000 ]; do i=$((i+1)); ( : ); done'

real    0m0.779s
user    0m0.092s
sys     0m0.168s
root@VM-Stretch:/tmp# time sh -c 'i=0; while [ $i -lt 1000 ]; do i=$((i+1)); ( : ); done'

real    0m0.844s
user    0m0.036s
sys     0m0.252s
root@VM-Stretch:/tmp# time sh -c 'i=0; while [ $i -lt 1000 ]; do i=$((i+1)); { :; }; done'

real    0m0.031s
user    0m0.028s
sys     0m0.000s
root@VM-Stretch:/tmp# time sh -c 'i=0; while [ $i -lt 1000 ]; do i=$((i+1)); { :; }; done'

real    0m0.030s
user    0m0.024s
sys     0m0.008s
root@VM-Stretch:/tmp# time bash -c 'i=0; while [ $i -lt 1000 ]; do i=$((i+1)); ( : ); done'

real    0m1.559s
user    0m0.144s
sys     0m0.304s
root@VM-Stretch:/tmp# time bash -c 'i=0; while [ $i -lt 1000 ]; do i=$((i+1)); ( : ); done'

real    0m1.538s
user    0m0.136s
sys     0m0.344s
root@VM-Stretch:/tmp# time bash -c 'i=0; while [ $i -lt 1000 ]; do i=$((i+1)); { :; }; done'

real    0m0.097s
user    0m0.092s
sys     0m0.000s
root@VM-Stretch:/tmp# time bash -c 'i=0; while [ $i -lt 1000 ]; do i=$((i+1)); { :; }; done'

real    0m0.109s
user    0m0.104s
sys     0m0.004s

Little benefit, when we use bash syntax:

root@VM-Stretch:/tmp# time bash -c 'for i in {0..999}; do ( : ); done'

real    0m1.513s
user    0m0.052s
sys     0m0.308s
root@VM-Stretch:/tmp# time bash -c 'for i in {0..999}; do ( : ); done'

real    0m1.539s

user    0m0.072s
sys     0m0.312s
root@VM-Stretch:/tmp# time bash -c 'for i in {0..999}; do { :; }; done'

real    0m0.032s
user    0m0.028s
sys     0m0.000s
root@VM-Stretch:/tmp# time bash -c 'for i in {0..999}; do { :; }; done'

real    0m0.033s
user    0m0.028s
sys     0m0.004s

However, just wanted to verify that cron should use:
25 1 * * * root test -x /usr/sbin/anacron || { cd / && run-parts --report /etc/cron.daily; }
instead of
25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
even that it uses /bin/sh.

[MichaIng]

Testing better solutions to get config values

• Currently we use timezone=$(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt | sed 's/^[^=]*=//') to get values from even shell syntax conform config files.
• These are two external commands already and it is not 100% safe, e.g. if spaces or in-line comments are present.
• An alternative I found is timezone=$(sed -n '/^[[:blank:]]*AUTO_SETUP_TIMEZONE=/{s/^[^=]*=//p;q}' /DietPi/dietpi.txt). But this is not (much) faster, I guess since sed always loops through the whole file before stopping output after first match.
• In one case we use another | awk '{print $1}' to allow comments, but this does still not allow multi word arguments and adds another very heavy external command. Parsing quotes for multi word values makes everything worse.
• The best methods, at least in terms of performance, are actually to use pure shell/bash syntax. Bash source handles all variable assignment, respecting quotes for multi word values and same line comments, as we want. So the basic idea I had is using grep only to find the setting/line(s) we want and source it to current shell. Of course this means that the variable has the same name as within the config file, but for consistency this is actually not too bad anyway.

Command substitution

2019-01-12 15:55:04 root@micha:/tmp# $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt)
bash: AUTO_SETUP_TIMEZONE=Europe/Berlin: No such file or directory
2019-01-12 15:57:06 root@micha:/tmp# `grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt`
bash: AUTO_SETUP_TIMEZONE=Europe/Berlin: No such file or directory
2019-01-12 15:57:17 root@micha:/tmp# test=$(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt)
2019-01-12 15:57:33 root@micha:/tmp# $test
bash: AUTO_SETUP_TIMEZONE=Europe/Berlin: No such file or directory
2019-01-12 15:59:53 root@micha:/tmp# test='echo success'
2019-01-12 16:00:13 root@micha:/tmp# $test
success

• 🈴 This works for commands (with arguments), but not for bash syntax => declarations

STDIN to source

2019-01-12 14:51:51 root@micha:/tmp# grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt | source
bash: source: filename argument required
source: usage: source filename [arguments]
2019-01-12 14:53:13 root@micha:/tmp# source <<< $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt)
bash: source: filename argument required
source: usage: source filename [arguments]

• 🈴 Piping and here strings do not work.

2019-01-12 14:56:08 root@micha:/tmp# echo $AUTO_SETUP_TIMEZONE

2019-01-12 14:56:09 root@micha:/tmp# . /dev/stdin <<< $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt)
2019-01-12 14:56:11 root@micha:/tmp# echo $AUTO_SETUP_TIMEZONE
Europe/Berlin

• 🈯️ Workaround sourcing STDIN works fine

xargs

2019-01-12 15:14:35 root@micha:/tmp# echo 'echo test' | xargs xargs
test
2019-01-12 15:13:23 root@micha:/tmp# grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt | xargs xargs
xargs: AUTO_SETUP_TIMEZONE=Europe/Berlin: No such file or directory
2019-01-12 15:14:19 root@micha:/tmp# grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt | xargs source
xargs: source: No such file or directory
2019-01-12 15:14:24 root@micha:/tmp# grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt | xargs .
xargs: .: Permission denied

• Generally works to even get the command from the pipe, not just it's arguments.
• 🈴 But it does need a command and no bash syntax. Declaring variables, even using source, does not work, which is basically the same issue as above trying STDIN to source above. It simply does not take a command/declaration as input, but the input really needs to be a "real" file.

eval

2019-01-12 14:50:20 root@micha:/tmp# echo $AUTO_SETUP_TIMEZONE

2019-01-12 14:51:03 root@micha:/tmp# eval "$(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt)"
2019-01-12 14:51:11 root@micha:/tmp# echo $AUTO_SETUP_TIMEZONE
Europe/Berlin

• 🈯️ Variables are available in main shell, so eval does not execute stuff in a sub shell.

Process substitution: <(command)

Ref: http://tldp.org/LDP/abs/html/process-sub.html

2019-01-12 16:04:35 root@micha:/tmp# echo $AUTO_SETUP_TIMEZONE

2019-01-12 16:04:36 root@micha:/tmp# source <(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt)
2019-01-12 16:04:38 root@micha:/tmp# echo $AUTO_SETUP_TIMEZONE
Europe/Berlin
2019-01-12 16:16:13 root@micha:/tmp# unset AUTO_SETUP_TIMEZONE
2019-01-12 16:18:16 root@micha:/tmp# . <(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt)
2019-01-12 16:18:23 root@micha:/tmp# echo $AUTO_SETUP_TIMEZONE
Europe/Berlin

• 🈯️ Works perfectly

2019-01-12 16:04:43 root@micha:/tmp# echo <(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt)
/dev/fd/63
2019-01-12 16:05:11 root@micha:/tmp# cat <(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE= /DietPi/dietpi.txt)
AUTO_SETUP_TIMEZONE=Europe/Berlin
2019-01-12 16:05:20 root@micha:/tmp# <(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt)
bash: /dev/fd/63: Permission denied

• Process substitution creates a temp file in /dev/fd/ (similar to here string/document) with the commands content and places the path to that file as argument.
• So it's a mixture of here string (creating an actual temp file) and command substitution (placing output directly into shell as argument/text, instead of passing to STDIN).
• So this is great for all commands, that do not read STDIN, but require input from a file via argument (like source).

local variables

2019-01-12 16:20:18 root@micha:/tmp# cat script
#!/bin/bash

assign(){

        local $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE' /DietPi/dietpi.txt)
        echo $AUTO_SETUP_TIMEZONE

}

assign
2019-01-12 16:20:47 root@micha:/tmp# ./script
Europe/Berlin

• 🈯️ In case of local variables, where local is a command, command substitution of course works.

---

About performance

. /dev/stdin <<< $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt)

root@VM-Stretch:~# time for i in {1..100}; do . /dev/stdin <<< $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.709s
user    0m0.244s
sys     0m0.272s
root@VM-Stretch:~# time for i in {1..100}; do . /dev/stdin <<< $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.714s
user    0m0.248s
sys     0m0.268s
root@VM-Stretch:~# time for i in {1..100}; do . /dev/stdin <<< $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.720s
user    0m0.276s
sys     0m0.244s
root@VM-Stretch:~# time for i in {1..100}; do . /dev/stdin <<< $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.720s
user    0m0.244s
sys     0m0.272s
root@VM-Stretch:~# time for i in {1..100}; do . /dev/stdin <<< $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.711s
user    0m0.244s
sys     0m0.264s

eval "$(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt)"

root@VM-Stretch:~# time for i in {1..100}; do eval $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.697s
user    0m0.220s
sys     0m0.284s
root@VM-Stretch:~# time for i in {1..100}; do eval $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.700s
user    0m0.268s
sys     0m0.232s
root@VM-Stretch:~# time for i in {1..100}; do eval $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.705s
user    0m0.228s
sys     0m0.276s
root@VM-Stretch:~# time for i in {1..100}; do eval $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.702s
user    0m0.224s
sys     0m0.276s
root@VM-Stretch:~# time for i in {1..100}; do eval $(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.700s
user    0m0.232s
sys     0m0.272s

. <(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt)

root@VM-Stretch:~# time for i in {1..100}; do . <(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.871s
user    0m0.196s
sys     0m0.316s
root@VM-Stretch:~# time for i in {1..100}; do . <(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.859s
user    0m0.256s
sys     0m0.256s
root@VM-Stretch:~# time for i in {1..100}; do . <(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.852s
user    0m0.244s
sys     0m0.264s
root@VM-Stretch:~# time for i in {1..100}; do . <(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.863s
user    0m0.244s
sys     0m0.276s
root@VM-Stretch:~# time for i in {1..100}; do . <(grep -m1 '^[[:blank:]]*AUTO_SETUP_TIMEZONE=' /DietPi/dietpi.txt); done

real    0m0.861s
user    0m0.260s
sys     0m0.252s

• eval is fastest here, sadly not the shortest process substution

Sourcing whole file

root@VM-Stretch:~# time for i in {1..100}; do ./DietPi/dietpi.txt 2> /dev/null; done

real    0m0.266s
user    0m0.016s
sys     0m0.056s
root@VM-Stretch:~# time for i in {1..100}; do ./DietPi/dietpi.txt 2> /dev/null; done

real    0m0.274s
user    0m0.028s
sys     0m0.048s
root@VM-Stretch:~# time for i in {1..100}; do ./DietPi/dietpi.txt 2> /dev/null; done

real    0m0.278s
user    0m0.020s
sys     0m0.060s

• Damn :rofl:!

Reduce influence of grep

root@VM-Stretch:~# time for i in {1..1000}; do . /dev/stdin <<< $(echo 'TEST=1'); done

real    0m3.018s
user    0m0.216s
sys     0m0.792s
root@VM-Stretch:~# time for i in {1..1000}; do . /dev/stdin <<< $(echo 'TEST=1'); done

real    0m3.022s
user    0m0.204s
sys     0m0.824s
root@VM-Stretch:~# time for i in {1..1000}; do . /dev/stdin <<< $(echo 'TEST=1'); done

real    0m2.998s
user    0m0.184s
sys     0m0.824s
root@VM-Stretch:~# time for i in {1..1000}; do eval $(echo 'TEST=1'); done

real    0m2.808s
user    0m0.128s
sys     0m0.680s
root@VM-Stretch:~# time for i in {1..1000}; do eval $(echo 'TEST=1'); done

real    0m2.807s
user    0m0.128s
sys     0m0.668s
root@VM-Stretch:~# time for i in {1..1000}; do eval $(echo 'TEST=1'); done

real    0m2.830s
user    0m0.132s
sys     0m0.668s
root@VM-Stretch:~# time for i in {1..1000}; do . <(echo 'TEST=1'); done

real    0m2.692s
user    0m0.236s
sys     0m0.752s
root@VM-Stretch:~# time for i in {1..1000}; do . <(echo 'TEST=1'); done

real    0m2.598s
user    0m0.192s
sys     0m0.728s
root@VM-Stretch:~# time for i in {1..1000}; do . <(echo 'TEST=1'); done

real    0m2.732s
user    0m0.204s
sys     0m0.804s

• Very strange... Generally process substutution is fastest, as I expected first, but it seems that running non-builtin commands is somehow inefficient within the process substitution.

root@VM-Stretch:/tmp# cat test
TEST=1
root@VM-Stretch:/tmp# time for i in {1..1000}; do eval $(cat test); done

real    0m4.724s
user    0m0.460s
sys     0m1.504s
root@VM-Stretch:/tmp# time for i in {1..1000}; do eval $(cat test); done

real    0m4.703s
user    0m0.428s
sys     0m1.268s
root@VM-Stretch:/tmp# time for i in {1..1000}; do . <(cat test); done

real    0m6.344s
user    0m0.204s
sys     0m0.904s
root@VM-Stretch:/tmp# time for i in {1..1000}; do . <(cat test); done

real    0m6.288s
user    0m0.192s
sys     0m0.864s
root@VM-Stretch:/tmp# time for i in {1..1000}; do . /dev/stdin <<< $(cat test); done

real    0m4.908s
user    0m0.540s
sys     0m1.492s
root@VM-Stretch:/tmp# time for i in {1..1000}; do . /dev/stdin <<< $(cat test); done

real    0m4.958s
user    0m0.592s
sys     0m1.652s

---

Result

• So at least in combination with grep, we should go with eval, although I like the process substitution method for it's simplicity.
• If we anyway need get some more settings values within a script, we can think about sourcing the whole file. Of course this creates much unused variables, but it terms of performance it wins dramatically against even a single grep. For this we should recheck that dietpi.txt is fully source-able. Currently at least the global PW entry, after replaced by info, is not. Redirecting errors solves this, but better to then assure for all lines.

[MichaIng]

Single quotes in left side double brackets argument

Wow, didn't know that using single quotes to prevent magic character/syntax expansion has such a measurable effect:

2019-04-14 17:45:17 root@micha:/tmp# time for i in {1..10000}; do [[ -f '/DietPi/dietpi/.version' ]] && :; done

real    0m0.589s
user    0m0.475s
sys     0m0.111s
2019-04-14 17:45:18 root@micha:/tmp# time for i in {1..10000}; do [[ -f '/DietPi/dietpi/.version' ]] && :; done

real    0m0.588s
user    0m0.536s
sys     0m0.048s
2019-04-14 17:45:19 root@micha:/tmp# time for i in {1..10000}; do [[ -f '/DietPi/dietpi/.version' ]] && :; done

real    0m0.607s
user    0m0.556s
sys     0m0.048s
2019-04-14 17:45:20 root@micha:/tmp# time for i in {1..10000}; do [[ -f /DietPi/dietpi/.version ]] && :; done

real    0m0.660s
user    0m0.577s
sys     0m0.079s
2019-04-14 17:45:26 root@micha:/tmp# time for i in {1..10000}; do [[ -f /DietPi/dietpi/.version ]] && :; done

real    0m0.655s
user    0m0.572s
sys     0m0.079s
2019-04-14 17:45:28 root@micha:/tmp# time for i in {1..10000}; do [[ -f /DietPi/dietpi/.version ]] && :; done

real    0m0.659s
user    0m0.580s
sys     0m0.074s
2019-04-14 17:45:29 root@micha:/tmp# time for i in {1..10000}; do [[ -f /DietPi/dietpi/.version ]] && :; done

real    0m0.666s
user    0m0.591s
sys     0m0.068s

I did this much more often on three different systems and with no double, when not using single quotes, the check for at least variable expansion, as this is the only expansion that is actually done here (./*/?/[]/{} all are taken literally).

[MichaIng]

var='' vs unset var

2019-04-14 17:51:44 root@micha:/tmp# time for i in {1..10000}; do unset G_PROGRAM_NAME; done

real    0m0.507s
user    0m0.507s
sys     0m0.000s
2019-04-14 17:51:46 root@micha:/tmp# time for i in {1..10000}; do unset G_PROGRAM_NAME; done

real    0m0.495s
user    0m0.495s
sys     0m0.000s
2019-04-14 17:51:47 root@micha:/tmp# time for i in {1..10000}; do unset G_PROGRAM_NAME; done

real    0m0.498s
user    0m0.497s
sys     0m0.000s
2019-04-14 17:51:48 root@micha:/tmp# time for i in {1..10000}; do G_PROGRAM_NAME=''; done

real    0m0.288s
user    0m0.287s
sys     0m0.000s
2019-04-14 17:51:55 root@micha:/tmp# time for i in {1..10000}; do G_PROGRAM_NAME=''; done

real    0m0.266s
user    0m0.266s
sys     0m0.000s
2019-04-14 17:51:56 root@micha:/tmp# time for i in {1..10000}; do G_PROGRAM_NAME=''; done

real    0m0.267s
user    0m0.267s
sys     0m0.000s

Sadly calling unset clearly is slower. However in this case I still prefer it to not accumulate empty variables, especially in login console. Hitting $+<tab> should be as clean as possible IMO.

[MichaIng]

Bash string comparison: glob vs ext regex

root@VM-Buster:~# time for i in {1..10000}; do [[ $test =~ [[:blank:]]UP[[:blank:]] ]]; done

real    0m1.095s
user    0m1.095s
sys     0m0.000s
root@VM-Buster:~# time for i in {1..10000}; do [[ $test =~ [[:blank:]]UP[[:blank:]] ]]; done

real    0m1.089s
user    0m1.089s
sys     0m0.000s
root@VM-Buster:~# time for i in {1..10000}; do [[ $test =~ [[:blank:]]UP[[:blank:]] ]]; done

real    0m1.086s
user    0m1.086s
sys     0m0.000s
root@VM-Buster:~# time for i in {1..10000}; do [[ $test == *[[:blank:]]UP[[:blank:]]* ]]; done

real    0m0.281s
user    0m0.281s
sys     0m0.000s
root@VM-Buster:~# time for i in {1..10000}; do [[ $test == *[[:blank:]]UP[[:blank:]]* ]]; done

real    0m0.294s
user    0m0.294s
sys     0m0.000s
root@VM-Buster:~# time for i in {1..10000}; do [[ $test == *[[:blank:]]'UP'[[:blank:]]* ]]; done

real    0m0.288s
user    0m0.288s
sys     0m0.000s
root@VM-Buster:~# time for i in {1..10000}; do [[ $test == *[[:blank:]]'UP'[[:blank:]]* ]]; done

real    0m0.295s
user    0m0.295s
sys     0m0.000s
root@VM-Buster:~# time for i in {1..10000}; do [[ $test =~ [[:blank:]]'UP'[[:blank:]] ]]; done

real    0m1.109s
user    0m1.109s
sys     0m0.000s
root@VM-Buster:~# time for i in {1..10000}; do [[ $test =~ [[:blank:]]'UP'[[:blank:]] ]]; done

real    0m1.120s
user    0m1.120s
sys     0m0.000s

• What I always wanted to know 😄: Checking against extended regex indeed is much slower than checking against simple glob only. Quoting the literal parts has no effect.