Msys2-packages: awk regexp escape sequence \# not known

[dlwlb]

AWK script written for MSYS[1] now outputs warning under MSYS2. I found similar reported and fixed for gawk: https://dev.gnupg.org/T4492 . I'm not sure if this is a new or fixed issue. It seems that MSYS awk may have required # to be escaped but MSYS2 not. I simplified the incident below, to output X instead of n or # . Note that # must be escaped for echo.

$ echo n | awk '/[n#]/ { printf("X"); }'
X
$ echo # | awk '/[n#]/ { printf("X"); }'

$ echo -e # | awk '/[n#]/ { printf("X"); }'

$ echo -e # | awk '/[n#]/ { printf("X"); }'
X
$ echo -e # | awk '/[n#]/ { printf("X"); }'
awk: cmd. line:1: warning: regexp escape sequence `#' is not a known regexp operator
X

[dlwlb]

The issue was discovered with gawk 5.0.1-1:

$ pacman -Q awk
gawk 5.0.1-1

[jannick0]

If I understand your test cases correctly, I cannot see the issue you are raising.:

• Using gawk 5.0.1 (MSYS2) and awk 3.1.7 (MSYS[1]), I cannot confirm the behaviour you describe.
• echo # and echo -e # emit an empty line, so it is very likely that those tests do not work. So I am not sure if this did really happen:

$ echo -e # | awk '/[n#]/ { printf("X"); }'
X

• With the (g)awk versions above # in regexps can be escaped, but need not to be.
• The fix you mention above replaces \# by #, since the escaped char could not be digested by the (g)awk version used in this very case.

[dlwlb]

@jannick0, thanks for the reply. It seems that when I pasted my original example that backslashes were stripped after posted. When I "edit" it they reappear. I had considered putting that in a "code" block but then decided not to since the output was intermingled. But I tried to submit my examples (I expanded to include exhaustive variants) again as code to hopefully preserve the backslashes but they were not, yet visible when I edit this post. So now I am editing this post again (I now see the Preview tab) to find a way to preserve them. Is there a way to disable Markdown? It is further complicated by the fact that the output contains the tick and back-tick that this editor uses for code blocking. "insert a quote" does not either. I'll [re]attach a [corrected] text file.
awkbackhash.txt

[jannick0]

Umm, I think I cannot see a real difference though. In the github editor you might want to put your code in a block starting and ending with ~~~ on separate lines. This should leave the code in this block as is, i.e. display it verbatim.

[jannick0]

OK - here the content of your file enclosed in ~~~ [EDIT] ... and after the update of the source file:
~~~echo n | awk '/[n#]/ { printf("X"); }'
echo n | awk '/[n#]/ { printf("X"); }'
echo # | awk '/[n#]/ { printf("X"); }'
echo # | awk '/[n#]/ { printf("X"); }'
echo # | awk '/[n#]/ { printf("X"); }'
echo # | awk '/[n#]/ { printf("X"); }'
echo -e # | awk '/[n#]/ { printf("X"); }'
echo -e # | awk '/[n#]/ { printf("X"); }'
echo -e # | awk '/[n#]/ { printf("X"); }'
echo -e # | awk '/[n#]/ { printf("X"); }'

$ echo n | awk '/[n#]/ { printf("X"); }'
X
dbulgrien@DennisBLaptop MSYS ~
$ echo n | awk '/[n#]/ { printf("X"); }'
awk: cmd. line:1: warning: regexp escape sequence `#' is not a known regexp operator
X
dbulgrien@DennisBLaptop MSYS ~
$ echo # | awk '/[n#]/ { printf("X"); }'

dbulgrien@DennisBLaptop MSYS ~
$ echo # | awk '/[n#]/ { printf("X"); }'
~~~

[dlwlb]

@jannick0 Sorry. I was flustered and my attachment was flawed, thus so is your output above. I edited the attachment with corrections.

[dlwlb]

Note the inconsistency of the warning when the input is an empty line.

[jannick0]

Ok - got it. I think the main issue is that since gawk 5.0 the regex engine is changed and that this issue here might be closely related to the one you are currently raising on the gawk userlist. So I'll better leave it with gawk's maintainers to sort this out. :)

[dlwlb]

I don't think I am currently raising an issue on the gawk userlist. Your reference is to error: "Invalid range end" by Denis Shirokov which does not readily seem apparent. My reference to another backslash-hash problem by JW is closed and resolved, so I don't expect any more work.

[jannick0]

Aah, my apologies! My assumption that two persons named Dennis here and Denis on the gawk userlist posting an issue most likely related to gawk's regex engine change are the same was certainly one bridge too far. :( :)

Good to hear that the issue is solved now, and so fast. You might want to close this issue now?

[dlwlb]

I don't know if it is fixed. Do you think what I am experiencing is correct behavior, that MSYS[1] awk needs backslash-hash but MSYS2 awk must NOT, and the the warning should be suppressed if the input is empty?

[jannick0]

My reference to another backslash-hash problem by JW is closed and resolved, so I don't expect any more work.
I don't know if it is fixed.

I thought it was because of what you said above.

Do you think what I am experiencing is correct behavior, that MSYS[1] awk needs backslash-hash but MSYS2 awk must NOT, and the the warning should be suppressed if the input is empty?

If in doubt I would consult the latest POSIX version on awk. My tests below show when using the POSIX flag (which is not different from not using it, BTW):

• MSYS1: both # and \# enclosed in a char class or not work. See below.
• MSYS2: all work, too, but gawk 5.0.x throws a warning, not a fatal error.

Test on MSYS1:
~~~bash
$ cat ./hash.sh

! /usr/bin/bash

file hash.sh

awk --version | head -n 1
echo '#' | awk --posix '/#/ {print "|" $0 "|" }'
echo '#' | awk --posix '/[#]/ {print "|" $0 "|" }'
echo '#' | awk --posix '/#/ {print "|" $0 "|" }'
echo '#' | awk --posix '/[#]/ {print "|" $0 "|" }'

$ ./hash.sh
GNU Awk 3.1.7
|#|
|#|
|#|
|#|
~~~

Test on MSYS2:
~bash
$ ./hash.sh
GNU Awk 5.0.1, API: 2.0 (GNU MPFR 4.0.2, GNU MP 6.1.2)
|#|
|#|
awk: cmd. line:1: warning: regexp escape sequence \#' is not a known regexp operator |#| awk: cmd. line:1: warning: regexp escape sequence#' is not a known regexp operator
|#|
~

[dlwlb]

Isn't your test flawed because it prints $0 due to the empty regexp // resulting from the warning?
~
echo '#' | awk '// {print "|" $0 "|" }'
~

[jannick0]

... depends what one wants to test. For your purposes something like this might be a starting point other combinations could be generted from:
~
echo '1#' | awk '/[1#]/'
~

[dlwlb]

I was wrong about //. Regex # works with or without the backslash which is discarded such that a backslash input won't match and the warning to stderr can be ignored. Ubuntu 12.04.64 functions the same (but without a warning). So the only flaw might be the lack of a warning on blank input. Thank you for the engagement. PS. I like your continued simplification of the test cases.

[jannick0]

Great - glad you found a solution to your problem.