Packages: [Shell] Clear string scope in "interpolated subcommands"

[FichteFoll]

The new shell script syntax uses string even within interpolated subcommands, i.e. $(xsel -ob). It would be nice if it cleared the string scope for the contents of these. Same for backticks, probably.

See also the corresponding RFC in #715 with an example from how Python does it.

cc @rwols

[FichteFoll]

It would also be nice if it didn't use punctuation.definition.variable for the $( and ) and instead one of the interpolation punctuation scopes (I have a selector for that in my color scheme 😉).

[rwols]

I have my mind on bugfix-mode for the time being but feel free to make the changes.

If you use clear_scopes during an interpolation, how would you handle:

ECHO=echo
$ECHO 'hello, world!'

Here, the $ECHO still gets a variable.function scope, but it would disappear when using clear_scopes.

[FichteFoll]

clear_scopes would only be used to clear the last string scope from the stack. Within the interpolation expression everything would then be coloredscoped as it would outside of it.

I haven't looked at the bash syntax yet to check how involved this change would be, but it works really great in Python and JavaScript.

[michaelblyons]

All you need is a clear_scopes: 1 in the two pushes in expansion-command.

You might also want to add a meta_content_scope with a non-meta. scope, so that color schemes can "officially" target it. 😉 (In my test, I'm re-using the root scope of source.shell.bash to trigger the source source coloring.)

[rwols]

All you need is a clear_scopes: 1 in the two pushes in expansion-command.

OK, I can see how this can be beneficial. Feel free to make the PR 👍

[michaelblyons]

Now I'm actually going to walk that back a little. There are indeed some context-dependent weirdnesses.

# works good
foo=$(echo "bar")
echo "$(echo "bar")"

# accidentally pops `meta.function-call.arguments`
echo $(echo "bar")

I also noticed a minor bug while I was testing:

foo=bar
#   ^^^ string.unquoted
foi="baz"
#   ^^^^^ string.unquoted string.quoted.double

[rwols]

Your last example is intended. Though I guess it may look a bit weird. The syntax uses a meta_scope: string.unquoted after the equals-sign. Because e.g.

foo="b"'a'z

is still the string baz.

[FichteFoll]

Yes, we're probably looking towards to variants of subcommands, one for within strings that has clear_scopes: 1 and one without them.

@michaelblyons, when working on a PR, please also ensure proper punctuation scopes and consider adding a meta scope for the entire string (like in Python).

[deathaxe]

I've got a fix for that, which can be pushed upstream once #2309 and #2310 are merged.

[FichteFoll]

Does the change conflict with the two referenced prs?

[deathaxe]

It is based on those two PRs and another fix for #1781 including some other optimizations to get rid of duplicated embedded contexts. The main issue I guess are overlapping test cases, which cause conflicts.

Otherwise I'd need to create a larger PR with a "patchset" for several issues.