Xonsh: How to split command line in xonsh?

[anki-code]

Hello! Thank you for xonsh it's great!

Is there a way to do shlex.split in xonsh?

For example I want to split the line with filename that contains spaces and quotes:

line = 'ls \'1 "\\\' 2\'\n'
print(line)  # ls '1 "\' 2'
import shlex
shlex.split(line)

I've got:

Traceback (most recent call last):
  File "/opt/miniconda/lib/python3.8/site-packages/xonsh/base_shell.py", line 362, in default
    run_compiled_code(code, self.ctx, None, "single")
  File "/opt/miniconda/lib/python3.8/site-packages/xonsh/codecache.py", line 67, in run_compiled_code
    func(code, glb, loc)
  File "<xonsh-code>", line 1, in <module>
  File "/opt/miniconda/lib/python3.8/shlex.py", line 311, in split
    return list(lex)
  File "/opt/miniconda/lib/python3.8/shlex.py", line 300, in __next__
    token = self.get_token()
  File "/opt/miniconda/lib/python3.8/shlex.py", line 109, in get_token
    raw = self.read_token()
  File "/opt/miniconda/lib/python3.8/shlex.py", line 191, in read_token
    raise ValueError("No closing quotation")
ValueError: No closing quotation

Expected:

shlex.split(line)
# ['ls', '1 "\' 2']

Is there a way to shlex.split in xonsh? Thanks!

For community

⬇️ Please click the 👍 reaction instead of leaving a +1 or 👍 comment

[scopatz]

I think you use Lexer.split(), via __xonsh__.execer.parser.lexer.split(). For example: https://github.com/xonsh/xonsh/blob/master/xonsh/built_ins.py#L1035

[scopatz]

This could probably have an easier-to-find API

[anki-code]

@scopatz is there a way to split the line and get the python evaluation content? For example:

echo 1 | head -n @(1 + ( 1 ) + (1 if 1 > 0 else 0 ) ) | grep 1

Split to something like:

['echo', '1', '|', 'head', '-n', '@(1 + ( 1 ) + (1 if 1 > 0 else 0 ) )', '|', 'grep', '1']

I tried __xonsh__.execer.parser.lexer.split():

$ __xonsh__.execer.parser.lexer.split('echo 1 | head -n @(1 + ( 1 ) + (1 if 1 > 0 else 0 ) ) | grep 1')
['echo', '1', '|', 'head', '-n', '@(1', '+', '(', '1', ')', '+', '(1', 'if', '1', '>', '0', 'else', '0', ')', ')', '|', 'grep', '1']```

But python evaluation was splitted.

[scopatz]

I think you would want to reconstruct this from the AST