Lark: Standalone parser cannot be used more than once

[supposedly]

That is, Lark_StandAlone().parse() cannot be called more than once on different inputs. This is from master. I found two reasons for it:

• rule.alias is overwritten permanently in ParseTreeBuilder.create_callback(), causing errors elsewhere on subsequent runs.
• In Python 3, the wrapper_chain = filter(...) in ParseTreeBuilder._init_builders() creates a filter object that can only be consumed once.

Now, within an already-generated standalone parser, I've found it suffices to make the following edits:

• Rule.__init__(): add self.cb_name = None
• Line 2 of Lark_StandAlone.__init__(): change to callbacks = {rule: getattr(callback, rule.cb_name or rule.origin, None) for rule in RULES.values()}
• Line 19 of ParseTreeBuilder.create_callback(): change to rule.cb_name = internal_callback_name
• Line 6 of ParseTreeBuilder._init_builders(): change to wrapper_chain = list(filter(None, [

However, when trying to edit these into Lark itself (so that lark.tools.standalone would create a parser file with these changes already in place), the addition of the Rule().cb_name attribute caused lots of errors/failed tests -- and I couldn't be positive where in the package to replace rule.alias and where not to. So I'm pretty sure it's not the way to go. Thoughts?

[erezsh]

It seems to be working for me just fine.

Can you please provide a minimal example that demonstrates this issue?

Also, make sure you're using the latest master branch.

[supposedly]

Pardon the late response (got tied up) and the in-retrospect-very-incomplete initial post: I hadn't realized that these issues are only noticeable when you're using a transformer. Here's an example of that, assuming these three files are in the same directory:

// grammar.lark
start: /a/+

```py

main.py

from standalone import Lark_StandAlone, Transformer

class MyTransformer(Transformer):
def start(self, children):
return children

print(MyTransformer().transform(Lark_StandAlone().parse('aaa')))
print(MyTransformer().transform(Lark_StandAlone().parse('aaa')))

And [this](https://gist.github.com/supposedly/50892d837019a3fb9ddfd70b43ddb495) is a standalone-parser file generated by running `python3.7 -m lark.tools.standalone grammar.lark > standalone.py` with Lark grabbed from master.

----

Running with no modifications causes it to print the following:
```py
[Token(__ANON_0, 'a'), Token(__ANON_0, 'a'), Token(__ANON_0, 'a')]
Tree(_cb0_NonTerminal('start'), [Tree(_cb2_NonTerminal('__anon_plus_0'), [Tree(_cb2_NonTerminal('__anon_plus_0'), [Tree(_cb1_NonTerminal('__anon_plus_0'), [Token(__ANON_0, 'a')]), Token(__ANON_0, 'a')]), Token(__ANON_0, 'a')])])

#

Here's a gist showing possible edits, the result of each of which follows.

With the Rule().cb_name change applied (but not list(filter(...))), it prints the following:

[Token(__ANON_0, 'a'), Token(__ANON_0, 'a'), Token(__ANON_0, 'a')]
[Tree(__anon_plus_0, [Tree(__anon_plus_0, [Tree(__anon_plus_0, [Token(__ANON_0, 'a')]), Token(__ANON_0, 'a')]), Token(__ANON_0, 'a')])]

With the list(filter(...)) change applied (but not Rule().cb_name), it prints the following:

[Token(__ANON_0, 'a'), Token(__ANON_0, 'a'), Token(__ANON_0, 'a')]
Tree(_cb0_NonTerminal('start'), [Token(__ANON_0, 'a'), Token(__ANON_0, 'a'), Token(__ANON_0, 'a')])

(That one's only observable on Python 3, of course.)

With both changes applied, it prints the following:

[Token(__ANON_0, 'a'), Token(__ANON_0, 'a'), Token(__ANON_0, 'a')]
[Token(__ANON_0, 'a'), Token(__ANON_0, 'a'), Token(__ANON_0, 'a')]

...which appears to be the correct output!

(In my own project, the filter() part of this bug also caused Meta objects stop being given attributes after the first parse, raising e.g. AttributeError: 'Meta' object has no attribute 'line', but with this example that actually happens on the first parse if MyTransformer is prefixed with @v_args(meta=True). I'm not sure if this is a separate issue, sorry. edit: appears to be #275 as I was using 0.6.4)

And, again, I'm not sure whether these changes are applicable considering that they'd need to be applied across all of Lark (all the files the parser-generator composes a standalone parser from, that is).

[erezsh]

Thanks for the bug report. I know what's the problem, and I plan to fix it soon.

[erezsh]

Btw, the bug occurs when you instantiate Lark_StandAlone more than once. If you can avoid that for now, you shouldn't experience any issues.

[erezsh]

Should work now on master.