Lark: Tree transform to dict

[aravindkumaremis]

What is your question?

I am trying with some code to transform but im finding difficult to transform the tree to a dict

import json
from lark import Lark, Transformer, v_args, Tree


grammar = r'''start: instruction

?instruction: filters_function

property: "property-insensitive"        -> property_insensitive
        | "property"                    -> property
property_date: "age"                    -> age_filter
             | "date"                   -> date_filter

// To run over special characters
TEXT: (LETTER+) (LETTER+|DIGIT+|"-"|"_")*
FILTER: TEXT
SPECIAL_VALUE: "${" (LETTER+) (LETTER+|DIGIT+|"-"|"_") "}" *
WILD_CARD: "%" TEXT | TEXT | NUMBER
VALUE: WILD_CARD|SPECIAL_VALUE|TEXT

ARGSEP: ":" // argument separator

STAR: "*"
filters_function: FILTER (property | property_date) VALUE *

// find the whitespace so we can ignore
WHITESPACE: (" " | "\n")+

%ignore WHITESPACE
%import common.LETTER
%import common.WORD
%import common.DIGIT
%import common.INT -> NUMBER
%ignore ARGSEP
%ignore " "
'''

parser = Lark(grammar, parser='earley') # lalr
print(parser.parse("filter:property-insensitive:Surname:%Name"))
print(parser.parse("filter:property-insensitive:Sex:M:F:U"))
print(parser.parse("filter:age:on:${today}:Equal:26:years"))
print(parser.parse("filter:date:Equal:${today}"))
print(parser.parse("filter:property:Type:regular:temp"))
print(parser.parse("filter:property:Status:active:deducted:deceased"))
print(parser.parse("filter:date:LessThanOrEqual:YMD-2020-04-15:plus:1:days"))
print(parser.parse("filter:date:LessThanOrEqual:YMD-2020-04-15:plus:1:months"))

final = parser.parse("filter:property-insensitive:Sex:M:F:U")

class TreeTransformer(Transformer):
    out = []

    def start(self, content):
        #out = []
        for token in content:
            if isinstance(token, str):
                print('string')

    def filters_function(self, content):
        for token in content:
            print(token)
        return content

    def property_insensitive(self, content):
        return 'property_insensitive'

    def FILTER(self,content):
        return content

    def property(self, args):
        return 'property'

transformer = TreeTransformer()
print(transformer.transform(final))

What i'm trying with the above parser:

• The second value of the string always define the next action filter:property: so im trying to add those as the parent of the dict property. Basically im trying to transform the value to return as below.

{ "property-insensitive": ["Sex", "M", "F", "U"]}

{ "property": ["Type", "regular", "temp"]}



md5-9db203b20ce59104f80f92794b28a521



{"date": ["LessThanOrEqual", "YMD-2020-04-15", "plus", "1", "days"]}

• for me the property_insensitive tree is always [] empty, im trying to get the same value of property_insensitive or the rest of the value followed after the first two string separated by :

Note: Im a new starter so the code will look bad :( It would also be nice to get a link for the documents of Transform.

[MegaIng]

Why are you trying to use lark? This looks like a perfect use case for the the python string .split() function.

The documentation for Transformer is linked from the README.md via read-the-docs.

[aravindkumaremis]

@MegaIng Yes your correct, our existing solutions are written with split, but we end up with lot of conditional statements, as you can see the filters are vary.. 100+ and we dont want to add conditions for all, so we checked for a parser, and gone through antlr and its not that effective and lark is good so we kind of trying out.

[erezsh]

@aravindkumaremis There's a lot to do here, and I think you should just read more of Lark's example to get a feel of how it works best.

For example, by using ignore ":", you're saying that Sex:::M::F::::U is valid syntax

It's better if you make it explicit, imho

And in general, you should make sure that rules contain the same elements that you would want in the AST, in the same orders.

Look at tree-shaping operators to see what's what (?, ! etc.)

In general read this top-to-bottom before proceeding: https://lark-parser.readthedocs.io/en/latest/grammar/

[aravindkumaremis]

Thanks for the Help.