Lark: About group items

[zeeqy]

Hi, in grammar, what's the rule for "word and some special characters"

Example

I'm sad.

The goal is to have "I'm" and "sad".

Using %import common.WORD gives UnexpectedCharacters error

I tried to use group item but not sure about the syntex.

wordset: ("I" "m" "'")

UnexpectedCharacters: No terminal defined for 'I' at line 1 col 1

[erezsh]

That is because ' isn't considered part of a word. Try something like:

ENG_WORD: WORD ["'" WORD]

[erezsh]

Closing the issue, but feel free to continue the discussion or ask more questions

[zeeqy]

Thanks for helping. I do have one question about number combined with words.

First everything works well with grammar:

ENG_WORD: WORD ["'" WORD]

Then I encounter strings like "100 students". The goal is to parse it into two ENG_WORD: 100, students.

Also how about word with number in it, like "ww2" and "Iphone8"

I have tried ENG_WORD: WORD ["'" WORD] [NUMBER WORD], it won't work.

(I'm so noob!)

[erezsh]

What exactly is the result you're hoping for?

Do you want ww2 to parse into WORD, or WORD NUMBER?

[zeeqy]

my goal is that any consecutive combination of letter and number should be parsed into a "word".

special examples like
I'm happy should be parsed into 2 words I'm and happy.
book about ww2 should be parsed into 3 words book, about and ww2

The issue now is when it has number combined with letter, it throws an error. I know it's my grammar, but I don't know how to fix it

[erezsh]

Sounds like you just need to define word yourself:

WORD: (LETTER | DIGIT) +

%import common (LETTER, DIGIT)