Antlr4: Error with customized channel names in ANTLR 4.6

[blindpirate]

As reported in #965 , the customized channel name aren't recognized with ANTLR4.6.

I'm testing examples in The Definitive ANTLR4 Reference 12.1, Page 205.

In ANTLR 4.5.3 and earlier version, customized channel name can be declared with

@lexer::members {
public static final int WHITESPACE = 1; 
public static final int COMMENTS = 2;
}

And it may result in some warnings.

With 4.6, the code above will cause an error:

error(177): Cymbol.g4:59:30: WHITESPACE is not a recognized channel name
error(177): Cymbol.g4:62:33: COMMENTS is not a recognized channel name

However, when I turned to use channels block as said in doc, I got errors like:

error(177): Cymbol.g4:58:30: WHITESPACE is not a recognized channel name
error(177): Cymbol.g4:61:33: COMMENTS is not a recognized channel name
error(164): Cymbol.g4:6:0: custom channels are not supported in combined grammars

So I broke them into a lexer grammar and imported it in the combined grammar, but the error still occured:

error(177): Cymbol.g4:12:30: WHITESPACE is not a recognized channel name
error(177): Cymbol.g4:15:33: COMMENTS is not a recognized channel name

Somebody also met this in #965 . But in 4.6, it means that I cannot use custom channel name at all - both the old @lexer::members and the new channels will cause errors.

What is the right way to define a custom channel in ANTLR 4.6+ ?

[KvanTTT]

Can you upload the full version of your grammars or point a link?

[blindpirate]

@KvanTTT You can reproduce it with code snippet in #965

Chan.g4

lexer grammar Chan;
channels { CHAN }
WS : ' ' -> channel(CHAN);
Char : .;

Parse.g4

grammar Parse;
import Chan;
file : Char* EOF;

And then run antlr4 Parse.g4 with ANTLR 4.6

P.S, it works with earlier version, say 4.5.3.

[KvanTTT]

Token, Channel and Mode types checking is more strict since 4.6 version, see this issue for detail: https://github.com/antlr/antlr4/issues/1411. Your code-defined channels will not be recognized.

You should use options { tokenVocab = ChanLexer; } syntax instead of import ChanLexer;. So, the correct version of your grammars is:

ChanLexer.g4:

lexer grammar ChanLexer;
channels { CHAN }
WS : ' ' -> channel(CHAN);
Char : .;

ChanParser.g4:

parser grammar ChanParser;
options { tokenVocab = ChanLexer; }
file : Char* EOF;

[blindpirate]

OK, thank you very much!

[sharwell]

📝 If you really must use @lexer::members, I believe you can use the integer literals instead of the identifiers. For example, the following would likely work in your original grammar:

-> channel(1); // was channel(WHITESPACE)

[davesisson]

I've done the @lexer::members approach with the raw numbers as part of fixing older code when upgrading to 4.6. But what I haven't figured out is why custom channels aren't allowed in a combined grammar. Is the issue only in determining which state we're in (and thus that the channel is legal at that point)?

[blindpirate]

@KvanTTT @sharwell Does the change means that I can no longer use customized channel names in combined grammar?

[sharwell]

@blindpirate That could be the case, and if so I'm not sure that aspect was intentional. That said, my overwhelming recommendation in all cases is to not use combined grammars since they prevent using some of the most powerful features (e.g. lexer modes) and make it much easier to make mistakes that will not be reported by the code generator (e.g. referencing an undefined literal in a parser rule).

[KvanTTT]

@blindpirate it's not possible to use customized channel names in combined grammars in earlier versions of ANTLR too (< 4.6). Check added by yourself issue https://github.com/antlr/antlr4/issues/965

[blindpirate]

@KvanTTT @sharwell Thank you two.