Tree-sitter: get_column causes infinite loop

[worudso]

I tried get_column but it didn't work as I expected.
So I just put lexer->get_column(lexer); at the beginning of scan function to check the behavior.

bool tree_sitter_cham_external_scanner_scan(
    void *payload,
    TSLexer *lexer,
    const bool *valid_symbols)
{

  lexer->get_column(lexer);

But then for every input it never stops and gives any result neither.
My tree-sitter-cli version is 0.16.2.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.89. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[worudso]

I think I figure out when it happens.
It happens when the lexer reaches EOF.

bool tree_sitter_cham_external_scanner_scan(
    void *payload,
    TSLexer *lexer,
    const bool *valid_symbols)
{
  if(!lexer->eof) {
    lexer->get_column(lexer);
  }

After I put a guard before get_column call, it worked well.

[maxbrunsfeld]

Interesting. Note that lexer->eof is a function though, so with that code, get_column will never be called.

Try this:

bool tree_sitter_cham_external_scanner_scan(
    void *payload,
    TSLexer *lexer,
    const bool *valid_symbols)
{
  if(!lexer->eof(lexer)) {
    lexer->get_column(lexer);
  }

  // ...
}

[worudso]

@maxbrunsfeld I've checked it and it's working. If my guess is right, then I think this is a common mistake for beginners and it would be better to document.