Boa: [lexer] in token position counting end boundary should be excluded

[croraf]

For example if you have JS code

let a = 5;

One of the tokens the lexer will output it will say: Token: Identifier "a", start position: 5, end position: 5

Instead it should say ... end position: 6

[Razican]

If I understand the title right, if the boundary should be excluded, then 5 is correct, right? 6 would also include the boundary.

So, in that example, the 5th token is the a token. The first position where that token appears is the 5, and the last position where that token appears is the 5 too.

For let for example, the first position where it appears is 1, and the last one is 3.

[croraf]

No. The right boundary should be excluded meaning that if the right boundary is 5, the character at position 5 is not part of the token.

So for let you will have boundaries 1 and 4, including characters at position 1, 2 and 3. Excluding the character at position 4.

[Razican]

No. The right boundary should be excluded meaning that if the right boundary is 5, the character at position 5 is not part of the token.

So for let you will have boundaries 1 and 4, including characters at position 1, 2 and 3. Excluding the character at position 4.

So, then, for a, boundaries should be 5 and 6, with the 6 excluded, right?

[croraf]

Yes

[Razican]

Yes

Sorry, then I don't understand the bug. The span is set correctly, from position 5 to position 5, right?

[IovoslavIovchev]

Yes

Sorry, then I don't understand the bug. The span is set correctly, from position 5 to position 5, right?

I think the issue is that the end boundary should be exclusive, e.g. in the example a's span is [5, 5], when it should be [5, 6).

[Lan2u]

This will be fixed as part of #486.