Rust: const_generics: parsing fails with const expression in type parameter

[jonhoo]

I tried this code:

#![feature(const_generics)]

struct Foo<const N: usize>([(); N]);
type A = Foo<std::mem::size_of::<u8>()>;

It fails to parse with

error: expected one of `!`, `+`, `,`, `::`, or `>`, found `(`
  --> src/lib.rs:12:37
   |
 4 | type A = Foo<std::mem::size_of::<u8>()>;
   |                                     ^ expected one of `!`, `+`, `,`, `::`, or `>`

I assume that this _is_ supposed to work though, since this code is accepted:

const X: usize = std::mem::size_of::<u8>();
type A = Foo<X>;

Meta

rustc --version --verbose:

1.44.0-nightly (2020-04-02 537ccdf3ac44c8c7a8d3)

[jonhoo]

I also tried this:

macro_rules! n {
    ($t:ty) => {
        std::mem::size_of::<$t>()
    }
}

type B = Foo<n!(u8)>;

But that does not work for a _different_ reason:

error: macro expansion ignores token `(` and any following
  --> src/lib.rs:7:32
   |
7  |         std::mem::size_of::<$t>()
   |                                ^^
...
12 | type A = Foo<n!(u8)>;
   |              ------ caused by the macro expansion here
   |
   = note: the usage of `n!` is likely invalid in type context

[Centril]

I assume that this _is_ supposed to work though, since this code is accepted:

It's not supposed to work. In general, it's not possible, using finite look-ahead (LL(k) for some fixed constant k), without backtracking, to determine whether some arbitrary generic argument should be interpreted syntactically as a type or as an expression.

With finite look-ahead, the best we can do is recognize that std is the start of a path, parse the entire path until std::mem::size_of::<u8> and then decide on ( that this is in fact an expression and not a type. This however, complicates parsing notably.

What we have currently encoded is that simple identifiers, literals, and generic arguments starting with { are interpreted as an expression, and otherwise the generic argument is interpreted as a type. This logic is encoded in fn parse_generic_arg.

But that does not work for a _different_ reason:

According to the rules for I've noted above, n!(u8) is interpreted as a type, specifically as a type macro. The interpretation of the macro expansion result (the token stream) is interpreted by the context (types). When parsing a type from std::mem::size_of::<u8>(), a type std::mem::size_of::<u8> is parsed, and then () is left-over. However, the language requires that the token stream be completely consumed by the context, which isn't the case here. Accordingly, you get an error about tokens including and following ( being ignored.

cc @varkor @eddyb @petrochenkov

[jonhoo]

Ah, interesting, thank you for the explanation! That {} is valid as a disambiguator in type parameter position is what I was missing. Is this documented somewhere? It'd be _great_ if the error message actually suggested this course of action. Maybe this issue could be converted to a diagnostics issue?

[Centril]

Is this documented somewhere?

In the compiler implementation. ;) Most of const generics isn't documented in any user facing docs. The rustc-dev-guide might be the next best thing, but I don't think it's well documented here.

It'd be _great_ if the error message actually suggested this course of action. Maybe this issue could be converted to a diagnostics issue?

Sure, although it's sorta a duplicate of https://github.com/rust-lang/rust/issues/61175. To implement the diagnostics here, we would need to snapshot the parser state before parsing an argument, then if parsing a type failed, we can backtrack and attempt parsing as an expression, emitting a suitable diagnostic for that. The drawback here is that backtracking can slow down the parser (and thus compile-times). Whether that is notable is something we'd need to measure. Perhaps it's not significant, in which case we could add those diagnostics.

[jonhoo]

Up to you really whether you think the two issues are distinct enough to be worth keeping them both. You know better than I do :)

[varkor]

Is this documented somewhere?

In the compiler implementation. ;)

It's documented in the const generics RFC. We don't have proper documentation for const generics yet, because it's a work in progress.

[estebank]

@jonhoo This was my previous attempt at handling this and provide suggestions to the abiguous subset of cases: https://github.com/rust-lang/rust/pull/64700.

[varkor]

It may be sufficient to revive @estebank's following pull requests here:

• https://github.com/rust-lang/rust/pull/64700
• https://github.com/rust-lang/rust/pull/71592
• https://github.com/rust-lang/rust/pull/72273

[varkor]

If no-one else is intending to take this issue, I'm going to take a look at it this weekend.

[varkor]

I've filed https://github.com/rust-lang/rust/pull/77502 as a follow up to #71592, which fixes the original issue here, though there's still work to be done.