Rust: nested macros don't allow repetitions in binding patterns

Created on 20 Aug 2016 · 13Comments · Source: rust-lang/rust

Example:

macro_rules! foo {
    () => {
        macro_rules! bar {
            ( $( $any:tt )* ) => { $( $any )* };
        }
    };
}

fn main() { foo!(); }

https://play.rust-lang.org/?gist=f7355a6828cc2af68cc17f280a982ad8&version=beta&backtrace=0

results in:

error: attempted to repeat an expression containing no syntax variables matched as repeating at this depth

 --> <anon>:4:15
  |
4 |             ( $( $any:tt )* ) => { $( $any )* };
  |               ^^^^^^^^^^^^

Note

If the repetition is removed, i.e. the offending part ( $( $any:tt )* ) => { $( $any )* }; is changed to ( $any:tt ) => { $any };, the error disappears on rustc 1.12+ due to the fix of #6994: _macros should be parsed more lazily_. I find it a bit inconsistent, that nested macros now support binding patterns, but no repetitions.

Most helpful comment

@colin-kiegel How would that help? Consider this silly example:

macro_rules! make_println {
    ($name:ident, $fmt:expr) => {
        macro_rules! $name {
            ($($args:expr),*) => {           // (1)
                println!($fmt, $($args),*);  // (2)
            }
        }
    };
}

If matching and expansion syntax were different, then (1) wouldn't trigger an error, but the "args" part in (2) would still do, because it's intentionally written to look like expansion syntax, except it's not intended for the macro being parsed.

I can only see two solutions:

Whenever an expansion repeater does not contain a valid variable, it should be left alone, suppressing (or delaying) the error "attempted to repeat an expression containing no syntax variables matched as repeating at this depth," in the same way as the error about unknown variables is delayed until expansion (IIUC) which is what allows macro-generating-macros-with-parameters in the first place.

Or:

Provide some kind of escaping for the $ symbol, such as $$. Then it would just be a matter of duplicating (or quadrupling...) any dollar intended for the child (resp. grandchild...) macro:

macro_rules! make_println {
    ($name:ident, $fmt:expr) => {
        macro_rules! $name {
            ($$($$args:expr),*) => {           // (1)
                println!($fmt, $$($$args),*);  // (2)
            }
        }
    };
}

I would go with 2. because it's good to have explicit error messages about common mistakes and misplacing repeater variables is a common mistake. If someone wants to write a (macro-generating)ⁿ-macro, using 2ⁿ dollars instead of one seems like a small price to pay. Backslashes in string constants already work this way. (I've seen up to 8 consecutive backslashes in real-world code; 4 is not uncommon; 2 are everywhere.)

tobia on 15 Jan 2017

👍14

All 13 comments

It appears to be non-trivial to distinguish expansions and binding patterns, because they can both appear in the same position, like this:

macro_rules! foo {
    ( $( $some:tt )* ) => {
        macro_rules! bar {
            ( $( $some )* $( foo )* $( $any:tt bar )* ) => { $( $any )* };
        }
    };
}

Note that $( foo )* is expected to be repeated as often as there are matches for $some:tt. But $( $any:tt bar )* would be expected a matching binding pattern. Wouldn't it?

These distinctions seem to be very delicate and subtle and prone to error. I wonder if it would be better to have explicit escaping (as suggested in https://github.com/rust-lang/rust/issues/6994#issuecomment-48829503).

PS: And something like this should be detected as invalid:

macro_rules! foo {
    ( $( $some:tt )* ) => {
        macro_rules! bar {
            ( $( $some foo $any:tt )* ) => { $( $any )* };
        }
    };
}

colin-kiegel on 20 Aug 2016

And another possible problem, this time on the most inner expansion $( $some $any )*

macro_rules! foo {
    ( $( $some:tt )* ) => {
        macro_rules! bar {
            ( $( $any:tt )* ) => { $( $some $any )* };
        }
    };
}

I have the feeling, that it's not possible to address all these questions without new syntax. Other opinions?

colin-kiegel on 20 Aug 2016

If matching and expansion syntax were different, this would not be a problem.

@nrc fyi - maybe relevant for macros 2.0?

colin-kiegel on 20 Aug 2016

👍2

cc @jseyfried

durka on 21 Aug 2016

This error also occurs for macro definitions inside macro _invocations_, not just other definitions:

macro_rules! foo {
    ($($a:tt)*) => ($($a)*)
}
foo! {
    macro_rules! bar { // same as foo
        ($($a:tt)*) => ($($a)*)
    }
}

Definitely no ambiguity in this case.

comex on 16 Sep 2016

@colin-kiegel How would that help? Consider this silly example:

macro_rules! make_println {
    ($name:ident, $fmt:expr) => {
        macro_rules! $name {
            ($($args:expr),*) => {           // (1)
                println!($fmt, $($args),*);  // (2)
            }
        }
    };
}

I can only see two solutions:

Whenever an expansion repeater does not contain a valid variable, it should be left alone, suppressing (or delaying) the error "attempted to repeat an expression containing no syntax variables matched as repeating at this depth," in the same way as the error about unknown variables is delayed until expansion (IIUC) which is what allows macro-generating-macros-with-parameters in the first place.

Or:

Provide some kind of escaping for the $ symbol, such as $$. Then it would just be a matter of duplicating (or quadrupling...) any dollar intended for the child (resp. grandchild...) macro:

macro_rules! make_println {
    ($name:ident, $fmt:expr) => {
        macro_rules! $name {
            ($$($$args:expr),*) => {           // (1)
                println!($fmt, $$($$args),*);  // (2)
            }
        }
    };
}

tobia on 15 Jan 2017

👍14

I have found myself today wishing this was an implemented feature in Rust.
What would be required to get this issue moving forward?
Specifically I'm interested in pursuing option 2, as it seems the more pragmatic choice i.e. it is the easiest to use and remember rules for.

jjpe on 25 Aug 2018

Small note: the problem from @comex's comment has been fixed already (not sure when).

durka on 25 Aug 2018

For the second problem (@tobia), there exists a workaround, though it kinda sucks: pass in the dollar sign as a token, so you get kind of a "private escape sequence".

macro_rules! make_println {
    ($d:tt $name:ident, $fmt:expr) => {
        macro_rules! $name {
            ($d($d args:expr),*) => {           // (1)
                println!($fmt, $d($d args),*);  // (2)
            }
        }
    };
}

make_println!($ dbg, "{:?}");

~The dollar sign has to be passed in from the top level, because that's the only time you can write it without being interpreted as a macro variable. So there's no way (that I can see) to wrap this up in a meta-macro like with_dollar_sign or something.~ (edit: I was wrong, see below)

You can wrap this to avoid making the ugliness visible at the top level:

macro_rules! with_dollar_sign {
    ($($body:tt)*) => {
        macro_rules! __with_dollar_sign { $($body)* }
        __with_dollar_sign!($);
    }
}

macro_rules! make_println {
    ($name:ident, $fmt:expr) => {
        with_dollar_sign! {
            ($d:tt) => {
                macro_rules! $name {
                    ($d($d args:expr),*) => {           // (1)
                        println!($fmt, $d($d args),*);  // (2)
                    }
                }
            }
        }
    };
}

make_println!(my_dbg, "{:?}");

fn main() {
    my_dbg!(42);
}

durka on 25 Aug 2018

👍9

It can be passed from inside the macro with ($)

See https://github.com/bluss/defmac/blob/6886f04d412e1ee2b6d4d240feb51e51a9caf808/src/lib.rs