Rust: Tracking issue for libsyntax breaking changes

[Manishearth]

See also: https://internals.rust-lang.org/t/please-consider-stability-of-libsyntax/2947

We decided to batch up breaking changes to oft-used unstable compiler internals (mainly, libsyntax) until they're large enough; so that the impact of these breaking changes is minimized. (For example, every time libsyntax breaks, aster and transitively everyone using serde on nightly has to fix it)

Let's try to coordinate these in the future through this issue. We can list breaking PRs here and batch them up when we feel that they're large enough (or when there's another inevitable breaking change about to merge). Merge them when there's enough time for the folks below to know about it, and preferably make PRs to the larger projects below beforehand.

CCing stakeholders:

• libsyntax: @nrc @huonw @eddyb
• clippy: me, @llogiq
• diesel: @sgrif
• phf: @sfackler
• regex_macros: @BurntSushi, @alexcrichton
• rust-cpp, other plugins: @mystor

Add more if you know of them!

[BurntSushi]

I would also like to re-submit my plea that commits with breaking changes are tagged, and if possible, at least some message that hints at or explains how to fix breakages.

Other crates with macros in use (apparently) are docopt_macros and quickcheck_macros.

[Manishearth]

Yeah, they get tagged with [breaking-change]. With this bug in place they should also be posted here first and we can notify everyone well in advance before a rollup.

[Manishearth]

https://github.com/rust-lang/rust/pull/32767 is a breaking change that will land soon, along with #32688.

I'd normally wait a few days before merging these so that everyone can fix their code early, but in this case #32688 already landed so there's not much I can do.

This will definitely break syntex and friends, cc @erickt. I've never looked at diesel's code so it may or may not break it, @sgrif you may want to have a look before this lands in the nightly.

[sgrif]

Thanks for the ping, definitely going to cause breakage.

[Manishearth]

https://github.com/rust-lang/rust/pull/33041 will land in a few days. It changes the parser and some of the token tree representation

• Definitely will break syntex/aster/serde (cc @erickt)
• Might break Diesel, Regex, phf (cc @sgrif, @sfackler, @BurntSushi): anyone who uses macros operating on token trees.

[Manishearth]

Note that https://github.com/rust-lang/rust/pull/31414 has been added to the rollup. It will break derive plugins, not sure what else.

[Manishearth]

Also https://github.com/rust-lang/rust/pull/33125

So far, we will be landing

• #33125 (r=me)
• #31414
• #33041 (r=nrc)
• #33157 (r=me)

once they all get approved

[Manishearth]

https://github.com/rust-lang/rust/pull/33157 is also in the list

[Manishearth]

Merging now: https://github.com/rust-lang/rust/pull/33179

[Manishearth]

It landed

[Manishearth]

For next batch (probably won't be any time soon):

• https://github.com/rust-lang/rust/pull/33351 (r=pnkfelix)
• https://github.com/rust-lang/rust/pull/33639 (r=niko)
• https://github.com/rust-lang/rust/pull/33644 (r=nrc)
• https://github.com/rust-lang/rust/pull/33839 (r=niko)

[jimmycuadra]

I don't think @erickt works on Serde very actively at this point, since he has lots of other duties. Is there another maintainer that should be CCed on this issue?

[Manishearth]

I was going to start doing serde ups but I haven't had the time and I might not get it for a month.

[Manishearth]

@petrochenkov Probably should make your PRs nowish given that the .. one will need to land soon.

[Manishearth]

Batch should happen in a day or two: https://github.com/rust-lang/rust/pull/33864

cc @erickt @dtolnay @sgrif

[Manishearth]

Batch landed!

[kamalmarhubi]

@Manishearth

I was going to start doing serde ups but I haven't had the time and I might not get it for a month.

What's involved? I'm interested in keeping syntex_syntax close to libsyntax so that rustfmt keeps up.

[Manishearth]

Um, ask @erickt

[Manishearth]

@kamalmarhubi https://github.com/serde-rs/syntex/blob/master/CONTRIBUTING.md#how-to-do-a-syntex_syntax-release

and then you fix up aster and quasi to do the same

[kamalmarhubi]

@Manishearth I was about to come here and link that to answer myself, but you're quick! :-)

[Manishearth]

PRs for next batch:

• https://github.com/rust-lang/rust/pull/34213, r=nrc (affects aster, maybe: cc @erickt @dtolnay )
• https://github.com/rust-lang/rust/pull/34316, r=eddyb (probably affects everyone? @sgrif @sfackler @eddyb)
• https://github.com/rust-lang/rust/pull/34339, r=petrochenkov,me
• https://github.com/rust-lang/rust/pull/34368, r=me (probably affects everyone)
• https://github.com/rust-lang/rust/pull/34385, r=nrc
• #34403, r=alexcrichton

mergerollup josephDunne:trait_item_macros 34213 nrc
mergerollup jseyfried:refactor_ast_stmt 34316 eddyb
mergerollup jseyfried:thin_vec 34339 petrochenkov,Manishearth
mergerollup petrochenkov:astqpath 34368 Manishearth
mergerollup cgswords:tstream 34385 nrc
mergerollup jonathandturner:move_liberror 34403 alexcrichton

[dtolnay]

In case it affects anyone else - #34272 was a breaking change for Aster. https://github.com/serde-rs/aster/issues/88. I guess nobody noticed because rustup has been serving up nightly 2016-06-15 until recently.

This would be a good time to merge the other 5 if they are ready.

[mcarton]

Cc me.

(sorry for the noise but for some reason GitHub never notifies me for this despite I've subscribe to the thread)

[jseyfried]

@dtolnay sorry about that, I underestimated how much of the libsyntax API was being used by third parties. I'll be sure to check aster/serde in the future before landing anything borderline.

[Manishearth]

• #35591 (r=jntrnr)
• https://github.com/rust-lang/rust/pull/35618 (r=eddyb)
• #35480 (r=nikomatsakis)
• https://github.com/rust-lang/rust/pull/35728 (r=manishearth)
• https://github.com/rust-lang/rust/pull/35850 (r=nrc)
• https://github.com/rust-lang/rust/pull/35917 (r=nrc)
• #36016 (r?eddyb)

[Manishearth]

Next breaking batch will contain the above PRs, waiting on the last one. cc @dtolnay @sfackler @mystor @sgrif @BurntSushi

[Manishearth]

Everything seems reviewed, breaking batch will be made todayish (and land over the weekend).

cc @dtolnay @BurntSushi @sgrif @sfackler @llogiq

[Manishearth]

Batch at https://github.com/rust-lang/rust/pull/36066

[Manishearth]

Landed!

[Manishearth]

Stuff for next batch:

• https://github.com/rust-lang/rust/pull/35874 (r=Manishearth)
• https://github.com/rust-lang/rust/pull/36599 (r=pnkfelix)
• #34764 (r=eddyb)

- https://github.com/rust-lang/rust/pull/36551

mergerollup CensoredUsername:stmt_let_typed_fix 35874 Manishearth
mergerollup jonas-schievink:whats-a-pirates-favorite-data-structure 36599 pnkfelix
mergerollup pnkfelix:attrs-on-generic-formals 34764 eddyb

[mcarton]

@Manishearth don't know if it's to late for the two other PRs to make it into the next nightly, but https://github.com/rust-lang/rust/pull/36551 was a syntax breaking change and has already been merged.

[Manishearth]

Too late :/ But we're rolling it up now anyway

[Manishearth]

Breaking batch at https://github.com/rust-lang/rust/pull/36857 cc @dtolnay @sgrif @BurntSushi

Will merge tomorrowish.

[dtolnay]

This is exciting - I am not planning to do a serde_macros release so we will see if people revolt or move to serde_derive.

[Manishearth]

Should we PSA this?

[dtolnay]

The announcement that we are dropping serde_macros was at the top of reddit for a while and also popular on u.r-l.o. I think people will put two and two together when their builds fail. Worst case we get a handful of issues filed and we point them to the announcement. Do you know a better way to reach people who have not seen either of those already?

[sfackler]

You could potentially cut a new release that panics with a message telling people to move to serde-derive.

[mcarton]

@dtolnay Has this been announced in the Project Updates section of TWiR?

[dtolnay]

No we just missed the last one.

[Manishearth]

The panicky method feels a bit invasive imo.

It should be worth mentioning somewhere (perhaps on the crates.io page, or in the readme) that the method for upgrading is to switch over to the new macro version.

[eddyb]

If it has access to the diagnostics machinery, printing a deprecation warning wouldn't be that hard.

[Manishearth]

This landed. beware the breakage!

[sgrif]

We should be done with the macros 1.1 port later today so I no longer need to get pinged on these. Thank you for including me though!

[petrochenkov]

Proc macros are not tied to libsyntax now, so this is no longer relevant.