Runtime: Announcement: Driving towards zero 2.0 bugs by May 10 (ZBB)

[karelz]

As we are getting closer to .NET Core 2.0 Preview and .NET Standard 2.0 Preview ship dates, we would like to drive 2.0.0 bugs down to 0 by May 10 (see milestone details) = ZBB (zero bug bounce) date.

We would like to encourage and ask our contributors to help us drive towards that goal. Be part of .NET Core 2.0 release!

How can you, the CoreFX contributors, help?

1. If you're picking up issues to work on, prefer issues marked as 2.0.0 bugs (some are even up-for-grabs).
2. If you spot something in Future milestone that should be part of 2.0, please say so and explain why (loop in @karelz). Conversely if there's something in 2.0 that perhaps is not as significant as it seems.
   
   
   
   • We strive to deliver high-quality product, so impactful bugs, compatibility, reliability and performance issues are top priority. Migration to .NET Core 2.0 from .NET Core 1.1 or from .NET Framework is one of the key priorities.
   
   
   • Adding additional new APIs, fixing pre-existing bugs or fixing corner-case issues with lower impact and with existing workaround is lower priority, not tracked as 2.0.
   
   
3. Issues with no milestone - we expect to set their milestone in upcoming weeks (part of ongoing "triage"), so if you see something must-have for 2.0 there, please say so and explain why (loop in @karelz).
4. Watch for upcoming official preview announcements (on .NET blog or on dotnet/Announcements repo) to help validate and dogfood the end-to-end .NET Core 2.0 experience before final release.
   
   
   
   • Note: You can dogfood even our daily builds - see dogfooding steps
   
   

Not surprisingly, to deliver overall end-to-end .NET Core 2.0 experience, similar ZBB efforts happen in our sister repos like CoreCLR, core-setup, etc. - we will welcome help in those as well, if you're interested.

Twitter announcement: https://twitter.com/ziki_cz/status/846839252629385216

[karelz]

cc'ing recently active contributors (in random order)

@hughbe @justinvp @JonHanna @xoofx @AlexRadch @stephenmichaelf @nietras
@benaadams @fujiy @jamesqo @SamuelEnglard @couven92 @tintoy @sparraguerra @vcsjones
@bartdesmet @lmolkova @dnickless @bbowyersmyth @matekm @fitzchak @omajid
@seanshpark @hseok-oh @ofirmakmal @hqueue @chunseoklee @Gregory-Bell @jcouv
@Clockwork-Muse @jyoungyun @odyth @lemmaa @kant2002 @lucenticus @bmeverett @ViIvanov @craigajohnson @tdupont750 @KirillOsenkov @svick @jirisykora83
@parjong @tralivali1234 @shrah @khellang @cshung @nikitozz @andrewlock @cesarbs

[karelz]

@roji @alaatm @cakine @TiagoSoczek @brfalcon @ChrisEelmaa @arc001 @thiagocamargos
@mramosMS @anthonylangsworth @borgdylan @stevedesmond-ca @StanislavUshakov @tuespetre @rionmonster @jesperll @kwaazaar @onovotny @NickCraver @cod7alex @IvanKishchenko @daflame76 @cydhaselton @pdeligia @patryk9200 @attilah @reaction1989 @mikedn @Jaedson33 @jaredcnance @TylerBrinkley @darxis @msalsbery @dhoehna @manofstick @FireAlkazar @SWY1985 @willdean @tmds @WinCPP @dcwuser @dennisdietrich @tinchou @the-dwyer @TMiNus @yaakov-h @ofirmakmal

[ctolkien]

milestone details links to the edit page (which non owners can't access) should be: https://github.com/dotnet/corefx/milestone/4

[danmosemsft]

Thanks @ctolkien I fixed the link. As you see from that list we have a lot to do :)

[tarekgh]

@karelz we are keeping this issue just for announcing and no action needed against this issue. right?

[karelz]

Yep, I'd like to keep it opened until 5/10. I can move it into Future milestone, so it doesn't affect our bug count and reports ;-)

[karelz]

FYI if you're not on Twitter:
Here's progress we're making on CoreFX, CoreCLR, core-setup and WCF repos towards 2.0: https://twitter.com/ziki_cz/status/854409726754299904 (as of Tuesday 4/18)

Huge thanks to all contributors who jumped in and actively help us drive the 2.0 bugs!

[karelz]

Announcement: For next 2.5 weeks (until May 10) all API additions and larger changes will get more scrutiny. If they are risky and non-essential for .NET Core 2.0, we may delay merging PR until post-5/10.

Also note that release/2.0.0 branch is now created and will receive last full integration from master on 5/10. Changes after 5/10 will be cherry-picked from master to release/2.0.0 branch.

New update on Twitter with latest progress on 2.0 bugs: https://twitter.com/ziki_cz/status/856692569492774912 (as of Monday 4/24)

[karelz]

Another update on remaining issues: https://twitter.com/ziki_cz/status/858042980673957889 (as of couple of hours ago)

Huge progress in both fixes and triage!

Note that we have new Post-ZBB label to mark issues which are important for 2.0, but don't affect product source code (e.g. documentation bugs, infrastructure issues), so they will be exempt from ZBB rule (so far only 9 across all 4 repos).

[karelz]

Another update on remaining issues: https://twitter.com/ziki_cz/status/859459337042837504
Only 1 week and 1 day left before 5/10

Great progress on issues without milestone (potential work).

So far only 13 bugs are marked as Post-ZBB (documentation bugs, infrastructure issues).
We will likely slip a few Networking issues past 5/10.

[karelz]

Another update: https://twitter.com/ziki_cz/status/861641617865711616
Only 3 days left before 5/10 (end of day)

So far only 15 bugs are marked as Post-ZBB (documentation bugs, infrastructure issues and couple of late small features based on feedback from community).
We will slip few Networking issues past 5/10.

[karelz]

Quick update on our short-term plans:

• We will likely do the final integration into release/2.0.0 branch on Monday 5/15 (to stabilize infra). All PRs which are "on hold" will be "on hold" until then (3 additional business days against original plan).
• core-setup repo will need more churn afterwards to land some infrastructure work necessary for 2.0. We will lock down core-setup repo (master branch) and either:
  
  
  
  • Consume rel/2.0.0 bits of CoreFX/CoreCLR,/WCF repos in core-setup master, or
  
  
  • Move active work in core-setup into rel/2.0.0 and then integrate it back into master.
  
  

[karelz]

We are very close to ZBB - few more hours left.
Remaining: 27 bugs for ZBB: CoreFX 13 + CoreCLR 1 + core-setup 6 + WCF 7

90 issues have ZBB exception from theses reasons:

• Marked as Post-ZBB (documentation, infrastructure) - currently 21
• area-Infrastructure (does not directly affect product) – currently 45
• area-System.Net* (known exception – the area will slip past ZBB) – currently 17
• area-Tizen (issues tracked by Samsung for 2.0) – currently 12

Only 14 issues without milestone - except one, all of them created in last 24 hours.

[karelz]

Yesterday we have released .NET Core 2.0 preview1: https://github.com/dotnet/core/issues/640
If you can, please try it out. Report any issues back to us ASAP (ideally in next week), so that we can fix it for final release.

[karelz]

We’re down to last 12 issues for ZBB. Almost there!

• core-setup - 7
• CoreFX repo - 4 - 1 issue has pending PR
• CoreCLR - 1
• wcf - 0

Congratulations to WCF repo to hitting ZBB first!

[karelz]

What's next, after ZBB?
We will keep aggressively triaging incoming issues (i.e. set milestone), to make sure we react to feedback quickly.
If you see anything blocking final release, please ping area owners, or @karelz / @danmosemsft.

Currently we have 15 issues without milestone:

• CoreCLR repo - 7 (all <1d old)
• Core-setup repo - 4 (all <2d old)
• CoreFX repo - 3 (only 1 issue is 4d old)
• wcf repo - 1 (<1d old)

[garfbradaz]

Core-setup repo - 4 (all <2d old)

404 on link :)

[tarekgh]

@garfbradaz I fixed the link.

[karelz]

Last 5 issues for ZBB (queries restricted to issues created before 5/10). Almost there!

• CoreFX - 3
• core-setup - 1
• CoreCLR - 1
• wcf - 0

[karelz]

Last 3 issues for ZBB (queries restricted to issues created before 5/10). Getting closer and closer!

• CoreFX - 1 - dotnet/corefx#19179 (PR merged, blocked on CoreCLR update, additional question about test coverage)
• core-setup - 1 - https://github.com/dotnet/core-setup/issues/2334
• CoreCLR - 1 - https://github.com/dotnet/coreclr/issues/11453 (in PR, blocked on CoreCLR update)
• wcf - 0

[karelz]

Here are 2.0 bugs filed after 5/10 - some of them by community and some of them even fixed right away!

• CoreFX - 18
• CoreCLR - 11
• core-setup - 2
• wcf - 1

Community contributions (excluding Microsoft and Samsung employees):

• Fixed: dotnet/corefx#19710 (@slozier), dotnet/corefx#19712 (@NickCraver), dotnet/corefx#19673 (@FransBouma), dotnet/corefx#19651 (@RickStrahl)
  
  
  
  • Shout out to @justinvp for fixing dotnet/corefx#19651 - thanks!
  
  
• Still opened: dotnet/corefx#19708 (@NickCraver), dotnet/corefx#19694 (@yaakov-h), https://github.com/dotnet/coreclr/issues/11578 (@slozier)

[tarekgh]

dotnet/coreclr#11578 (@slozier) still opened because we need to add the test for the fixed case. We are waiting to get successful coreclr package in corefx repo.

[karelz]

We have hit ZBB (zero 2.0 bugs filed before 5/10) !!! Hooray!!!

• CoreFX repo - 0
• CoreCLR repo - 0
• core-setup repo - 0
• wcf repo - 0

Congratulations to the teams driving hard to get here! And to all contributors submitting fixes and helping us validate the bits!

Announced here: https://twitter.com/ziki_cz/status/864614714847903744
Pretty graphs here: https://twitter.com/ziki_cz/status/864679857472454656 (note graphs capture also 20 extra wcf bugs in milestone S118 which are not part of 2.0 - the graphs will be adjusted in next update)

We awarded a few exceptions along the way (one-off Post-ZBB label, and 3 areas: Networking, Infrastructure and Tizen). Here are details:

| Repo | Post-ZBB | Created before 5/10 | Remaining
work: | All 2.0 bugs | No milestone bugs
|----------------|-----------|------------------------|-------------------|--------------|-----------------------
| CoreFX | 14 | Networking - 4
Infrastructure - 2
Tizen - 1 | | 44 | 19
| CoreCLR | 8 | Infrastructure - 3
Tizen - 9 | | 30 | 13
| core-setup | 1 | Infrastructure - 26
Tizen - 3 | | 35 | 7
| wcf | 0 | 0 | | 0 | 0

We also started receiving high-value bug reports from people trying out Preview 1 - thanks everyone for your help!

What's next for .NET Core?

We are in "ask mode" until 5/26 (1.5 weeks), i.e. high-impactful bug fixes can get in, we have to ask shiproom (risk vs. value will be assessed).
Starting 5/27, we will be in "Escrow mode", i.e. the bar for bug fixes goes even higher (to stabilize) and only truly important fixes (show-stoppers) will get in.
Every change will require shiproom discussion if taking it is worth the risk of "resetting Escrow". In practice it means rel/2.0.0 branch will be locking down for final release.

[karelz]

Latest update on 2.0 bugs -- see graphs in https://twitter.com/ziki_cz/status/866981185267707904

• Note: Our overall bug count went up, we will have to bring it down again. Documentation bugs will carry over into Escrow.

Last week of Ask mode, then we are in Escrow mode (i.e. truly impactful bugs only, the bar will be high)

• CoreFX - 67
• CoreCLR - 33
• core-setup - 15
• wcf - 3

Thanks everyone for filing great bugs! Highlights:

• We've got a stream of bugs in System.Data.SqlClient, we adjusted investment in the area and asked everyone for help -- see dotnet/corefx#19970 for details.
• We've got great stream of compat bugs for HttpListener by @hughbe (he created tests, filed 28 bugs and submitted some fixes - thanks a lot @hughbe!)

Overall 2.0 bugs filed by community:

• System.Net (HttpListener) - 24 bugs
• System.Data.SqlClient - 8 bugs
• System.Net - 4 bugs
• System.Security - 3 bugs
• System.Runtime - 3 bugs
• various areas - 6 bugs

[yaakov-h]

@karelz that twitter link 404s.

[karelz]

I deleted the original tweet couple of minutes after posting the GitHub reply, I immediately retweeted it with fixed typo.
I updated GH to the new link right away, so you likely clicked the old link from email notification. Here it is again: https://twitter.com/ziki_cz/status/866981185267707904

[karelz]

Thanks everyone for your help with ZBB and post-ZBB work. We are now deep in Escrow - i.e. taking only the most critical bug fixes.

I don't think we need this announcement issue around anymore.