Enhancements: Priority and Fairness for API Server Requests

[yue9944882]

Enhancement Description

• One-line enhancement description (can be used as a release note): Priority and Fairness for API Server Requests
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/1040-priority-and-fairness
• Primary contact (assignee): @yue9944882 @MikeSpreitzer
• Responsible SIGs: sig-api-machinery
• Enhancement target (which target equals to which milestone):
  
  
  
  • Alpha release target (x.y) 1.17
  
  
  • Beta release target (x.y) 1.20
  
  
  • Stable release target (x.y) TBD
  
  

_Please to keep this description up to date. This will help the Enhancement Team track efficiently the evolution of the enhancement_

[kacole2]

@yue9944882 the 1.15 enhancement freeze was on 4/30. The KEP is still in a provisional state and still requires graduation criteria. This will not make it in 1.15 without filing an exception and fixing what I had just mentioned.

[kacole2]

/milestone v1.15
/sig api-machinery
/stage alpha

[makoscafee]

Hey, @yue9944882 👋 I'm the v1.15 docs Lead.
Does this enhancement require any new docs (or modifications)?

Just a friendly reminder we're looking for a PR against k/website (branch dev-1.15) due by Thursday, May 30th. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

[kacole2]

Hi @yue9944882 @MikeSpreitzer . Code Freeze is Thursday, May 30th 2019 @ EOD PST. All enhancements going into the release must be code-complete, including tests, and have docs PRs open.

Please list all current k/k PRs so they can be tracked going into freeze. If the PRs aren't merged by freeze, this feature will slip for the 1.15 release cycle. Only release-blocking issues and PRs will be allowed in the milestone.

If you know this will slip, please reply back and let us know. Thanks!

[kacole2]

Hi @yue9944882 @MikeSpreitzer , today is code freeze for the 1.15 release cycle. I do not see a reply for any k/k PRs to track for this merge. It's now being marked as At Risk in the 1.15 Enhancement Tracking Sheet. If there is no response, or you respond with PRs to track and they are not merged by EOD PST, this will be dropped from the 1.15 Milestone. After this point, only release-blocking issues and PRs will be allowed in the milestone with an exception.

[kacole2]

/milestone clear

[evillgenius75]

Hi @MikeSpreitzer and @yue9944882 , I'm a 1.16 Enhancement Shadow. Is this feature going to be graduating to alpha stage in 1.16? Please let me know so it can be added to the 1.16 Tracking Spreadsheet. If not's graduating, I will remove it from the milestone and change the tracked label.

I see you have PR kubernetes/kubernetes#78966 referenced above, please include any other relevant k/k PRs in this issue so they can be tracked properly.

Milestone dates are Enhancement Freeze 7/30 and Code Freeze 8/29.

Thank you.

[yue9944882]

hey, yes please track the feature in the 1.16 milestone. matter of fact this feature is literally active from 1.15 on, as you see in the pull above there's ~300 discussions over the api model since april this year. there're also other poc repos under development:

• https://github.com/aaron-prindle/fq-inflight-rate-limit
• https://github.com/aaron-prindle/fq-apiserver
• https://github.com/mars1024/shuffle-sharding

we will get into 1.16 this time 👏🏻

[kacole2]

/milestone v1.16

[sethmccombs]

Hey @yue9944882 @MikeSpreitzer,

I'm one of the v1.16 docs shadows.
Does this enhancement (or the work planned for v1.16) require any new docs (or modifications to existing docs)? If not, can you please update the 1.16 Enhancement Tracker Sheet (or let me know and I’ll do so)

If so, just a friendly reminder we're looking for a PR against k/website (branch dev-1.16) due by Friday, August 23rd, it can just be a placeholder PR at this time. Let me know if you have any questions!

[MikeSpreitzer]

@evillgenius75 : Yes, this feature needs docs. I do not see how to update the tracking spreadsheet myself; is that something I leave to you?

@sethmccombs : please see my plea for help on the sig-docs Slack channel of the Kubernetes workspace

[kacole2]

@MikeSpreitzer @yue9944882 code freeze for 1.16 is on Thursday 8/29. Are there any outstanding k/k PRs that still need to be merged for this to go Alpha?

[MikeSpreitzer]

Yes. At least #81834. I think @yue9944882 is planning a revision to that PR. I think there may be a little more want to do after that.

[kacole2]

Hi @MikeSpreitzer @yue9944882 it looks as though https://github.com/kubernetes/kubernetes/pull/81834 didn't merge before code freeze and it's not in the Tide Merge Pool. This feature is going to be bumped from v1.16. If you would still like to have this be a part of the 1.16 release, please file an exception

[MikeSpreitzer]

What about the doc side of things? Isn't there some undoing needed there too?

[kacole2]

@simplytunde can you comment WRT to docs?

[simplytunde]

@kacole2 @MikeSpreitzer The docs PR needs to be closed.

[jeremyrickard]

Hey there @MikeSpreitzer -- 1.17 Enhancements shadow here 👋 . I wanted to check in and see if you think this Enhancement will be graduating to alpha in 1.17?

The current release schedule is:

• Monday, September 23 - Release Cycle Begins
• Tuesday, October 15, EOD PST - Enhancements Freeze
• Thursday, November 14, EOD PST - Code Freeze
• Tuesday, November 19 - Docs must be completed and reviewed
• Monday, December 9 - Kubernetes 1.17.0 Released

If you do, I'll add it to the 1.17 tracking sheet (https://bit.ly/k8s117-enhancements). Once coding begins please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

[yue9944882]

@jeremyrickard thanks, we are targeting 1.17 this time, the implementations are almost finished in 1.16 but we missed it out of some reason

[jeremyrickard]

Thanks @yue9944882. I'll flip the status in our tracking spreadsheet and add it to the milestone.

[jeremyrickard]

/milestone v1.17

[daminisatya]

Hello, @yue9944882 @MikeSpreitzer I'm 1.17 docs lead.

Does this enhancement (or the work planned for v1.17) require any new docs (or modifications to existing docs)? If not, can you please update the 1.17 Enhancement Tracker Sheet (or let me know and I'll do so)

If so, just a friendly reminder we're looking for a PR against k/website (branch dev-1.17) due by Friday, November 8th, it can just be a placeholder PR at this time. Let me know if you have any questions!

[yue9944882]

@daminisatya we have got a doc PR opened from the last milestone https://github.com/kubernetes/website/pull/15899, we will revive this one to file user-facing documentation to the kubernetes site.

[jeremyrickard]

Hey, @yue9944882 @MikeSpreitzer

Jeremy from the 1.17 enhancements team here 👋

We're closing in on the 1.17 code freeze pretty soon (Nov 14th) so I wanted to check in and see how this was going? You mentioned before that it was mostly implemented for 1.16, could you please let us know if you'll be finishing for 1.17? It looks like there are a few incomplete items in your umbrella tracking issue: https://github.com/kubernetes/kubernetes/issues/76846

Thanks in advance!

[daminisatya]

Hey @yue9944882 @MikeSpreitzer

Just a friendly reminder, we're hoping to have a placeholder Docs PR against k/website (branch dev-1.17) by Friday, Nov 8th. (4 more days left)

[daminisatya]

Hello @yue9944882 @MikeSpreitzer

Just a friendly reminder, We're hoping to have a placeholder Docs PR against k/website (branch dev-1.17) by Friday, Nov 8th. (2 more days left)

[jeremyrickard]

Hi @yue9944882 @MikeSpreitzer, today is code freeze for the 1.17 release cycle. It looks like the k/k PRs have not yet been merged. We're flagging as At Risk in the 1.17 Enhancement Tracking Sheet.
Do you think they will be merged by the EoD of the today? After that point, only release-blocking issues and PRs will be allowed in the milestone with an exception.

[jeremyrickard]

Hey @yue9944882 @MikeSpreitzer Code freeze is now in effect for the 1.17 release. It doesn't look like the some of the PRs in your umbrella issue were added to a merge pool in time. I'm going to go ahead and remove this from the milestone, if you feel it is release blocking or urgent that it gets merged, please file an exception request with @guineveresaenger. Thanks!

[jeremyrickard]

/milestone clear

[jeremyrickard]

If you'd like to file an exception, please see here

[jeremyrickard]

Hi @yue9944882 @MikeSpreitzer,

1.18 enhancements team reaching out 👋 It looks like this just missed the code freeze last time, so I wanted to check and see if you're planning on introducing this in 1.18?

Thanks in advance!

[yue9944882]

@jeremyrickard the feature has been partially merged into master branch and we're moving on to 1.18. plz mark us as tracked, thanks

[jeremyrickard]

/milestone v1.18

[iheanyi1]

Hello @yue9944882 ,

I'm 1.18 docs shadow.

Just want to know if this enhancement work planned for 1.18 require any new docs (or modifications to existing docs)? If not, can you please update the 1.18 Enhancement Tracker Sheet (or let me know and I'll do so)

If so, just a friendly reminder we're looking for a PR against k/website (branch dev-1.18) due by Friday, Feb 28th, it can just be a placeholder PR at this time. Let me know if you have any questions!

Warm regards,

chima

[MikeSpreitzer]

Yes, we plan to add doc. This is a new feature and definitely requires doc.

[johnbelamaric]

Hi @MikeSpreitzer I am an enhancements shadow for 1.18. Are there open PRs that are needed to complete the 1.18 work for this? If so, please list them here so we can track them in the release team. Thanks.

[MikeSpreitzer]

We have one open PR, https://github.com/kubernetes/kubernetes/pull/87754 . That will be the first time that the plumbing for this feature is complete. We expect to do further polishing before the release, as time allows.

[johnbelamaric]

Thanks Mike, we'll track that one. If others come up, please link them here.

[iheanyi1]

Yes, we plan to add doc. This is a new feature and definitely requires doc.
Hi @MikeSpreitzer ,
Remember your update quoted above, just a friendly reminder we're looking for a PR against k/website (branch dev-1.18) due by Friday, Feb 28th, it can just be a placeholder PR at this time.
Keep the good work, we hope you will meet deadline we are working with.
Thank you.

[VineethReddy02]

Hi @MikeSpreitzer
My name is Vineeth, 1.18 Docs release lead. This message is in regard to docs placeholder PR for this enhancement.
As we are left with 2 more days for the deadline. It would be great to have placeholder PR in place asap.
Thanks!

[MikeSpreitzer]

We had a doc PR: https://github.com/kubernetes/website/pull/17463 , but it got closed because it wasn't ready in time for 1.17.
We have a new independent doc PR: https://github.com/kubernetes/website/pull/19319

[palnabarun]

/milestone clear

(removing this enhancement issue from the v1.18 milestone as the milestone is complete)

[msedzins]

Hey there @yue9944882 , @MikeSpreitzer -- 1.19 Enhancements shadow here. I wanted to check in and see if you think this Enhancement will be graduating in 1.19?

In order to have this part of the release:

1. The KEP PR must be merged in an implementable state
2. The KEP must have test plans
3. The KEP must have graduation criteria.

The current release schedule is:

• Monday, April 13: Week 1 - Release cycle begins
• Tuesday, May 19: Week 6 - Enhancements Freeze
• Thursday, June 25: Week 11 - Code Freeze
• Thursday, July 9: Week 14 - Docs must be completed and reviewed
• Tuesday, August 4: Week 17 - Kubernetes v1.19.0 released

If you do, I'll add it to the 1.19 tracking sheet (http://bit.ly/k8s-1-19-enhancements). Once coding begins please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

Thanks!

[yue9944882]

@msedzins thanks, the alpha version is already merged/announced and we're working on moving the feature to beta stage in 1.19. plz add us to the tracking list.

[msedzins]

Awesome, thank you! I'll go ahead and add it to the tracking sheet :)

[msedzins]

One more thing:

please remember about requirement list, specifically point 2:
"The KEP must have graduation criteria."
Should cover beta stage.

[palnabarun]

/stage beta
/milestone v1.19

[msedzins]

Hi @yue9944882 , @MikeSpreitzer ,

Note that recently the KEP format has changed.

1620 merged recently, adding production readiness review questions to the KEP template. Please take this opportunity to reformat your KEP and also answer the questions add to the template in that PR.

Also please remember about graduation criteria.

Thanks!

[msedzins]

Hi @yue9944882 , @MikeSpreitzer ,

Please not that enhancement freeze is on Tuesday, May 19.
There are still not graduation criteria in KEP.

Regards,
Mirek

[yue9944882]

hi @msedzins, there's an opening pull adding beta critiria to the KEP https://github.com/kubernetes/enhancements/pull/1632. do we need to merge it before the freeze?

[kikisdeliveryservice]

@yue9944882 yes graduation criteria is required in the merged KEP.

[kikisdeliveryservice]

Unfortunately the deadline for the 1.19 Enhancement freeze has passed. The existing KEP is missing graduation criteria for beta which is what was being targeted for 1.19. It looks like the open PR #1632 has some unaddressed comments and is not approved/has not merged.

For now this is being removed from the milestone and 1.19 tracking sheet. If there is a need to get this in, please file an enhancement exception.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[yue9944882]

/remove-lifecycle stale

[kikisdeliveryservice]

Hi @yue9944882 !

Enhancements Lead here, do you still intend to go beta in 1.20?

Thanks!
Kirsten

[yue9944882]

@kikisdeliveryservice thanks for reminding, and we're moving toward 1.20 as discussed in last week's sync'up.

[kikisdeliveryservice]

Thanks for the update @yue9944882

If you could please update the description of this issue to reflect this and also ensure that your kep is in the proper format as per :https://github.com/kubernetes/enhancements/tree/master/keps/NNNN-kep-template (it seems to be missing a kep.yaml) that would be great.

Best,
Kirsten

[kikisdeliveryservice]

/milestone v1.20

[MorrisLaw]

Hi @yue9944882 -- 1.20 Enhancements Shadow here 👋

Friendly ping to remind you that the Enhancements Freeze deadline is October 6th.

Please update the KEP to match the new format detailed in https://github.com/kubernetes/enhancements/tree/master/keps/NNNN-kep-template.

Appreciate your work on this!

Regards,
Jeremy

[MorrisLaw]

Hi @yue9944882 ,

Since your Enhancement is scheduled to be in 1.20, please keep in mind the important upcoming dates:
Friday, Nov 6th: Week 8 - Docs Placeholder PR deadline
Thursday, Nov 12th: Week 9 - Code Freeze

As a reminder, please link all of your k/k PR as well as docs PR to this issue so we can track them.

Best,
Jeremy

[reylejano-rxm]

Hello @MikeSpreitzer @yue9944882 , 1.20 Docs shadow here.

Does this enhancement work planned for 1.20 require any new docs or modification to existing docs?

If so, please follows the steps here to open a PR against the dev-1.20 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Nov 6th.

Also take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.

Thank you!

[reylejano-rxm]

Hi @MikeSpreitzer @yue9944882

The docs placeholder deadline is almost here. Please make sure to create a placeholder PR against the dev-1.20 branch in the k/website before the deadline

Also, please keep in mind the important upcoming dates:

• Friday, Nov 6th: Week 8 - Docs Placeholder PR deadline
• Thursday, Nov 12th: Week 9 - Code Freeze
• Monday, Nov 23rd: Week 11 - Docs PR Ready for Review

As a reminder, please link all of your k/k PR as well as docs PR to this issue for the release team to track.

[yue9944882]

@reylejano-rxm i created a placeholder PR here: https://github.com/kubernetes/website/pull/24877

[MorrisLaw]

Hi @MikeSpreitzer @yue9944882

The code freeze deadline is almost here. Please make sure to link k/k PRs here so that we can track them and make sure to have them merged by the code freeze deadline or an exception will need to be made to keep this in the 1.20 milestone.

Also, please keep in mind the important upcoming dates:

• Thursday, Nov 12th: Week 9 - Code Freeze
• Monday, Nov 23rd: Week 11 - Docs PR Ready for Review
  As a reminder, please link all of your k/k PR as well as docs PR to this issue for the release team to track.

[MikeSpreitzer]

We want to get this feature to beta in 1.20. That means we need

• https://github.com/kubernetes/kubernetes/pull/96213 - Promoting the APF API to beta

The following PRs would be nice to have but are not considered blockers:

• https://github.com/kubernetes/kubernetes/pull/91389 - Introduce concurrency limits to PriorityLevelConfigurationSatus
• https://github.com/kubernetes/kubernetes/pull/90541 - Introduce delays in controller fights of FlowSchemaStatus

[MikeSpreitzer]

There is one doc PR:

• https://github.com/kubernetes/website/pull/24975 - update doc to track promotion to beta

[kikisdeliveryservice]

@lavalamp @adtac @MikeSpreitzer @yue9944882 (not sure who to ping here)

https://github.com/kubernetes/kubernetes/pull/96527 is replacing https://github.com/kubernetes/kubernetes/pull/96213 entirely due to test issues correct? And is the only required PR for this to move to beta?

[adtac]

https://github.com/kubernetes/kubernetes/pull/96527 is the last code change PR to graduate this to beta. There is still https://github.com/kubernetes/website/pull/24975 which is open that is pending approval -- that updates the website and docs.

[kikisdeliveryservice]

Hi @adtac

Code Freeze is now in effect. Your PR is still unmerged, and is missing a LGTM. If you believe this should be in the 1.20 Release, please file an Exception.

Best,
Kirsten

[lavalamp]

/milestone 1.20

this had an lgtm earlier and will have one shortly

[k8s-ci-robot]

@lavalamp: The provided milestone is not valid for this repository. Milestones in this repository: [keps-beta, keps-ga, v1.17, v1.18, v1.19, v1.20, v1.21, v1.22, v1.23]

Use /milestone clear to clear the milestone.

In response to this:

/milestone 1.20

this had an lgtm earlier and will have one shortly

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[lavalamp]

Looks like we might need another day after all. We'll file an exception request.

[kikisdeliveryservice]

Looks like we might need another day after all. We'll file an exception request.

I spoke with @adtac earlier tonight and they are planning on filing an exception to have enough time to complete. :smile:

[adtac]

exception request: https://groups.google.com/g/kubernetes-sig-api-machinery/c/o--_PcGMxik

[kikisdeliveryservice]

To circle back around on this: @lavalamp @hasheddan @jeremyrickard @dims

Can I get a signoff that this is now Done? (we were waiting for the outcomes of some tests but I didnt hear an official word)

[jeremyrickard]

I think it's safe to go, we haven't seen any negative impact on the testing AFAICT.

[adtac]

Can I get a signoff that this is now Done?

perhaps I misunderstood, but are you asking if we can close this issue? I'd like to keep this open till the feature becomes GA if that's alright (which won't be for many more months)

[jeremyrickard]

@adtac no more around if we can consider this done for 1.20. Since we had the contigency PR ready to go, we still had this marked as "At Risk" in case we needed to back it out.

[kikisdeliveryservice]

@adtac what Jeremy said above :smile:

Also we keep these issues open until you update the KEP and mark its status as implemented. So you'll need to go GA and then _after_ that is done-done you'll pr the KEP, marking as implemented and then at the end of that release, we go thru and close this issue.

Until then, keep linking all PRs to this.

HTH!