Enhancements: Scheduling Framework

[bsalamat]

Feature Description

• One-line feature description (can be used as a release note): Implement a scheduling framework where many of the existing and future features of the existing scheduler can be written as plug-ins for the scheduler.
• Primary contact (assignee): @bsalamat
• Responsible SIGs: scheduling
• Design proposal link (KEP): https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/20180409-scheduling-framework.md
• Link to e2e and/or unit tests:
• Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred: @k82cn @misterikkit
• Approver (likely from SIG/area to which feature belongs): @k82cn @bsalamat
• Feature target (which target equals to which milestone):
  
  
  
  • Alpha release target (x.y) 1.15
  
  
  • Beta release target (x.y) 1.17
  
  
  • Stable release target (x.y)
  
  

This is a long term project and requires many changes to the existing scheduler. In order to ensure quality and reliability of the scheduler, we plan to build the features gradually and in steps. The first extension points are targeted for 1.13. We will add more extension points in the future releases. We will prioritize the addition of the extension points based on need for them.

/sig scheduling

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[mlmhl]

/remove-lifecycle stale

[claurence]

@bsalamat Hello - I’m the enhancement’s lead for 1.14 and I’m checking in on this issue to see what work (if any) is being planned for the 1.14 release. Enhancements freeze is Jan 29th and I want to remind that all enhancements must have a KEP - I don't see a relevant KEP for this issue can you drop the link to the KEP? Thanks

[bsalamat]

KEP for this enhancement is found at: https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/20180409-scheduling-framework.md

[kacole2]

I'm the Enhancement Lead for 1.15. Is this feature going to be graduating alpha/beta/stable stages in 1.15? Please let me know so it can be tracked properly and added to the spreadsheet.

Once coding begins, please list all relevant k/k PRs in this issue so they can be tracked properly.

[justaugustus]

/assign @bsalamat

[bsalamat]

@kacole2 Yes, this feature is going to be alpha in 1.15.
One PR which was merged a while back is referenced above is: https://github.com/kubernetes/kubernetes/pull/70227
Another PR which will be merged soon: https://github.com/kubernetes/kubernetes/pull/75848
I would expect to see a number of more PRs in the new future. I will make sure that they reference this issue.

[kacole2]

/milestone v1.15
/stage alpha

[kacole2]

@bsalamat, After reviewing your KEP, it's currently in a draft state. It must be "implementable" to be included in 1.15. In addition, it's missing test plans and graduation criteria which is required information per the KEP Template. Please update the KEP to include the required information before the Kubernetes 1.15 Enhancement Freeze date of 4/30/2019.

[bsalamat]

@kacole2 Thanks for the reminder. I just sent #1026 to add test plans, graduation criteria, and change status to 'implementable'.

[simplytunde]

Hey @bsalamat Just a friendly reminder we're looking for a PR against k/website (branch dev-1.15) due by Thursday, May 30. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

[bsalamat]

@simplytunde I created PR for adding documentation to the website. It is not complete, but I will finish it in the next couple of weeks: https://github.com/kubernetes/website/pull/14388

[kacole2]

Hi @bsalamat. Code Freeze is Thursday, May 30th 2019 @ EOD PST. All enhancements going into the release must be code-complete, including tests, and have docs PRs open.

Please list all current k/k PRs so they can be tracked going into freeze. If the PRs aren't merged by freeze, this feature will slip for the 1.15 release cycle. Only release-blocking issues and PRs will be allowed in the milestone.

I see these issues tagged here but not sure which are required by this issue to be merged:
kubernetes/kubernetes#77998
kubernetes/kubernetes#78001
kubernetes/kubernetes#78002
kubernetes/kubernetes#78004
kubernetes/kubernetes#78006
kubernetes/kubernetes#77566

If you know this will slip, please reply back and let us know. Thanks!

[kacole2]

Hi @bsalamat, today is code freeze for the 1.15 release cycle. I do not see a reply for any k/k PRs to track for this merge. However, I did notice these two have been merged previously: https://github.com/kubernetes/kubernetes/pull/70227 and https://github.com/kubernetes/kubernetes/pull/75848. Are there any others that are missing? After this point, only release-blocking issues and PRs will be allowed in the milestone with an exception.

[draveness]

Hi @kacole2, I believe the following merged PRs are related to this feature.

• https://github.com/kubernetes/kubernetes/pull/77567 Post-bind extension point
• https://github.com/kubernetes/kubernetes/pull/77559 Permit extension point
• https://github.com/kubernetes/kubernetes/pull/77529 Queue-sort extension point
• https://github.com/kubernetes/kubernetes/pull/77501 Scheduling Framework Configuration
• https://github.com/kubernetes/kubernetes/pull/77598 Unreserve extension point (alpha)
• kubernetes/kubernetes#70227 Reserve and Prebind extension point (alpha)

And according to the KEP, only three extension points need to be built in alpha stage(1.15) which has already been merged into master.

• Extension points for Reserve, Unreserve, and Prebind are built.
• Integration tests for these extension points are added

cc/ @bsalamat, please correct me if there is something wrong. thanks :)

[kacole2]

@draveness this is remaining on the tracking sheet for 1.15. All good here.

[kacole2]

Hi @draveness @bsalamat, I'm the 1.16 Enhancement Lead/Shadow. Is this feature going to be graduating alpha/beta/stable stages in 1.16? Please let me know so it can be added to the 1.6 Tracking Spreadsheet. If not's graduating, I will remove it from the milestone and change the tracked label.

Once coding begins or if it already has, please list all relevant k/k PRs in this issue so they can be tracked properly.

Milestone dates are Enhancement Freeze 7/30 and Code Freeze 8/29.

Thank you.

[bsalamat]

Hi @kacole2. We are graduating this feature to beta in 1.16. There are many PRs to implement the feature. Tracking issues for all those PRs are referenced in this issue.

[VineethReddy02]

Hi @bsalamat, I'm the v1.16 docs release shadow.

Does this enhancement require any new docs (or modifications)?

Just a friendly reminder we're looking for a PR against k/website (branch dev-1.16) due by Friday,August 23rd. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

Thanks!

[bsalamat]

Hi @VineethReddy02,
This feature had a document already: https://kubernetes.io/docs/concepts/configuration/scheduling-framework/

We initially wanted to move the feature to beta in 1.16, but we decided to postpone promotion to beta to 1.17 in order to have higher confidence about stability of the API.

[kacole2]

/milestone v1.17

[mrbobbytables]

Hey there @bsalamat -- 1.17 Enhancements lead here. I wanted to check in and see if you think this Enhancement will be graduating to beta in 1.17?

The current release schedule is:

• Monday, September 23 - Release Cycle Begins
• Tuesday, October 15, EOD PST - Enhancements Freeze
• Thursday, November 14, EOD PST - Code Freeze
• Tuesday, November 19 - Docs must be completed and reviewed
• Monday, December 9 - Kubernetes 1.17.0 Released

If you do, once coding begins please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

Thanks!

[bsalamat]

Hi @mrbobbytables,
@ahg-g is the new SIG Scheduling lead. He will update this issue with the new plan.

[ahg-g]

Hi @mrbobbytables, we will discuss this during the next sig meeting, there is a possibility that graduating the framework to Beta will be delayed until the next release.

[palnabarun]

Hi @ahg-g, -- 1.17 enhancements shadow here.

Just a friendly reminder. We are just 5 days away from the Enhancements Freeze.

Please let us know if the graduation to beta will happen in this release.

[ahg-g]

Hi @palnabarun, the framework will not graduate to Beta in 1.17

[palnabarun]

Okay. Thanks for the update. I will be removing this from the tracking sheet.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[kikisdeliveryservice]

Hey there @ahg-g -- 1.18 Enhancements shadow here. I wanted to check in and see if you think this Enhancement will be graduating to [alpha|beta|stable] in 1.18?
The current release schedule is:
Monday, January 6th - Release Cycle Begins
Tuesday, January 28th EOD PST - Enhancements Freeze
Thursday, March 5th, EOD PST - Code Freeze
Monday, March 16th - Docs must be completed and reviewed
Tuesday, March 24th - Kubernetes 1.18.0 Released
To be included in the release, this enhancement must have a merged KEP in the implementable status. The KEP must also have graduation criteria and a Test Plan defined.
If you would like to include this enhancement, once coding begins please list all relevant k/k PRs in this issue so they can be tracked properly. 👍
We'll be tracking enhancements here: http://bit.ly/k8s-1-18-enhancements
Thanks!

[draveness]

/remove-lifecycle stale

[ahg-g]

The framework will not graduate to Beta in 1.18

[kikisdeliveryservice]

thanks @ahg-g for the update!

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[draveness]

/remove-lifecycle stale
/lifecycle frozen

[fejta-bot]

Enhancement issues opened in kubernetes/enhancements should never be marked as frozen.
Enhancement Owners can ensure that enhancements stay fresh by consistently updating their states across release cycles.

/remove-lifecycle frozen

[harshanarayana]

Hey there @ahg-g, 1.19 Enhancements shadow here. I wanted to check in and see if you think this Enhancement will be graduating in 1.19?

In order to have this part of the release:

1. The KEP PR must be merged in an implementable state
2. The KEP must have test plans
3. The KEP must have graduation criteria.

The current release schedule is:

• Monday, April 13: Week 1 - Release cycle begins
• Tuesday, May 19: Week 6 - Enhancements Freeze
• Thursday, June 25: Week 11 - Code Freeze
• Thursday, July 9: Week 14 - Docs must be completed and reviewed
• Tuesday, August 4: Week 17 - Kubernetes v1.19.0 released

If you do, I'll add it to the 1.19 tracking sheet (http://bit.ly/k8s-1-19-enhancements). Once coding begins please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

Thanks!

[harshanarayana]

Hey @ahg-g, I'm following up on my previous update on this Enhancement being part of the v1.19 release.

Do you happen to have any update on the possiblity of this being included in the release v1.19?

Thanks again for your time and contributions. 🖖

[ahg-g]

Hi @harshanarayana, sorry for the late reply, no this is not going to graduate as part of 1.19.

[harshanarayana]

Hey @ahg-g, thanks for following up on this with an update. I have update the tracking sheet accordingly. 🖖

Thanks

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[ahg-g]

/remove-lifecycle stale

[kikisdeliveryservice]

Hi @ahg-g

Enhancements Lead here. Are there any plans to graduate this to beta in 1.20?

Thanks!
Kirsten

[ahg-g]

Hi kikisdeliveryservice@, this feature is not guarded by a feature flag and there is really no notion of versions for this api yet. We may declare this as implemented and graduated in 1.20 and close this issue.

[kikisdeliveryservice]

Thanks @ahg-g ! For now I will track this enhancement and just keep us updated :+1:

Best,
Kirsten

[kikisdeliveryservice]

Hi @ahg-g

Any update as to whether this is going to get marked as implemented? I see that you got some agreement on doing so, but want to check.

Thanks
Kirsten

[ahg-g]

I opened an issue for discussion: https://github.com/kubernetes/kubernetes/issues/95037 there doesn't seem to be any objection so far, I will wait few more days and then send a PR to change the KEP to implemented.

[kikisdeliveryservice]

Thanks for the update @ahg-g !!

[ahg-g]

PR for marking this as implemented: https://github.com/kubernetes/enhancements/pull/2054

[kikisdeliveryservice]

The above PR merged marking 624 as implemented in 1.19.

Please feel free to close this issue.

Thanks @ahg-g !!