Enhancements: Support CSI plugins in Windows

[ddebroy]

Enhancement Description

• One-line enhancement description (can be used as a release note): In-tree Enhancements and new out-of-tree components (csi-proxy) to support CSI node plugins in Windows worker nodes.
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-windows/20190714-windows-csi-support.md
• Primary contact (assignee): @ddebroy
• Responsible SIGs: sig-storage, sig-windows
• Enhancement target (which target equals to which milestone):
  
  
  
  • Alpha released: 1.16
  
  
  • Beta released: 1.19
  
  
  • Stable release target: (1.20)
  
  

_Please to keep this description up to date. This will help the Enhancement Team track efficiently the evolution of the enhancement_

[ddebroy]

/sig storage

[ddebroy]

/kind feature

[ddebroy]

/sig windows

[kacole2]

/stage alpha
/milestone v1.16

@ddebroy As a reminder, every enhancement requires a KEP in an implementable state with Graduation Criteria explaining each alpha/beta/stable stages requirements. Please link to the KEP when it's created. Once coding begins or if it already has, please list all relevant k/k PRs in this issue so they can be tracked properly.

Thanks

[PatrickLang]

KEP #1141

[mrbobbytables]

@ddebroy @PatrickLang one of the 1.16 Enhancement shadows here -- Just as a general reminder -- the Enhancement freeze is End-of-Day tomorrow (7/30).

The KEP must be merged and in an implementable state with graduation criteria to be considered a part of the release. For now it is being labeled as At Risk and will be dropped from the milestone if it's not merged tomorrow.

Thanks!

[ddebroy]

A provisional version of the KEP has been merged: https://github.com/kubernetes/enhancements/pull/1141 and description updated with a link to it.

I will have the PR to move to implementable by EOD and seek approval from approvers.

[ddebroy]

PR to move KEP to implementable: https://github.com/kubernetes/enhancements/pull/1184

[sethmccombs]

Hey @ddebroy

I'm one of the v1.16 docs shadows.
Does this enhancement (or the work planned for v1.16) require any new docs (or modifications to existing docs)? If not, can you please update the 1.16 Enhancement Tracker Sheet (or let me know and I’ll do so)

If so, just a friendly reminder we're looking for a PR against k/website (branch dev-1.16) due by Friday, August 23rd, it can just be a placeholder PR at this time. Let me know if you have any questions!

[ddebroy]

@sethmccombs, yes there will be docs changes for this feature. I will open a place holder PR to track the docs work against this.

[ddebroy]

1.16 PRs for this in-tree:
https://github.com/kubernetes/kubernetes/pull/81397
out-of-tree:
https://github.com/kubernetes-csi/node-driver-registrar/pull/47
everything under: https://github.com/kubernetes-csi/csi-proxy

[ddebroy]

Placeholder docs PR: https://github.com/kubernetes/website/pull/16017
cc @sethmccombs

[kacole2]

@ddebroy friendly reminder that code freeze for 1.16 is on Thursday 8/29. We will be watching https://github.com/kubernetes/kubernetes/pull/81397 to be merged by then.

[ddebroy]

https://github.com/kubernetes/kubernetes/pull/81397 is now in as well as https://github.com/kubernetes-csi/csi-release-tools/pull/26

[mrbobbytables]

Hey there @ddebroy -- 1.17 Enhancements lead here. I wanted to check in and see if you think this Enhancement will be graduating to alpha/beta/stable in 1.17?

The current release schedule is:

• Monday, September 23 - Release Cycle Begins
• Tuesday, October 15, EOD PST - Enhancements Freeze
• Thursday, November 14, EOD PST - Code Freeze
• Tuesday, November 19 - Docs must be completed and reviewed
• Monday, December 9 - Kubernetes 1.17.0 Released

If you do, please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

/milestone clear

[ddebroy]

@mrbobbytables We plan to stay in Alpha for this in v1.17 and aim for a Beta in v1.18

[mrbobbytables]

Thanks for the quick response. I'll bump it to v1.18 :)

/milestone v1.18

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[ddebroy]

/remove-lifecycle stale

[jeremyrickard]

Hey @ddebroy, 1.18 Enhancements Lead checking in. Are you still targeting this for Beta?

The current release schedule is:
Monday, January 6th - Release Cycle Begins
Tuesday, January 28th EOD PST - Enhancements Freeze
Thursday, March 5th, EOD PST - Code Freeze
Monday, March 16th - Docs must be completed and reviewed
Tuesday, March 24th - Kubernetes 1.18.0 Released

If you do, please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

[ddebroy]

hi @jeremyrickard the plan is to stay Alpha in 1.18

[jeremyrickard]

Thanks @ddebroy!

[palnabarun]

/milestone clear

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[palnabarun]

/remove-lifecycle stale

[harshanarayana]

Hey there @ddebroy, 1.19 Enhancements shadow here. I wanted to check in and see if you think this Enhancement will be graduating in 1.19?

In order to have this part of the release:

1. The KEP PR must be merged in an implementable state
2. The KEP must have test plans
3. The KEP must have graduation criteria.

The current release schedule is:

• Monday, April 13: Week 1 - Release cycle begins
• Tuesday, May 19: Week 6 - Enhancements Freeze
• Thursday, June 25: Week 11 - Code Freeze
• Thursday, July 9: Week 14 - Docs must be completed and reviewed
• Tuesday, August 4: Week 17 - Kubernetes v1.19.0 released

If you do, I'll add it to the 1.19 tracking sheet (http://bit.ly/k8s-1-19-enhancements). Once coding begins please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

Thanks!

[ddebroy]

hey @harshanarayana, yes, we are planning to graduate this feature to Beta in 1.19.

[harshanarayana]

@ddebroy Thanks for following up with an update on this. I have updated the tracking sheets accordingly. 👍

/milestone v1.19
/stage beta

[k8s-ci-robot]

@harshanarayana: You must be a member of the kubernetes/milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your and have them propose you as an additional delegate for this responsibility.

In response to this:

@ddebroy Thanks for following up with an update on this. I have updated the tracking sheets accordingly. 👍

/milestone v1.19
/stage beta

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[palnabarun]

/milestone v1.19

[harshanarayana]

Hi @ddebroy, member of Enhancements team for the v1.19 here with a quick update.

PR #1620 was merged recently, adding production readiness review questions to the KEP template. We are not making it mandatory for the 1.19 release cycle, but it would be great if the PRR questionnaire is filled in the KEP.

[divya-mohan0209]

Hello @ddebroy , I'm one of the v1.19 docs shadows.

Could you please let me know if this piece of enhancement work planned for 1.19 release requires any new docs (or modifications to existing docs)?

If not, can you please update the 1.19 Enhancement Tracker Sheet (or let me know and I'll do so)

If yes, just a friendly reminder we're looking for a PR against k/website (branch dev-1.19) due by Friday, June 12, it can just be a placeholder PR at this time.

Please do let me know if you have any questions!

Regards,
Divya

[harshanarayana]

Hey @ddebroy, I am with the enhancements team for the v1.19 release cycle as a shadow.

The code freeze deadline for the Enhancement is Thursday, June 25. I am checking in to see if there is any k/k PR that you have already opened for this enhancement and if so, would you be able to point me in the direction of the PR so that the same can be updated in the tracking sheet

Have a wonderful day. 🖖

[divya-mohan0209]

@ddebroy : Gentle reminder on my note above. Do feel free to get in touch if you have any concerns or queries.

[ddebroy]

Hi, all the work around csi-proxy is happening in https://github.com/kubernetes-csi/csi-proxy. We do not anticipate any k/k PRs for 1.19.

[divya-mohan0209]

@ddebroy : This is just to clarify my understanding that there would be no docs required for this release?

[divya-mohan0209]

Hello @ddebroy , please could you clarify the above at your earliest? This is just so that we can ensure it is being tracked appropriately :)

[harshanarayana]

Hi, @ddebroy

This is a follow-up to the communication that went out to k-dev today. There has been a revision to the release schedule of v1.19 as follows.

Thursday, July 9th: Week 13 - Code Freeze
Thursday, July 16th: Week 14 - Docs must be completed and reviewed
Tuesday, August 25th: Week 20 - Kubernetes v1.19.0 released
Thursday, August 27th: Week 20 - Release Retrospective

You can find the revised Schedule in the sig-release Repo

Please let me know if you have any questions. 🖖

[harshanarayana]

Hi @ddebroy, Since there is no k/k anticipated for this Enhancement, I am checking in to see if you can help me with the link to the placeholder PR against the kubernetes-csi/csi-proxy and any other Repo that is applicable, so that the tracking sheet can have a reference to the same.

Please let me know if you have any questions. 🖖

[immuzz]

SIG-Windows Triage

Things to do in K8 1.20

1. Promote this to GA
2. Consider Privileged Containers Impact
3. AWS, VSphere disks etc

[kikisdeliveryservice]

/milestone clear

(removing this enhancement issue from the v1.19 milestone as the milestone is complete)

[kikisdeliveryservice]

Hi @immuzz @ddebroy

Enhancements Lead here. Any plans for any k/k work in 1.20 on this?

Thanks!
Kirsten

[ddebroy]

hi @kikisdeliveryservice, further enhancements around CSI support in Windows through csi-proxy (Beta => GA) are all expected to happen out-of-tree from k/k.

[kikisdeliveryservice]

@ddebroy Thanks, I appreciate the update, do let me know if anything changes. :+1:

[msau42]

@ddebroy @jingxu97 do we have a detailed plan of what items are going to be targeted for 1.20 and beyond?

[ddebroy]

hey @msau42, in the 1.20 timeframe, we do not expect any changes in k/k to enhance support for CSI node plugins for Windows in the existing unprivileged container deployment model. As privileged container support for Windows evolves (https://github.com/kubernetes/enhancements/pull/2037), there may be changes necessary in k/k to support CSI node plugins for Windows deployed using privileged containers. This issue can be used to track enhancements (if necessary) around privileged containers for Windows.

The major remaining items out-of-tree at this point in the context of csi-proxy for Windows are:

1. Enable e2e integration tests for csi-proxy [https://github.com/kubernetes-csi/csi-proxy/pull/69/]
2. Add docs for all existing API groups for csi-proxy under https://kubernetes-csi.github.io/docs/
3. Improve developer documentation inside csi-proxy around creation of new API groups.
4. Enhance the API groups as needed by different drivers (will be an ongoing item).
5. Audit the effects of protoc version bumps when adding new API groups on older client versions.
6. Smooth transition path for CSI plugins that use csi-proxy APIs today to migrate to using privileged containers as that support evolves.

[ddebroy]

We discussed 1.20 plans around CSI plugin support for Windows in SIG Windows [10/06]. The decision is to stay Beta for 1.20 to get a better idea of what changes will be necessary around privileged containers for Windows. Based on the developments around privileged containers in Windows, we want to plan for Stable/GA in 1.21 so we can provide as much clarity/direction as possible to CSI plugin authors around Windows support.

[msau42]

Thanks @ddebroy! Is there a doc/spreadsheet/project board that is being used to plan and track the individual tasks needed? If not, can you or @jingxu97 work on one?

[jingxu97]

I created a Windows GA project to track https://github.com/orgs/kubernetes-csi/projects/36

[marosset]

/milestone v1.21