Enhancements: Support Windows Server Containers for K8s

[michmike]

Description

Microsoft, in collaboration with Docker, is introducing a new feature called Windows Server Containers, which enables Docker containers to run on Windows. SIG-Windows would like to enhance Kubernetes to support Windows Server Containers (and Windows Server 2016 as the operating system of choice for the windows-based kubelet). With this feature, customers will be able to run Windows-based and .NET-based applications inside containers on Kubernetes. Kubernetes will be the first fully functional cross-platform cluster manager.

Feature Description

• One-line feature description (can be used as a release note): Support for Windows Server Containers
• Primary contact (assignee): @michmike
• Responsible SIGs: sig-windows
• Kubernetes Enhancement Proposal (KEP): https://github.com/kubernetes/enhancements/pull/676
• Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred: @brendandburns, @JiangtianLi, @alinbalutoiu
• Approver (likely from SIG/area to which feature belongs): @michmike
• Feature target (which target equals to which milestone):
  
  
  
  • Beta release target 1.9
  
  
  • GA release target 1.14
  
  

SIG-Windows approved GA release criteria: https://docs.google.com/document/d/1YkLZIYYLMQhxdI2esN5PuTkhQHhO0joNvnbHpW68yg8/edit (old)

New KEP release criteria:
https://github.com/kubernetes/enhancements/pull/676

[michmike]

cc: @sarahnovotny, @idvoretskyi, and @kubernetes/sig-node

[magicwang-cn]

@kubernetes/huawei

[luxas]

Relevant doc I wrote: https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/multi-platform.md

[michmike]

thanks for sending us this @luxas

[idvoretskyi]

Marked as "Alpha" per @michmike request.

[idvoretskyi]

@michmike can you confirm that this item targets beta in 1.6?

[michmike]

@idvoretskyi , yes that's right. thanks!
we will be in beta for v1.6

[michmike]

@idvoretskyi , please remove the stage/beta designation from this issue. we will stay in alpha for one more release. thanks!

[idvoretskyi]

@michmike done, thanks!

[davidtnorth]

Hi all - it's been more than one further release - any news of this progressing beyond alpha?

[michmike]

hi @davidtnorth , we are very close to releasing beta. we already got 3-4 PRs in right after the merge queues were opened after the 1.8 release. unfortunately we missed the boat for 1.8, but that means we will have more stability and more features with 1.9

[mabunixda]

Hi all, are there any updates available on kubernetes storage/volumes? Is there some progress going on?

[idvoretskyi]

@michmike please, update the feature description with the new template https://github.com/kubernetes/features/blob/master/ISSUE_TEMPLATE.md

[idvoretskyi]

@michmike also, based on the previous statements I've updated the stage label to beta in 1.9. Can you confirm?

[michmike]

yes, we will beta in v1.9. thanks @idvoretskyi

[michmike]

@idvoretskyi https://github.com/kubernetes/features/issues/512

[zacharysarah]

@michmike :wave: Please indicate in the 1.9 feature tracking board
whether this feature needs documentation. If yes, please open a PR and add a link to the tracking spreadsheet. Thanks in advance!

[zacharysarah]

@michmike Bump for docs ☝️

/cc @idvoretskyi

[michmike]

@zacharysarah here's our PR. i added it to the docs as well.
https://github.com/kubernetes/website/pull/6180

However, one small issue. Even though the PR is off kuberneteswebsite, it is off master and not the 1.9 release branch. is that an issue?
cc: @JMesser81

[zacharysarah]

@michmike Thanks for opening the PR. 👍

Yes: the PR needs to base off of release-1.9. The kubernetes/website repo has a different relationship with master than other K8s repos.

UPDATE: I changed the base, it looks fine.

[michmike]

yes, thank you for the update @zacharysarah . i noticed you did it!

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

[idvoretskyi]

@michmike @kubernetes/sig-windows-feature-requests this feature is still under active development, right?

[michmike]

yes this is still under active development. we are likely to get it out of beta in the next few months.

[michmike]

/remove-lifecycle stale

[michmike]

/remove-lifecycle rotten

[justaugustus]

@michmike
Any plans for this in 1.11?

If so, can you please ensure the feature is up-to-date with the appropriate:

• Description
• Milestone
• Assignee(s)
• Labels:
  
  
  
  • stage/{alpha,beta,stable}
  
  
  • sig/*
  
  
  • kind/feature
  
  

cc @idvoretskyi

[derekwaynecarr]

@PatrickLang is this an appropriate issue to track what was discussed in sig-node to propose GA for windows containers support? i cant find a better one, so added to milestone with node label.

[PatrickLang]

@derekwaynecarr Sure, this works for me.

@michmike can you update the issue to link to https://docs.google.com/document/d/1YkLZIYYLMQhxdI2esN5PuTkhQHhO0joNvnbHpW68yg8/edit as the proposed graduation criteria?

[michmike]

@PatrickLang done!

[kacole2]

@justaugustus is this going to require an exception to be put on the 1.12 tracking since we are past feature freeze and this is not on the sheet? The proposed graduation doc suggests 1.12 for GA/Stable

[justaugustus]

Thanks for the update. I've added this to the 1.12 tracking sheet.

/remove-stage beta
/stage stable
cc: @kacole2 @wadadli @robertsandoval @rajendar38

[zparnold]

Hey there! @michmike I'm the wrangler for the Docs this release. Is there any chance I could have you open up a docs PR against the release-1.12 branch as a placeholder? That gives us more confidence in the feature shipping in this release and gives me something to work with when we start doing reviews/edits. Thanks! If this feature does not require docs, could you please update the features tracking spreadsheet to reflect it?

[justaugustus]

@michmike @PatrickLang --
Any update on docs status for this feature? Are we still planning to land it for 1.12?
At this point, code freeze is upon us, and docs are due on 9/7 (2 days).
If we don't here anything back regarding this feature ASAP, we'll need to remove it from the milestone.

cc: @zparnold @jimangel @tfogo

[PatrickLang]

This needs to move to 1.13. It's not ready for 1.12. Can you help us update the milestone?

[michmike]

/milestone 1.13

[k8s-ci-robot]

@michmike: The provided milestone is not valid for this repository. Milestones in this repository: [v1.12, v1.13, v1.14]

Use /milestone clear to clear the milestone.

In response to this:

/milestone 1.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[michmike]

/milestone v1.13

[justaugustus]

Thanks for the update!

[AishSundar]

@michmike @PatrickLang how confident are we ok this going GA in 1.13? Do we have a list of pending PRs or issues to track for this?

[PatrickLang]

Things are much better than in v1.13 and I've been able to get some more time from @feiskyer and @andyzhangx and others to work on remaining issues.

The test passes are a prow+testgrid PR away from being online. @lachie83 is helping me reach out to the right folks to hand off the Azure credentials needed. @pjh and @yujuhong have also been making good progress on bringing up tests on GCE too.

The current engineering system issues should be done by early November giving us a month for bugfix level work.

I'm tracking the work across multiple projects (including Azure deployments) here:
https://github.com/PatrickLang/k8s-project-management/projects/1

On a related note - I'm not a milestone maintainer and can't mark things for 1.13. What's the best way to get them marked - should I work with you?

[feiskyer]

On a related note - I'm not a milestone maintainer and can't mark things for 1.13. What's the best way to get them marked - should I work with you?

@PatrickLang Ping me if you want something milestoned.

[AishSundar]

Thanks @PatrickLang for a detailed status update. As @feiskyer mentioned either ping him or any of us in the 1.13 release team including @kacole2 @nikopen.

[PatrickLang]

@michmike can you link this for tracking docs? https://github.com/kubernetes/website/pull/10875

[claurence]

@PatrickLang I'm an enhancements shadow checking in on this issue - sounds like progress is being made for this issue - as an FYI code slush is 11/9 and Code freeze is 11/15 - any concerns about making those dates

[bgrant0607]

There is concern about whether conformance testing will be ready, given that we just became aware of this on Thursday.
cc @jdumars @spiffxp

[PatrickLang]

@claurence thanks for checking. We are bringing tests online (Azure & GCE) and working through the docs now. The PR queue other than docs and a test changes in review is empty so we're on track from a code standpoint.

[AishSundar]

@PatrickLang can you plz point us to Docs and test PRs plz

/cc @tfogo

[PatrickLang]

@AishSundar - my master tracking board is here: https://github.com/PatrickLang/k8s-project-management/projects/1
That has links to doc PR kubernetes/website#10875 as well as test PRs

[AishSundar]

Thanks @PatrickLang. From Brian's comment above looks like lack of appropriate conformance coverage will be a blocker for this feature. I see a few conformance related and tests PRs in your dashboard. Do those address all of Sig-Arch's guidance? or can we expect more changes/PRs for conformance testing.

[AishSundar]

Also looking at the project dashboard, there are quite a few things waiting to merge and stabilize. We are just a week away from Code freeze. Are you still confident of making 1.13 ? ^^

[PatrickLang]

@AishSundar yes, still tracking for v1.13. Today in SIG-Architecture (notes) we decided to separate out this into two workflows:

1. Make a decision on Windows [stable] tag for v1.13 based on meeting requirements. This doc is the main discussion point, and there are threads happening in SIG-Architecture and SIG-Windows to close this. https://docs.google.com/document/d/1YkLZIYYLMQhxdI2esN5PuTkhQHhO0joNvnbHpW68yg8/edit

2. What is conformance for a cluster containing Windows nodes? This won't block item 1, as Windows is an optional feature that not all clusters will use. We'll continue this discussion on the conformance working group list and keep working on a joint solution that will probably be finished after v1.13.

[AishSundar]

Thanks @PatrickLang. Would you be able to join our 1.13 RT burndown meeting, either Monday or Wednesday of next week at 10am PST, to give a more latest update as we near Code freeze?

[PatrickLang]

@AishSundar sure - I can join Wednesday. Can you point me to the meeting details (notes/zoom link)? I don't see it on the kubernetes shared iCal list.

cc @michmike ^^

[AishSundar]

Meeting notes : http://bit.ly/k8s113-minutes
Zoom: https://zoom.us/j/611312756

[AishSundar]

Per discussion with Sig Arch, we decided to move Windows GA to 1.14

@PatrickLang @michmike @kacole2 @marpaia @kbarnard10 @tfogo

[kacole2]

/milestone v1.14

[michmike]

@spiffxp should we mark this as "Tracked/yes" now for v1.14 since we also have the KEP

[claurence]

@michmike - targeting stable for 1.14 correct?

[michmike]

@claurence that is accurate.

[michmike]

KEP was approved today for stable release with 1.14

[lledru]

Hello, 1.14 enhancement shadow here. Code Freeze is March 7th and all PRs must be merged by then to your issue to make the 1.14 release. What open K/K PRs do you still have that need to merge? Thanks

[michmike]

@lledru we have some work that's underway and includes a few PRs. you can see our tracking board for 1.14 here. https://github.com/orgs/kubernetes/projects/8

[craiglpeters]

The placeholder Documentation PR is in place

[claurence]

@michmike looking at that tracking board is everything in the 1.14 release blocking column what should be merged by code freeze? it looks like if I'm interpreting that correctly that is 9 open PRs in K/K that need to be merged by Friday?

[michmike]

@claurence that is correct. it is going to be a tight squeeze. a few of those PRs are ready to be merged today already.

[michmike]

3 years after the forming of SIG-Windows, we are pleased to announce that Kubernetes v1.14 will include stable support for windows

• KEP at https://github.com/kubernetes/enhancements/blob/master/keps/sig-windows/20190103-windows-node-support.md
• release notes created
• general 1.14 blog post created
• Docs preview at -
  https://5c914f680410ed000aeebc2e--kubernetes-io-vnext-staging.netlify.com/docs/setup/windows/add-windows-nodes/
• green test runs at https://testgrid.k8s.io/sig-windows#aks-engine-azure-1-14-windows

/close

[k8s-ci-robot]

@michmike: Closing this issue.

In response to this:

3 years after the forming of SIG-Windows, we are pleased to announce that Kubernetes v1.14 will include stable support for windows

• KEP at https://github.com/kubernetes/enhancements/blob/master/keps/sig-windows/20190103-windows-node-support.md
• release notes created
• general 1.14 blog post created
• Docs preview at -
  https://5c914f680410ed000aeebc2e--kubernetes-io-vnext-staging.netlify.com/docs/setup/windows/add-windows-nodes/
• green test runs at https://testgrid.k8s.io/sig-windows#aks-engine-azure-1-14-windows

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.