Enhancements: rkt container engine support

[euank]

Description

There is ongoing work for Kubernetes' support of rkt as a production-ready, generally usable container runtime option. This project is sometimes called rktnetes. :rocket:

There has already been significant work to implement this feature, and as of v1.3 it's already a supported option. However, it does not yet have full feature parity with the default Docker runtime, and I would be wary of calling it "production ready".

In order to have a sane scope for this feature, I'd like for it to track the rkt container runtime having full support for all Kubernetes features, support in Kubernetes deployment, well documented, and a great production choice.

Progress Tracker

• [x] Pre-release
  
  
  
  • [x] Write and maintain draft quality doc
  
  
  • [x] https://kubernetes.io/docs/getting-started-guides/rkt/
  
  
  • [ ] Design Approval
  
  
  • [x] Design Proposal: Does-not-exist
  
  
  • [x] My Shepherd is: @euank and see also @kubernetes/sig-rktnetes
  
  
  • [x] My Secondary Contact Point is: @philips
  
  
  • [x] Lots of PRs for code, tests, and docs.
  
  
• [x] Initial release
  
  
  
  • [x] Testing is sufficient for initial release
  
  
  • [x] User docs with tutorials - http://kubernetes.io/docs/getting-started-guides/rkt/
  
  
  • [x] Known issues are clearly documented - http://kubernetes.io/docs/getting-started-guides/rkt/notes/
  
  
• [ ] Released as fully supported/stable
  
  
  
  • [ ] All Kubernetes features are supported when using rkt as the container runtime
  
  
  • [x] subPath support https://github.com/kubernetes/kubernetes/pull/30934
  
  
  • [ ] Init containers -- Depends on #54
  
  
  • [ ] Privileged containers work for all known use-cases
  
  
  • [ ] Hostpath volume mounts that don't exist https://github.com/kubernetes/kubernetes/issues/26816, xref https://github.com/kubernetes/kubernetes/issues/31384
  
  
  • [ ] Kubernetes examples all work under rkt container runtime
    
    
    
    
    
    • [ ] Fluentd logging addon (bindmounts in docker specific directory right now)
    
    
    
    • [ ] to be added....
    
    
    
  
  
  • [ ] Well Tested
  
  
  • [x] K8s master is tested with rkt as the runtime
  
  
  • [ ] Failures against k8s master blocks merges
  
  
  • [ ] PRs are tested with rkt as the runtime
  
  
  • [ ] Failures on PRs block merges
  
  
  • [ ] Known to be used actively in at least one production environment
  
  
  • [ ] docs/design/rkt-container-runtime.md exists.
  
  
  • [ ] Soak, load testing, performance testing with awesome results
  
  
  • [ ] detailed user docs and examples
  
  

FEATURE_STATUS: IN_DEVELOPMENT (I'm not sure the right status; this has had an initial release suitable for use; development is being done to improve the integration and address known issues / caveats).

cc @kubernetes/sig-node

Short meta-note: This feature might or might not actually fit here since it's something that is and has been in progress for quite a while. Many of the bullet points don't work well for it because of its circumstances, and I'm hopeful that it can help refine this process a bit; I've taken some liberties with formatting.

[yifan-gu]

Can someone help adding 1.4 milestone for this? Thank you!

[yujuhong]

/cc @kubernetes/features-maintainers

[yujuhong]

My understanding is that we have switched to focusing on integrating rkt with kubelet using CRI, and the 1.4 milestone doesn't really apply anymore.
@euank @philips, should we remove the milestone and 1.4 labels?

[matchstick]

I will remove the 1.4 milestone for now. Let me know if that is not the right thing to do.
@euank @philips

[euank]

xref https://github.com/kubernetes/kubernetes/issues/8262

I also updated the checkpoints a little, specifically I broke out features a bit more and testing a bit more.

I'll also make a little set of "work done for v1.4" at some point (right now it's subPath, improved privilege, and a couple other bits). The mile-stoning thing makes sense because we're dependent on #54, which is not 1.4.

[idvoretskyi]

@philips @euank I'd like to insist on Milestone to be present for easier feature tracking. I've set a "next-candidate" Milestone for now.

[philips]

The rkt integration works mostly today (for example it works under minikube). I will put it under the v1.5 milestone as we hope the CRI refactor will get it to 100% parity with the Docker engine integration and CRI in this time frame.

[idvoretskyi]

@philips thank you for clarifying.

[euank]

This feature is straddling two different implementations. I don't think either of them should be marked stable in 1.5 and this feature should be focused on the new integration (dependent on the CRI feature) going forwards to end up stable.
For that integration we're now targeting alpha in 1.6.

I think updating to reflect that makes sense.

[idvoretskyi]

@euank I would suggest you update and close this feature as rkt support itself is already implemented in Kubernetes.
To solve the questions, that you've described in your post above, please, open a new feature request, targeting 1.6 /cc @philips @calebamiles

[calebamiles]

@idvoretskyi, I will update this issue. This feature covers the work to make rkt a first class container runtime for Kubernetes there is an existing implementation currently in tree that is mostly but not entirely feature complete with respect to the docker runtime, however, this is an implementation detail of the feature. Currently there are CoreOS developers working on a CRI conformant rkt integration, tracked here, which is related to https://github.com/kubernetes/features/issues/54. Our CRI work is targeting the 1.6 release of Kubernetes and I will move this issue into the 1.6 milestone for tracking. Finally, regarding the existing rkt integration as only small changes have been made during this release cycle we do not believe it makes sense to change the status of that implementation to stable; the CRI work can be considered an iteration on the existing implementation which is currently in tree.

cc: @philips, @euank, @kubernetes/sig-node

[idvoretskyi]

@calebamiles thank you!

[idvoretskyi]

Implemented.