Enhancements: Provide RunAsGroup feature for Containers in a Pod

Created on 19 Mar 2017  ·  109Comments  ·  Source: kubernetes/enhancements

Feature Description

As a Kubernetes User, i should be able to specify both user id and group id for the containers running inside a pod on a per Container basis, similar to how docker allows that using docker run options -u, --user="" Username or UID (format: <name|uid>[:<group|gid>]) format. Currently kubernetes only allows us to control the primary user id and allows us to add supplemental groups. There is no way to control the primary group id of the running container which is always 0(root).
This feature would enable enterprises to run containers as non root(non zero uid and non zero gid) and hence improve the level of security for the running containers. More discussion and agreement was gathered in this issue 22179

  • One-line feature description (can be used as a release note): Provide RunAsGroup feature for Containers in a Pod
  • Primary contact (assignee): @krmayankk
  • Responsible SIGs: sig-node, sig-auth
  • Design proposal link (community repo): https://github.com/kubernetes/community/pull/756
  • KEP: https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/20190131-runas-groupid.md
  • Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred: @pmorie @liggitt @tallclair
  • Approver (likely from SIG/area to which feature belongs): TBD
  • Feature target (which target equals to which milestone):
    Alpha release: v1.10
    Beta release: v1.14
    Stable release target v1.15

List of Work Items:-

  • [x] RunAsGroup Implementation
  • [x] Add feature flag , mark it alpha and disable by default
  • [x] PSP Implementation for RunAsGroup
  • [x] Verify e2e and Unit test Coverage
  • [ ] Verify Containerd and cri-o Test coverage

Containerd and Cri-o Implementation PR's

Test Results for CRI-O PR with latest Kubernetes Master
https://k8s-testgrid.appspot.com/sig-node-cri-o#crio-e2e-fedora

kinapi-change kinfeature siauth sinode stagstable trackeno

Most helpful comment

@pineking i have the proposal , and the code almost ready. Will send out the proposal by Friday while i try to figure the unit tests and api changes.

All 109 comments

Is the progress listed above accurate?

I started working on this earlier but got distracted by other higher
priority issues . We need this feature hence I will prioritize this again .
Expect a PRoposal and a first PR in the next couple weeks

On Tue, May 2, 2017 at 8:46 AM Dennis Schridde notifications@github.com
wrote:

Is the progress listed above accurate?


You are receiving this because you were assigned.

Reply to this email directly, view it on GitHub
https://github.com/kubernetes/features/issues/213#issuecomment-298675025,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AP4-NM7iZmBnKFNe_ywx_6BSd_Aecz_Vks5r10_BgaJpZM4MhoLz
.

>

-Mayank

@krmayankk any progress to update?

@pineking i have the proposal , and the code almost ready. Will send out the proposal by Friday while i try to figure the unit tests and api changes.

@krmayankk is this still on your radar?

@jduncan-rva yes the proposal is already out. I have some review comments which i will address. I should have a PR by next week.

@krmayankk any updates?

@kincl the proposal is already out and nearing lgtm. We are waiting one more reviewer to review. I was out last week on vacation. I should have the actual PR this week

Here is the proposal under review https://github.com/kubernetes/community/pull/756

Responsible SIGs: sig-node

Sounds like it falls into sig-auth area.

For the history: here is an implementation of the proposal -- https://github.com/kubernetes/kubernetes/pull/52077

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

/remove-lifecycle rotten

/sig auth

/sig node

@krmayankk
Any plans for this in 1.11?

If so, can you please ensure the feature is up-to-date with the appropriate:

  • Description
  • Milestone
  • Assignee(s)
  • Labels:

    • stage/{alpha,beta,stable}

    • sig/*

    • kind/feature

cc @idvoretskyi

@justaugustus yes the api changes for the pr is already out in 62216 . Please note that some part of it is already out in 1.10. Also can you assign this to 1.11 milestone

@krmayankk just to clarify, we're tracking a beta target for 1.11, correct?

@justaugustus yes i am trying that it depends on if i am able to get all changes in

Thanks for the update!

/kind feature

@krmayankk --
We're doing one more sweep of the 1.11 Features tracking spreadsheet.
Would you mind filling in any incomplete / blank fields for this feature's line item?

@krmayankk This feature was worked on in the previous milestone, so we'd like to check in and see if there are any plans for this to graduate stages in Kubernetes 1.12 since there is nothing in the original post. This still has the 1.11 milestone as well so we need to update it accordingly.

If there are any updates, please explicitly ping @justaugustus, @kacole2, @robertsandoval, @rajendar38 to note that it is ready to be included in the Features Tracking Spreadsheet for Kubernetes 1.12.


Please note that the Features Freeze is July 31st, after which any incomplete Feature issues will require an Exception request to be accepted into the milestone.

In addition, please be aware of the following relevant deadlines:

  • Docs deadline (open placeholder PRs): 8/21
  • Test case freeze: 8/28

Please make sure all PRs for features have relevant release notes included as well.

Happy shipping!

This didn't go to beta in 1.11. I'm marking it as targetting beta in 1.12.

@tallclair this has been added to the 1.12 tracking sheet
@justaugustus please add the /tracked label

Done.
cc: @kacole2 @wadadli @robertsandoval @rajendar38

Hey there! @krmayankk I'm the wrangler for the Docs this release. Is there any chance I could have you open up a docs PR against the release-1.12 branch as a placeholder? That gives us more confidence in the feature shipping in this release and gives me something to work with when we start doing reviews/edits. Thanks! If this feature does not require docs, could you please update the features tracking spreadsheet to reflect it?

@tallclair why has this moved to 1.13 ? The code freeze date is Sept 4.

@zparnold sorry i was on on vacation. can you point me to instructions on how to do this

@zparnold i created this PR https://github.com/kubernetes/website/pull/10076, let me know if this is in the right direction

It's still in alpha, and I don't see us promoting it to beta next week. We can still discuss getting your open PRs in, but I think we should keep it alpha for 1.12.

I don't understand this my account was used by ex he changed some stuff to
get access not really sure

On Fri, Aug 24, 2018, 1:36 PM Tim Allclair (St. Clair) <
[email protected]> wrote:

It's still in alpha, and I don't see us promoting it to beta next week. We
can still discuss getting your open PRs in, but I think we should keep it
alpha for 1.12.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/features/issues/213#issuecomment-415829215,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AkNk80eawLOpJeRslt0vPH6i1kiFFJMsks5uUDmLgaJpZM4MhoLz
.

@krmayankk Really close! Could you set the base as kubernetes:release-1.12 as opposed to kubernetes:master?

Removing this from the sheet, per @tallclair's comment:

It's still in alpha, and I don't see us promoting it to beta next week. We can still discuss getting your open PRs in, but I think we should keep it alpha for 1.12.

May be i misunderstand @tallclair @justaugustus does marking a feature issue for a specific milestone means promoting it to at least beta in that milestone ?

@krmayankk is this targeting to make it in 1.13?

This release is targeted to be more ‘stable’ and will have an aggressive timeline. Please only include this enhancement if there is a high level of confidence it will meet the following deadlines:

  • Docs (open placeholder PRs): 11/8
  • Code Slush: 11/9
  • Code Freeze Begins: 11/15
  • Docs Complete and Reviewed: 11/27

Thanks!

Hi @krmayankk, this docs PR was merged in 1.12: https://github.com/kubernetes/website/pull/10076

Was this documentation intended to go in during 1.12? If RunAsGroup is feature gated then we should make sure the documentation mentions this.

This specific documentation was earlier meant for 1.12, but the code didnt merge in time, so ideally it should go now in 1.13. Currently there are two pieces of documentation needed:-

  • RunAsGroup at pod/container level documentation
  • RunAsGroup in PodSecurityPolicy

I will try to find the right places for this documentation and make a PR

Ah, thanks for the info. Could you please create a PR against k/website master to remove that documentation if it's documenting something which isn't actually in 1.12?

Then please could you open a placeholder PR against the dev-1.13 for the 1.13 docs?

@krmayankk other than docs, is there any more pending code or test work for this feature in 1.13? if so can you plz point us to it and indicate when you expect to get all of them merged? thanks

@AishSundar need to see if there is unit and e2e coverage that need to be improved. No more new features are needed as far as i see. PR's are not yet ready

@krmayankk I'm an enhancements shadow checking in on how this issue is tracking. Code slush is on 11/9 and code freeze is coming up on 11/15 do you have a status update on the likelihood that this will make the the code freeze date?

@krmayankk can you please drop in a link to the unit and e2e coverage tests that are being improved as a part of this release?

@claurence @kacole2 there are no additional PR's that i already initiated. So 11/9 wont be possible. Will update with PR links in a week.

Hi @krmayankk I just wanted to follow up about this docs PR: kubernetes/website#10076

You mentioned the documentation was meant for 1.12, but the code didn't merge in time. Does that mean that kubernetes/website#10076 should be reverted?

there are no additional PR's that i already initiated. So 11/9 wont be possible. Will update with PR links in a week.

@krmayankk are you saying there are no PRs in-progress for this enhancement yet? or are all code PRs merged already?

We already entered Code slush today for 1.13 and Code freeze is coming up in a week (11/16). Having PRs opened next week (or) in a week will be too late for this cycle pushing us into cherrypicking the PRs during freeze. At this point in the cycle, we expect all 1.13 features to have merged code and tests or have PRs close to merging. Especially this feature is slated to go to Beta in 1.13 which means we need good e2e test coverage and atleast a few days of CI runs to see that its stable.

As I currently see the status of this enhancement, the release team is not comfortable taking this in 1.13. If my understanding of pending work is incorrect please clarify asap. In absence of this info, we plan to untrack this for 1.13 on Monday. Thanks

@kacole2 as FYI

This is being pulled from the 1.13 milestone. Will revisit if it will be on 1.14 or not during the next cycle

/milestone clear

@AishSundar @kacole2 1.14 is fine , thanks

@krmayankk Hello - I’m the enhancement’s lead for 1.14 and I’m checking in on this issue to see what work (if any) is being planned for the 1.14 release. Enhancements freeze is Jan 29th and I want to remind that all enhancements must have a KEP

@claurence we are taking it to beta in 1.14. https://github.com/kubernetes/kubernetes/pull/73007

@krmayankk is there a KEP for this issue? I'm not sure if I see one linked but might have missed it. Thanks

@krmayankk Hello, i'm one of the 1.14 release enhancements shadow, tomorrow enhancement freeze occurs and we still need a KEP. Otherwise you will be asked to fill in an exception.
we see you have a proposal, could you convert it please ? thanks

@claurence @lledru the proposal was done before the KEP process was in place. This is the proposal . https://github.com/kubernetes/community/pull/756 . Is it mandatory to convert to KEP immediately ?

@krmayankk yes! For Enhancement freeze if the KEP is mostly a link to the design proposal that is fine
but we want to see things like test plan and graduation criteria spelled out

@krmayankk since there is no KEP for this issue yet we will be removing it from the 1.14 milestone. To have it added back in please file an exception - information on the exception process can be found here: https://github.com/kubernetes/sig-release/blob/master/releases/EXCEPTIONS.md

@claurence I don’t work full time on kubernetes so I was expecting a week of turn around time . I will get the kep started today

@claurence finally added this here https://github.com/kubernetes/enhancements/pull/800 and sent out the exception request

Thanks @krmayankk! Exception request approved (I'll update in that thread as well)

Hey @krmayankk 👋 I'm the v1.14 docs release lead. Just a friendly reminder we're looking for a PR against k/website (branch dev-1.14) due by Friday, March 1. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

thanks @jimangel the PR is already there https://github.com/kubernetes/website/pull/12297

this is alpha->beta promotion of existing fields
/assign

@krmayankk looking over the KEP I don't see any testing plans - can someone help PR in testing plans for this enhancement? This information is helpful for knowing readiness of this feature for the release and is specifically useful for CI Signal.

Hello, 1.14 enhancement shadow here. Code Freeze is March 7th and all PRs must be merged by then to your issue to make the 1.14 release. What open K/K PRs do you still have that need to merge? Thanks

@claurence the graduation criteria in https://github.com/kubernetes/enhancements/pull/835/files at the bottom should cover it. Are you looking for testing plan for making it stable or for making it beta ?

@mariantalla for CI signal is that enough for testing plans ^?

Hi @krmayankk, the only PR not merged yet is #73691, correct? will this be merged before code freeze? Thanks

@lledru that is correct

@krmayankk That PR isn't on the 1.14 milestone - if this PR is needed for 1.14 can I add it to the milestone?

@claurence Please track this one https://github.com/kubernetes/kubernetes/pull/75164 and ignore 73691

I just ran into the issue that setting Containers[0].SecurityContext.RunAsUser = 3000 and Containers[0].SecurityContext.RunAsGroup = 3000 make the process running as 3000 but its group is still root (0). Is this expected?

@jingweno what version of Kubernetes? Is the RunAsGroup feature gate enabled?

@jingweno it’s an alpha feature being enabled by default in 1.14(moving to beta ). There was also a bug in containerd that was fixed sometime back . So depends on which runtime are you using , which kubernetes version and whether the alpha feature flags are enabled

Hello @krmayankk , I'm the Enhancement Lead for 1.15. Is this feature going to be graduating alpha/beta/stable stages in 1.15? Please let me know so it can be tracked properly and added to the spreadsheet.

Once coding begins, please list all relevant k/k PRs in this issue so they can be tracked properly.

@kacole2 this feature is beta in 1.14. I will work and figure out what does it take to graduate to Stable in 1.15

Did this get added in 1.15?

It's beta as of 1.14
@krmayankk what is the plan for bringing this to GA?

Restricting runAsGroup would likely be a requirement for GA. I seem to recall my PSP providing an error when I attempted to set the field. It should also support MustRunAsNonRoot similar to the below. Thanks!

spec:
  runAsGroup:
    rule: MustRunAsNonRoot

It should work with PSP, if not please file an issue. For MustRunAsNonRoot (on the pod spec), there were a lot of concerns about all the different ways to attach groups to a process, and whether we could reliably assert that it wasn't using the root group. I'm not categorically against it, but also don't want to have the feature if it's not a strong guarantee. I also don't think it should block GA, as it's functionality that can be layered onto the current approach.

@tallclair I think the main criteria is having the right test coverage for runtimes. This issue https://github.com/kubernetes/kubernetes/issues/72253 covers that , but i have no idea what needs to be done there ? Can you advise ?

@Elegant996 here is the discussion related to mustRunAsNonRoot on the pod spec https://github.com/kubernetes/kubernetes/pull/62216 But we ditched in it favor of only doing it in PSP

Hi @krmayankk @tallclair , I'm the 1.16 Enhancement Lead/Shadow. Is this feature going to be graduating alpha/beta/stable stages in 1.16? Please let me know so it can be added to the 1.16 Tracking Spreadsheet. If not's graduating, I will remove it from the milestone and change the tracked label.

Once coding begins or if it already has, please list all relevant k/k PRs in this issue so they can be tracked properly.

Milestone dates are Enhancement Freeze 7/30 and Code Freeze 8/29.

Thank you.

Hello @krmayankk @liggitt @tallclair , 1.17 Enhancement Shadow here! 🙂

I wanted to reach out to see *if this enhancement will be graduating to alpha/beta/stable in 1.17?

*
Please let me know so that this enhancement can be added to 1.17 tracking sheet.

Please note that the KEP is missing test plan.

Thank you!

🔔Friendly Reminder

  • The current release schedule is

    • Monday, September 23 - Release Cycle Begins

    • Tuesday, October 15, EOD PST - Enhancements Freeze

    • Thursday, November 14, EOD PST - Code Freeze

    • Tuesday, November 19 - Docs must be completed and reviewed

    • Monday, December 9 - Kubernetes 1.17.0 Released

  • A Kubernetes Enhancement Proposal (KEP) must meet the following criteria before Enhancement Freeze to be accepted into the release

    • PR is merged in
    • In an implementable state
    • Include test plan and graduation criteria
  • All relevant k/k PRs should be listed in this issue

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

/remove-lifecycle stale

Hey there @krmayankk @liggitt @tallclair -- 1.18 Enhancements shadow here. I wanted to check in and see if you think this Enhancement will be graduating to alpha/beta/stable in 1.18 or having a major change in its current level?

The current release schedule is:

  • Monday, January 6th - Release Cycle Begins
  • Tuesday, January 28th EOD PST - Enhancements Freeze
  • Thursday, March 5th, EOD PST - Code Freeze
  • Monday, March 16th - Docs must be completed and reviewed
  • Tuesday, March 24th - Kubernetes 1.18.0 Released

To be included in the release,

  1. The KEP PR must be merged
  2. The KEP must be in an implementable state
  3. The KEP must have test plans and graduation criteria.

If you would like to include this enhancement, once coding begins please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

We'll be tracking enhancements here: http://bit.ly/k8s-1-18-enhancements

Thanks! :)

@krmayankk - It's ok if it doesn't happen for 1.18, but would you be able to help get this feature over the finish line to GA?

@krmayankk @tallclair Just a friendly reminder, we are just 7 days away from the Enhancement Freeze (Tuesday, January 28th).

@krmayankk @tallclair Just a friendly reminder, we are just 2 days away from the Enhancement Freeze (3 PM Pacific Time, Tuesday, January 28th).

Unfortunately, the deadline for the 1.18 Enhancement freeze has passed. For now, this is being removed from the milestone. If there is a need to get this in, please file an enhancement exception.

Hey there @krmayankk -- 1.19 Enhancements shadow here. I wanted to check in and see if you think this Enhancement will be graduating in 1.19?

In order to have this part of the release:

  1. The KEP PR must be merged in an implementable state
  2. The KEP must have test plans
  3. The KEP must have graduation criteria.

The current release schedule is:

  • Monday, April 13: Week 1 - Release cycle begins
  • Tuesday, May 19: Week 6 - Enhancements Freeze
  • Thursday, June 25: Week 11 - Code Freeze
  • Thursday, July 9: Week 14 - Docs must be completed and reviewed
  • Tuesday, August 4: Week 17 - Kubernetes v1.19.0 released
  • Thursday, August 20: Week 19 - Release Retrospective

If you do, I'll add it to the 1.19 tracking sheet (http://bit.ly/k8s-1-19-enhancements). Once coding begins please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

Thanks!

Hi there @krmayankk ,

Kind reminder about my question above.

Regards,
Mirek

Hi there @krmayankk ,

Kind reminder about my question above.

Regards,
Mirek

Hi there @krmayankk ,

Kind reminder about my question above.

Regards,
Mirek

Hey @krmayankk , Enhancement shadow for the v1.19 release cycle here. Just following up on my earlier update to inform you of the
upcoming Enhancement Freeze scheduled on Tuesday, May 19.

Regards,
Mirek

@krmayankk -- Unfortunately the deadline for the 1.19 Enhancement freeze has passed. For now this is being removed from the milestone and 1.19 tracking sheet. If there is a need to get this in, please file an enhancement exception.

@krmayankk - It's ok if it doesn't happen for 1.18, but would you be able to help get this feature over the finish line to GA?

@tallclair yes if i know what needs to be done. Do you know what is the GA criteria ? What do i need to chase ?

We might not need any changes to the feature. The bare minimum would be to update the KEP to state the GA criteria, and update the flag to GA. Also update the KEP to fill in the Production Readiness Questionaire which might help identify issues that need to be addressed.

thanks @tallclair will start on this and see where we land.

Hi @tallclair @krmayankk

Enhancements Lead here. Can you confirm whether this will be graduating to stable in 1.20?

Thanks,
Kirsten

Yes, let's try and get it in 1.20.
/milestone v1.20

Hey @krmayankk @tallclair -- 1.20 Enhancements Shadow here 👋

Friendly reminder to have KEP updates done for the 1.20 milestone by the Enhancements Freeze deadline on October 6th.

Echoing @tallclair's previous comment, can we please update the graduation criteria for GA and add the PRR.

We also have a new KEP format, if we can update the KEP to match the new format detailed here, that would be awesome! 😄

Regards,
Jeremy

@MorrisLaw will do . Had started this https://github.com/kubernetes/enhancements/pull/1974/files
@tallclair can you help answer this https://github.com/kubernetes/kubernetes/issues/72253#issuecomment-695147696 ?

Hey @krmayankk @tallclair -- reminder that, in order to make the 1.20 milestone for graduating to GA, the KEP and it's changes to the graduation criteria have to be merged before the October 6th Enhancements Freeze deadline. Otherwise, you'll have to either make an exception or defer it to the next milestone.

As a reminder, Enhancements Freeze deadline is Tomorrow October 6th EOD PST.

Hi @krmayankk

Enhancements Freeze is now in effect. Unfortunately, your KEP needed to be updated and the PR has not yet merged. If you wish to be included in the 1.20 Release, please submit an Exception Request as soon as possible.

Best,
Kirsten
1.20 Enhancements Lead

Was this page helpful?
0 / 5 - 0 ratings

Related issues

liggitt picture liggitt  ·  7Comments

justinsb picture justinsb  ·  11Comments

wlan0 picture wlan0  ·  9Comments

xing-yang picture xing-yang  ·  13Comments

prameshj picture prameshj  ·  9Comments