Enhancements: Dynamic Audit Configuration

[pbarker]

Feature Description

• One-line feature description (can be used as a release note):
  Dynamic configuration of Audit facilities in the apiserver
• Primary contact (assignee):
  @pbarker
• Responsible SIGs:
  sig-auth
• KEP: https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/0014-dynamic-audit-configuration.md
• Design proposal link (community repo):
  https://github.com/kubernetes/community/pull/2188
  https://github.com/kubernetes/community/pull/2407
• Link to e2e and/or unit tests:
• Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred:
  @tallclair
  @liggitt
  @yliaog
  @caesarxuchao
• Approver (likely from SIG/area to which feature belongs):
  @tallclair
• Feature target (which target equals to which milestone):
  
  
  
  • Alpha release target (x.y)
    
    1.15
  
  
  • Beta release target (x.y)
    
    1.xx
  
  
  • Stable release target (x.y)
    
    1.xx
  
  

API PRs:

• [ ] https://github.com/kubernetes/kubernetes/pull/73981
• [ ] https://github.com/kubernetes/kubernetes/pull/73547

[pbarker]

/cc @tallclair
/sig auth

[pbarker]

/milestone v1.12

[justaugustus]

/kind feature
/stage alpha

[justaugustus]

/assign @pbarker

[zparnold]

Hey there! @pbarker I'm the wrangler for the Docs this release. Is there any chance I could have you open up a docs PR against the release-1.12 branch as a placeholder? That gives us more confidence in the feature shipping in this release and gives me something to work with when we start doing reviews/edits. Thanks! If this feature does not require docs, could you please update the features tracking spreadsheet to reflect it?

[pbarker]

@zparnold will do 👍 to be clear, do I write the docs there or is that something the docs folks do?

[zparnold]

You'll be writing the docs, and we'll be making sure that it matches our style and clarity. (Mostly because we may not understand the feature as well as you do at present.) If you need help, please just let me or @jimangel know

[pbarker]

ok here is the starter issue https://github.com/kubernetes/website/pull/9947

[zparnold]

Thank you! I'll mark it on the FT spreadsheet!

On Mon, Aug 20, 2018 at 9:07 PM Patrick Barker [e-mail]
wrote:

ok here is the starter issue kubernetes/website#9947
https://github.com/kubernetes/website/pull/9947

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/features/issues/600#issuecomment-414526005,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE81SELfIzplzIG3Xh8nq44M3EhHtsZDks5uS2t1gaJpZM4VnCpV
.

[justaugustus]

@pbarker --
Any update on docs status for this feature? Are we still planning to land it for 1.12?
At this point, code freeze is upon us, and docs are due on 9/7 (2 days).
If we don't here anything back regarding this feature ASAP, we'll need to remove it from the milestone.

cc: @zparnold @jimangel @tfogo

[pbarker]

@justaugustus unfortunately this one has gotten caught up in the review process and will not be making 1.12

[pbarker]

/milestone v1.13

[pbarker]

@justaugustus I don't have the power to change the milestone, but we'll be targeting 1.13

[liggitt]

/milestone v1.13

[justaugustus]

/milestone v1.13

[justaugustus]

(btw, pretty sure you have milestone powers here, @liggitt, but we hadn't turned the bot on until https://github.com/kubernetes/test-infra/pull/9252 merged)

[AishSundar]

@pbarker how confident are we of this going to Alpha in 1.13? Are the above 2 PRs the only pending ones?

[pbarker]

Hey @AishSundar, there are 3 open PRs
https://github.com/kubernetes/kubernetes/pull/67547 The API, about done but hung up on one last question.
https://github.com/kubernetes/kubernetes/pull/67257 The implementation, mostly waiting on the API to finish
https://github.com/kubernetes/kubernetes/pull/69902 a small one for an integration test.

This slipped on 1.12 and we certainly intend on making 1.13. If there is anything you all can do to help wrangle people toward the goal that would be very appreciated 🙏

[AishSundar]

Thanks @pbarker we will track the PRs listed above. Is there a doc PR associated with this feature? @tfogo as FYI

[pbarker]

Thank you @AishSundar docs PR is here https://github.com/kubernetes/website/pull/9947. Also a quick update, the implementation got split into two pieces https://github.com/kubernetes/kubernetes/pull/67257 and https://github.com/kubernetes/kubernetes/pull/70021 to make it more digestible.

[pbarker]

@tallclair is the main reviewer on this, but he will be in kubecon this next week so I wanted to give a quick update on what where we are here. The plugins PR kubernetes/kubernetes#70021 is very close to merging, should have that done by EOW. Changes made to the plugins piece has made it possible to simplify the handlers kubernetes/kubernetes#67257, those changes should be out today. Finally, there is an integration test https://github.com/kubernetes/kubernetes/pull/69902. We are coordinating to have a reviewer fill in for @tallclair while he's out for the last two PRs.

[claurence]

Hi @tallclair @pbarker I'm an enhancements shadow checking in on how this issue is tracking. Last comment stated it was close to merging. Code slush is on 11/9 and code freeze is coming up on 11/15 do you still feel confident this will make those dates?

[AishSundar]

@pbarker we are already in slush and fast approaching Code freeze for 1.13 this friday 11/16. Do you think this feature is still on track for 1.13?

Are kubernetes/kubernetes#67257 and kubernetes/kubernetes#69902 the only PRs we are tracking. I am more concerned about the latter merging soon. We would like for all pending work to be completed early this week, giving us atleast a couple fo days to watch and stabilize CI. Do you think this is feasible?

[pbarker]

Hey @AishSundar and @claurence, we just merged the big piece on friday. The next PR https://github.com/kubernetes/kubernetes/pull/67257 isn't that large and I've already had a number of people look it over. This should go quick, then we just need the integration https://github.com/kubernetes/kubernetes/pull/69902 which should be very easy as well. I think we're on track but I would be happy to take any help keeping these moving. Thank you!

[AishSundar]

Thanks @pbarker I added some missing labels to the PRs (we need /sig, kind, priority, and milestone) to merge. Let mee check back on wednesday for latest status and Go/NoGo for 1.13. thanks

[AishSundar]

@pbarker I see kubernetes/kubernetes#69902 is still in active review. Any ETA for getting this in by tomorrow?

[pbarker]

Hey @AishSundar, @liggitt has it in his queue, but he doesn't see it as a blocker for 1.13. I think we can stop tracking that here.

[AishSundar]

Just to clarify, this enhancement is still on for 1.13, it just kubernetes/kubernetes#69902 thats dropped from having to go in for 1.13?

So except for docs this is complete?

[liggitt]

Just to clarify, this enhancement is still on for 1.13, it just kubernetes/kubernetes#69902 thats dropped from having to go in for 1.13?

So except for docs this is complete?

correct

[omri86]

Hi there, I am a bit confused with some part of this feature, if someone can please help and clarify this it would be great!

I saw that there are 2 types of Policy objects, one on the static backend and another defined in the scope of this feature and although they have some similarities they are still different, I also noticed this was mentioned in the docs:

The AuditSink policy differs from the legacy audit runtime policy. 
This is because the API object serves different use cases. 
The policy will continue to evolve to serve more use cases.

I thought that the goal of this feature was to define the entire backend (including the policy) in a dynamic manner, so if the static policy and the dynamic one are different how can this be achieved?

Are they planned on being identical at some point?
Or if they actually do serve different purposes - what are those purposes?

I hope my question was clear enough, please let me know if I need to clarify further.
Thanks!

[pbarker]

Hey @omri86, the original goal was to just take the static definitions and make them available through the API. In the API review https://github.com/kubernetes/kubernetes/pull/67547, we decided it would be a good time to rethink how policy was handled. The outcome of those talks is https://github.com/kubernetes/kubernetes/pull/71230 which is a more composable way of handling policy, I'm in the middle of making an out of tree implementation here https://github.com/pbarker/audit-lab/tree/policy. There is further discussion needed on the matter that will likely happen after winter break. Let me know if there's anything else I can clarify

[claurence]

@pbarker Hello - I’m the enhancement’s lead for 1.14 and I’m checking in on this issue to see what work (if any) is being planned for the 1.14 release. Enhancements freeze is Jan 29th and I want to remind you that all enhancements for 1.14 need to have a KEP it looks like the KEP for this enhancement is here: https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/0014-dynamic-audit-configuration.md - let me know if that is not the correct KEP for this enhancement.

[lachie83]

@pbarker 1.14 release enhancements shadow here. We are less than a week out from 1.14 enhancements freeze. Friendly ping on @claurence comment above.

[pbarker]

Hey sorry @claurence I somehow missed your ping 🙂 This feature is in alpha in 1.13, I didn't know if it should be kept open as we track it toward beta or open a separate one. WDYT?

[claurence]

@pbarker I think it gets left open until it graduates to stable based on what other issues do - is it targeting beta for 1.14 or should it be removed from the 1.14 milestone? Thanks

[pbarker]

ok cool thanks @claurence we will be targeting 1.14 for beta

[claurence]

@pbarker looking over the KEP for this enhancement I don't see any testing plans - can someone help PR in testing plans for this enhancement? This information is helpful for knowing readiness of this feature for the release and is specifically useful for CI Signal.

If we don't have testing plans this enhancement will be at risk for being included in the 1.14 release

[pbarker]

@claurence what do you mean by a testing plan? We have an integration test and an e2e test in the works, is there a document I'm missing?

[claurence]

@pbarker yes e2e test and integration tests are what we are looking for - when you have those can you PR them into the KEP?

[pbarker]

will do 👍

[claurence]

Hi All,
We've entered code freeze and it looks like the two PRs y'all have in the description are still open: kubernetes/kubernetes#73981
kubernetes/kubernetes#73547

Because they are open and don't have "LGTM" and "Approve" labels we are removing this enhancement from the 1.14 release - if you feel this was done in error please talk with your relevant SIG or come in the sig-release slack channel to discuss

[tallclair]

/milestone v1.15

[kacole2]

@pbarker we're doing a KEP review for enhancements to be included in the Kubernetes v1.15 milestone. After reviewing your KEP, it's currently missing test plans which is required information per the KEP Template. This was also noted by @claurence above during the 1.14 release cycle. Please update the KEP to include the required information before the Kubernetes 1.15 Enhancement Freeze date of 4/30/2019.

[pbarker]

@kacole2 this KEP was made before those new reqs and the integration and e2e tests are already merged to master, is it necessary to retroactively add them here? I understood the KEP to be a design document, not an ongoing sort of thing (unless thats changed)?

[kacole2]

I'll defer to @claurence on what she wants to see since she is the lead for this release. Thank you for the quick response.

[pbarker]

https://github.com/kubernetes/enhancements/pull/1022

[simplytunde]

Hey @pbarker Just a friendly reminder we're looking for a PR against k/website (branch dev-1.15) due by Thursday, May 30. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

[simplytunde]

@pbarker The placeholder PR against k8s.io dev-1.15 is due Thursday May 30th.

[kacole2]

Hi . Code Freeze is Thursday, May 30th 2019 @ EOD PST. All enhancements going into the release must be code-complete, including tests, and have docs PRs open.

Should we be tracking more than:
kubernetes/kubernetes#73981
kubernetes/kubernetes#73547

If you know this will slip, please reply back and let us know. Thanks!

[kacole2]

Hi @pbarker, today is code freeze for the 1.15 release cycle. The k/k PRs have not yet been merged (https://github.com/kubernetes/kubernetes/pull/73547 https://github.com/kubernetes/kubernetes/pull/73981). It's now being marked as At Risk in the 1.15 Enhancement Tracking Sheet. Is there a high confidence these will be merged by EOD PST today? After this point, only release-blocking issues and PRs will be allowed in the milestone with an exception.

[pbarker]

yea this is unfortunately going to slip another release, we'll get it done in the next one, thanks @kacole2

[tamalsaha]

We would like to see this issue make progress on 1.16 cycle. Are there any blockers beyond review bandwidth?

[evillgenius75]

Hi @pbarker I'm the 1.16 Enhancement Shadow. Is this feature going to be graduating alpha/beta/stable stages in 1.16? Please let me know so it can be added to the 1.16 Tracking Spreadsheet. If not's graduating, I will remove it from the milestone and change the tracked label.

Milestone dates are Enhancement Freeze 7/30 and Code Freeze 8/29.

Thank you.

[kacole2]

@pbarker there wasn't any notification if this was going to make it in 1.16. Can you let me know if this should be tracked or not?

[VineethReddy02]

Hi @pbarker, I'm the v1.16 docs release shadow.

Does this enhancement require any new docs (or modifications)?

Just a friendly reminder we're looking for a PR against k/website (branch dev-1.16) due by Friday,August 23rd. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

Thanks!

[kacole2]

@pbarker @tamalsaha

Enhancement Freeze has passed for 1.16. I never received a response if this will be graduating stage in 1.16 or if work is being done. This is being removed from the 1.16 milestone. If this would like to be re-added, please file an exception and it will require approval from the release lead.

/milestone clear

[tamalsaha]

@kacole2 , I believe @liggitt is the right person to contact on this feature. Please see his recent response https://github.com/kubernetes/kubernetes/pull/71230#issuecomment-511535527

My reading of @liggitt's comment is that this features is back to KEP phase.

[jeremyrickard]

Hey there @liggitt @pbarker -- 1.17 Enhancements shadow here 👋 . I wanted to check in and see if you think this Enhancement will be graduating to beta or new features will land in 1.17?

The current release schedule is:

• Monday, September 23 - Release Cycle Begins
• Tuesday, October 15, EOD PST - Enhancements Freeze
• Thursday, November 14, EOD PST - Code Freeze
• Tuesday, November 19 - Docs must be completed and reviewed
• Monday, December 9 - Kubernetes 1.17.0 Released

If you do, I'll add it to the 1.17 tracking sheet (https://bit.ly/k8s117-enhancements). Once coding begins please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

Thanks!

[zetaab]

@pbarker @tallclair @liggitt what is status of this Enhancement? If I read previous comments it makes me feel is this feature going to beta at all? I see this feature very useful and would like to see it in beta. Could we push this forward and get this to 1.18?

There was already PR to make this enhancement to beta in 1.15 but it was closed. So could we just reopen it (and rebase)?

[tallclair]

I agree this feature needs a roadmap. I would say that at the moment, it's future is unknown. Can you share a little more about why you think it's useful, and how you plan to to use it?

[pbarker]

VMware bought Heptio and this got put on the back burner as we integrated. We are now coming back around to it with more resources and will probably begin work on it early next year.

[tamalsaha]

I have been quietly (well, I pinged pbarker a few times since he was working on it earlier this year.) following and waiting for this feature. We build a few k8s operators and we need this feature to do reliable and trusted usage-based-billing.

@tallclair , I thought you were working on it https://groups.google.com/forum/#!msg/kubernetes-sig-auth/Ha0C4cladCQ/JhAaOKBhFAAJ .

We don't have the resources to contribute to push code to upstream. But I would more than happy to contribute by reading design docs / KEPs . How can I do that?

[tallclair]

More eyes on designs & KEPs is definitely helpful. Watching this issue, attending sig-auth and the mailing list is probably the best way to keep up.

Unfortunately I don't have time to be actively working on this outside of design reviews.

[spiffxp]

My interest in seeing this go to beta is to get parity with the ability to update the config dynamically, as is supported today for admission webhooks (https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/).

For conformance we are currently unable to dynamically audit which components/tests are hitting which API endpoints without making a run around kubernetes, or setting the cluster up as such by default (which we're unable to do for hosted providers who don't expose alpha options). The goal would be to install a pod which registers itself as an audit webhook automagically, if installed by a user with sufficient privileges.

It's unclear to me whether PR's/KEP's like https://github.com/kubernetes/enhancements/pull/1259 imply there is more done in refining the config before enabling dynamic configuration

[johnbelamaric]

@pbarker Enhancements shadow for 1.18 here. Are you targeting anything for this in 1.18? We need to track it if so. The release schedule is:

Monday, January 6th - Release Cycle Begins
Tuesday, January 28th EOD PST - Enhancements Freeze
Thursday, March 5th, EOD PST - Code Freeze
Monday, March 16th - Docs must be completed and reviewed
Tuesday, March 24th - Kubernetes 1.18.0 Released

[johnbelamaric]

Nothing planned for 1.18 per @liggitt

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[palnabarun]

/remove-lifecycle stale

[tallclair]

FYI, we'll be further discussing 2 potential paths forward for this feature at the next sig-auth meeting (2020-04-29):

1. Dynamic audit proxy webhook design
2. Dynamic webhook sinks with static policy

[harshanarayana]

Hey there @pbarker, 1.19 Enhancements shadow here. I wanted to check in and see if you think this Enhancement will be graduating in 1.19?

In order to have this part of the release:

1. The KEP PR must be merged in an implementable state
2. The KEP must have test plans
3. The KEP must have graduation criteria.

The current release schedule is:

• Monday, April 13: Week 1 - Release cycle begins
• Tuesday, May 19: Week 6 - Enhancements Freeze
• Thursday, June 25: Week 11 - Code Freeze
• Thursday, July 9: Week 14 - Docs must be completed and reviewed
• Tuesday, August 4: Week 17 - Kubernetes v1.19.0 released

If you do, I'll add it to the 1.19 tracking sheet (http://bit.ly/k8s-1-19-enhancements). Once coding begins please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

Thanks!

[tallclair]

We're currently discussing the future of this feature in sig-auth, but no changes are planned for v1.19.

[harshanarayana]

@tallclair Thank you very much for following up. I've updated the tracking sheet accordingly. 👍

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[palnabarun]

/remove-lifecycle rotten

[liggitt]

This feature was removed in 1.19 (details in https://groups.google.com/g/kubernetes-sig-auth/c/aV_nXpa5uWU)