Enhancements: Portable Service Definitions

[pierreozoux]

The KEP already exists, but there were no issue, so creating this one.
(I'm new here, so please forgive me or give me guidance on how to proceed)

I'm interested in working on that, and I'd like guidance on how to move this forward.

About the secrets, I'd like to mention this project from appscode - AppsBinding which is the ServiceCatalog equivalent of binding.

I think that this KEP asks again the question of Operator vs Service Catalog, and I keep asking this to myself, but I think I found the answer in this thread.

And then, to move this forward, I think we need the following:

• standard CRDs
  
  
  
  • MySQL
  
  
  • PostegreSQL
  
  
  • Redis
  
  
  • ObjectStorage
  
  
  • Database Snapshot
  
  
  • ...
  
  
• kindly ask/PR the current implementation to use them

To list them in a cluster, we can just label them I guess. But at the end of the day, all CRDs are special services to the end user. So I don't see a big difference between a normal CRD and a standard CRD.

For kubernetes project, there is not much code to write/maintain. We'll need to write/maintain this list of CRDs, and then write automation/validation.

Looking forward to see this happen!

[pierreozoux]

/sig apps

[pierreozoux]

There are also these 3 links relevant to this discussion:

• https://blog.upbound.io/introducing-crossplane-open-source-multicloud-control-plane/
• https://blog.crossplane.io/postgresql-crossplane-rook/
• https://github.com/awslabs/aws-service-operator/issues/39

[pierreozoux]

Here is the repo I'd like to discuss: https://github.com/pierreozoux/StandardResourceDefinitions

[mattfarina]

@pierreozoux I'll poke at the different topics here as I get a chance. The first is the AppsBinding. Thanks for pointing me to the post on it.

One problem is the way the credentials are stored in the AppBinding CRs. They are stored in plain text inside etcd and don't appear to be mountable via things like environment variables. ConfigMaps and Secrets have a special place in Kubernetes in their ability to do that.

Secrets have been going through a bit of backend work lately. Where there used to be a flag for encrypting the data of secrets (alpha feature being enabled) there is now work to back secrets with KMS providers.

I would like to see anything we do with credentials get looped into the security mechanisms already being worked on.

Secrets have a type field. This is often set to Opaque. In our case we were looking to have different types with schemas. It's not currently as easy to validate as a CR/CRD but the tradeoff is in the security and being able to use the details for things like environment variables or volume mounting.

Kubernetes is starting to take securing credentials much more seriously, which is needed for many users and use cases, and I would like to see the work here leverage that.

[unteem]

@mattfarina from what I understand its not the secret part of the credentials that are set in clear in the CR. The part of the credentials that are secrets, are referenced in the CR by their secret name see for example how it is used in kubevault.

[kacole2]

Hello @pierreozoux , I'm the Enhancement Lead for 1.15. Can you update the original post to follow the standard format?

# Enhancement Description
- One-line enhancement description (can be used as a release note):
- Kubernetes Enhancement Proposal: (link to kubernetes/enhancements file, if none yet, link to PR)
- Primary contact (assignee):
- Responsible SIGs:
- Enhancement target (which target equals to which milestone):
  - Alpha release target (x.y)
  - Beta release target (x.y)
  - Stable release target (x.y)

Is this feature going to be graduating alpha/beta/stable stages in 1.15? Please let me know so it can be tracked properly and added to the spreadsheet.

Once coding begins, please list all relevant k/k PRs in this issue so they can be tracked properly.

[justaugustus]

/assign @pierreozoux

[alexeldeib]

The KEP for Portable Service Definitions piqued my interest a while back. I don't follow sig-apps too closely, is there active development in this area? KEP hasn't been updated in a few months. Is the direction e.g. https://github.com/kubernetes-sigs/application?

[evillgenius75]

Hi @pierreozoux , I'm a 1.16 Enhancement Shadow. Is this feature going to be graduating alpha/beta/stable stages in 1.16? Please let me know so it can be added to the 1.16 Tracking Spreadsheet.

Once coding begins or if it already has, please list all relevant k/k PRs in this issue so they can be tracked properly.

As a reminder, every enhancement requires a KEP in an implementable state with Graduation Criteria explaining each alpha/beta/stable stages requirements.

Milestone dates are Enhancement Freeze 7/30 and Code Freeze 8/29.

Thank you.

[pierreozoux]

@evillgenius75 ok, I'll have time in the coming days.

I'd like this KEP moving forward, but I'm kind of new as kubernetes contributor, and to be honet I have no idea how to get this moving.

To put it simple the idea is to have a list of CRDs, part of upstream, to describe popular resources then used by operators. My main use case is that:
As a WordPress operator developer, I want to specify MySQL dependency in a standard way, the same as when I use Ingress or PVC.

I think it would make sense that each resource (MySQL, Postgres) have a different alpha/beta/stable stage requirements.

Instead of tackling the full list, let's take only one. For me the most useful to start with would be Postgres.

I can make a proposal of a basic SRD (Standard Resource Definition) for Postgres, gather feedback from community, and I think we could have this before the 30th of July.

There is no code needed, just some documentation around it I guess, and the definition of the SRD.

@evillgenius75 do you have an idea where can I put this piece of code?

[kacole2]

@pierreozoux can you please format the original post per the standard guidelines? (below)

As per moving forward, this needs approval of the SIG lead and consensus to move forward. Please confer with them on best way to make this land in 1.16

@lachie83 another one for SIG conversations.

# Enhancement Description
- One-line enhancement description (can be used as a release note):
- Kubernetes Enhancement Proposal: (link to kubernetes/enhancements file, if none yet, link to PR)
- Primary contact (assignee):
- Responsible SIGs:
- Enhancement target (which target equals to which milestone):
  - Alpha release target (x.y)
  - Beta release target (x.y)
  - Stable release target (x.y)

[annajung]

Hello @pierreozoux , 1.17 Enhancement Shadow here! 🙂

I wanted to reach out to see *if this enhancement will be graduating to alpha/beta/stable in 1.17?

*
Please let me know so that this enhancement can be added to 1.17 tracking sheet.

Please note that the KEP is missing test plan and status is provisional

Thank you!

🔔Friendly Reminder

• The current release schedule is
  
  
  
  • Monday, September 23 - Release Cycle Begins
  
  
  • Tuesday, October 15, EOD PST - Enhancements Freeze
  
  
  • Thursday, November 14, EOD PST - Code Freeze
  
  
  • Tuesday, November 19 - Docs must be completed and reviewed
  
  
  • Monday, December 9 - Kubernetes 1.17.0 Released
  
  

• A Kubernetes Enhancement Proposal (KEP) must meet the following criteria before Enhancement Freeze to be accepted into the release

  • PR is merged in
  • In an implementable state
  • Include test plan and graduation criteria

• All relevant k/k PRs should be listed in this issue

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[palnabarun]

/remove-lifecycle stale

[helayoty]

Hey @pierreozoux -- 1.18 Enhancements shadow here. I wanted to check in and see if you think this Enhancement will be graduating in 1.18 or having a major change in its current level?

To be included in the release, this enhancement must have a merged KEP in the implementable status. The KEP must also have graduation criteria and a Test Plan defined.

Kindly if you can make sure to format the original post per the standard guidelines and to include all relevant k/k PRs once coding begins so they can be tracked properly.

The current release schedule is:

Tuesday, January 28th EOD PST - Enhancements Freeze
Thursday, March 5th, EOD PST - Code Freeze
Monday, March 16th - Docs must be completed and reviewed
Tuesday, March 24th - Kubernetes 1.18.0 Released

Thanks!

[pierreozoux]

I didn't receive enough feedback from people.
I still think that this would be valuable.

FYI, for people who read this thread, I see a pattern emerging here:
(I see it for S3 operator)

There is an indirection between the CRD defining how the resource looks like, the schema.
And the actual implementation.

For instance, Here is the CRD:
https://github.com/kube-object-storage/lib-bucket-provisioner/tree/master/deploy/crds

And here 2 different implementations that are hence compatible:
https://github.com/yard-turkey/aws-s3-provisioner/blob/master/go.mod#L20
https://github.com/rook/rook/blob/91480a717deb5cd91cc8b5edd175382ca24e92a8/Gopkg.toml#L133-L136

My proposal is that these CRDs (the schema of an S3 bucket) go upstream, so that people would collaborate more to have compatible operators.

@helayoty if you have any ideas on how to make this happen? I've been once to the sig-apps meeting, but I didn't get much feedback.
@mattfarina what do you think? How could we push this further?

I'll ping also this library and see if they have ideas.

[negz]

https://github.com/crossplaneio/crossplane/pull/1163

The above (very early sketch of a) design document contains some thoughts from our experiences with Crossplane that could be relevant to this KEP, particularly with regard to _requesting_ infrastructure. In short, we've found it limiting to have a project like Crossplane publish the CRDs used to request things like SQL databases, Redis caches, buckets, etc. We're investigating making it possible for infrastructure operators to publish their own CRDs and teach Crossplane how to use instances of the defined CRs to provision actual cloud resources without writing Go.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[palnabarun]

/remove-lifecycle stale

[johnbelamaric]

@pierreozoux Enhancements shadow for 1.19 here. Any plans for this in 1.19?

As for how to get something like this going, the most likely way would be to find someone that is very active in sig-apps and ask for their support to move it forward.

Alternatively, since the idea is to build this on CRDs, it could be done completely as an ecosystem project. That is, an independent project.

[pierreozoux]

I think we can close. When I see the amount of work that goes into the bucket resource definition.
Doing this for postgres and mysql and.. looks like a lifetime project.

Hopefully people buidling postgres operator will come to an agreement on a common CRD, but given the xkcd about standards, I have really little hope :)

[johnbelamaric]

:)

A couple years ago I started something like this at https://github.com/infobloxopen/atlas-db. Given the evolution of things since then it probably would need to be completely redone. Also I would design it with *Class objects to segregate platform-specific stuff instead of how it is done there. I think managing the schemas and schema versions is actually a pretty interesting and necessary extension of this (it's done in that package too).