Service-catalog: Implement Custom Resource Definitions storage backend

Created on 2 Aug 2017 · 23Comments · Source: kubernetes-sigs/service-catalog

As part of the transition from Third Party Resources (TPRs) to Custom Resource Definitions (CRDs) (see #987 for more detail), we'll need to implement a CRD storage backend, similar to the TPR storage backend. It will likely be possible to copy much of the code from the TPR implementation (at /pkg/storage/tpr) to implement CRD storage, but it's important that we don't _overwrite_ the TPR implementation so that we can allow either to be configured until we decide to deprecate and remove TPR support.

In addition to the work to implement this storage backend, the API server will need to gain some configuration (via command line flags) to turn on CRD storage.

cc/ @nilebox @mengqiy

Tasks

[ ] #1087 CRD storage health check on startup

third-party-resources lifecyclrotten

Source

arschles

All 23 comments

See https://github.com/atlassian/smith/pull/113 and https://github.com/atlassian/smith/pull/114 for inspiration :)

ash2k on 2 Aug 2017

https://kubernetes.io/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/

vaikas on 17 Aug 2017

Closing the https://github.com/kubernetes-incubator/service-catalog/pull/1105 PR as I think that we need to revisit the requirements and decide whether we want to support CRDs in Service Catalog

nilebox on 17 Nov 2017

@nilebox Is there anything in particular that blocks using CRDs as a backing for our custom API server? Are the versioning issues still a problem even in this case?

kibbles-n-bytes on 13 Mar 2018

They just added support for Status sub-resources to CRDs but versioning isn't in yet.

carolynvs on 13 Mar 2018

Let's be clear here: are we talking about using CRDs as a backing store for
the our existing API server, or replacing the API server with CRDs?

On Tue, Mar 13, 2018 at 2:39 PM, Carolyn Van Slyck <[email protected]

wrote:

They just added support for Status sub-resources to CRDs but versioning
isn't in yet.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes-incubator/service-catalog/issues/1088#issuecomment-372774216,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAWXmO7PBkvYT_HYgdw2lamy0PxznXZeks5teBJTgaJpZM4OqZRf
.

pmorie on 13 Mar 2018

Oops, I was referring to what's supported if we moved to CRDs (the latter). Never mind! 😊

carolynvs on 13 Mar 2018

Well, its a good question. I can't remember who from the main k/k team recommended it, but moving to use CRDs instead of our own API server is an option we may want to discuss at the f2f.

duglin on 13 Mar 2018

👍1

It was Eric Tune.

carolynvs on 13 Mar 2018

For the record we - KubeVirt - were also on this question - we actually wrote our own API server before going back to CRDs. The primary reason was data storage for our custom API server.
Today CRDs look promising, with validation, initializers and admission controllers you can do quite a lot.

fabiand on 13 Mar 2018

If someone wants to discuss this topic at the f2f, they should do a
detailed gap analysis first. We do a lot of things in our API that I am not
at all sure are on the roadmap for CRD, let alone currently acheivable.

On Tue, Mar 13, 2018 at 3:42 PM Fabian Deutsch notifications@github.com
wrote:

For the record we - KubeVirt - were also on this question - we actually
wrote our own API server before going back to CRDs. The primary reason was
data storage for our custom API server.
Today CRDs look promising, with validation, initializers and admission
controllers you can do quite a lot.

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
https://github.com/kubernetes-incubator/service-catalog/issues/1088#issuecomment-372793556,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAWXmKEIwCkbCTvytQL5n4AjmrYQiPEAks5teCE3gaJpZM4OqZRf
.

pmorie on 13 Mar 2018

@pmorie we are interested in bringing up CRDs as a backing store for
the our existing API server. We want this to be able to remove the dependency on another etcd instance.

n3wscott on 13 Mar 2018

For reference: https://docs.google.com/presentation/d/1IiKOIBbw7oaD4uZ-kNE-mA3cliFjx9FuFLd7pj-dj1Y/edit?ts=5a78bb75#slide=id.g30d931056f_0_12

duglin on 13 Mar 2018

@n3wscott you can take a look at #1105 which had most stuff working, but required quite a lot of code and still had some issues (IIRC watches in kubectl were dropping from time to time for some reason).
I am not sure this is a good idea. The better way would be supporting such storage out of the box in k8s.io/apimachinery or k8s.io/apiserver, otherwise it's a PITA to having to support this code.

nilebox on 13 Mar 2018

👍1

We do a lot of things in our API that I am not
at all sure are on the roadmap for CRD, let alone currently achievable.

Another aspect that with using so many features (admission controllers, validation etc) it might require as much code to use CRDs (if not more), as well as possibly even more complicated code.

So the only real benefit here is no need to manage a dedicated etcd (or share core etcd directly).

nilebox on 13 Mar 2018

I would like to work on this issue if no one else is. #dibs?

n3wscott on 21 Mar 2018

@n3wscott sure, but I think it would be better to discuss first whether we do want to bring it back, and if we do what's the best approach.

Also it would be nice to talk to API machinery folks to check whether they have any new recommendations for this problem (e.g. there might be some "blob store" coming in future, or they might want to support such use case in k8s.io/apiserver out of the box).

nilebox on 22 Mar 2018

👍1

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot on 24 Apr 2019

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot on 21 Aug 2019

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

fejta-bot on 20 Sep 2019

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

fejta-bot on 20 Oct 2019

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.