Origin: Move TLS-related secrets out of Route object into a Secret

[pweil-]

In order to not expose private keys to readers we should should reference a secret in the route that can define the key, private key, ca cert, and dest cert to the router. This will enable read access to routes without exposure of sensitive key data.

This also allows the reuse of keys between routes that share the same data, possibly making it easier to support client side wild card routes.

@liggitt

[liggitt]

Implications:

• Whatever creates routes could allow selecting an existing secret instead of uploading new certs
• Whatever watches routes for changes would also need to watch secrets (probably filtered by secret type) to know when to update.
• Moving route certs/keys to secrets is good, though it would mean granting general secret read access to the component watching routes. Ideally, you'd grant access to read secrets referenced by routes, or have an API that only surfaced secrets for a route.

[liggitt]

I'd say this is high priority... without it, I wouldn't feel comfortable giving API access to routes to any role other than project admin in our default policy. That means non-admins couldn't get routes, which means we couldn't show routable URLs (or the existence of routes) for components in the web UI.

[smarterclayton]

This is desirable..., but not required for 3.0. It will incur a migration cost post 3.0 to fix if we do ship like this

[liggitt]

@smarterclayton so editors and readers won't be able to see routes? which means they won't know the URLs for things?

[aweiteka]

This came up in recent discussion. We want to expose the docker-registry endpoint to auth'd users so they can discover the endpoint via API. This is problematic with the certs inline. This feature would enable us to safely expose the registry route.

cc @stefwalter

[smarterclayton]

From security perspective this would require us to give routers access
to all secrets in the system. We probably have to take into account a
way to restrict that.

[deads2k]

We probably have to take into account a
way to restrict that.

Upstream has suggested that the policy rule allow all and an admission plugin more tightly control access based on group. The alternative is to use our runtime.object marker to write code around it, but that's likely to be incompatible with switching to and upstream implementation.

[smarterclayton]

So we now have at least three referential sets of secrets:

nodes -> pods -> secrets (nodes are entitled to see secrets referenced by
pods that are assigned to them, and they can also see image pull secrets
referenced by service accounts of pods that are assigned to them)
registry -> image stream -> image secrets in the namespace (the registry
can see docker image secrets in namespaces it can view)
router -> route > secret (routers are entitled to see the secrets of routes
they can view)

We either:

1. Clearly express intent in the API call (a call to retrieve secret for a
   route, a call to retrieve secrets for an image stream, a call to retrieve
   certain types of secrets for pods)
2. Devise really crazy admission controllers that have to filter arbitrary
   API calls (if you watch on secrets that filter to a route that at one point
   referenced this secret) and guess 100% right all of the time

What's the admission controller overhead? What happens when the client
access pattern changes and the admission controller is wrong? How does
someone audit a complex admission controller? How do you prove an
admission controller does exactly what it says it should?

On Fri, Feb 26, 2016 at 10:52 AM, David Eads [email protected]
wrote:

We probably have to take into account a
way to restrict that.

Upstream has suggested that the policy rule allow all and an admission
plugin more tightly control access based on group. The alternative is to
use our runtime.object marker to write code around it, but that's likely
to be incompatible with switching to and upstream implementation.

—
Reply to this email directly or view it on GitHub
https://github.com/openshift/origin/issues/2162#issuecomment-189335053.

[deads2k]

Right, I always forget the subresource option because its so asymmetric (list, but not get).

routes/foo/secrets seems reasonable. The next problem is segmenting routers and only allowing "owners". I think short term we could allow all routers to see all routes/secrets and tighten it later.

[smarterclayton]

The counter argument to sub resources is watch. Does someone get to watch
on secrets routers can see? Watching requires either a new top level
virtual resource, or filtering in the list call.

Note that I'm not opposed to admission control fundamentally. But it seems
scary. On the other hand, if we do it right it's a positive check (must
have X permission to do this) and either the group allows you to do it or a
binding allows you to do it (virtual resource or virtual group).

Google likes to use groups instead of roles, i'm not opposed to group
checking or virtual role checking. I would say a role is a virtual group,
except that the virtual group is denormalized already for you in some cases
(certs)

On Fri, Feb 26, 2016 at 1:01 PM, David Eads [email protected]
wrote:

Right, I always forget the subresource option because its so asymmetric
(list, but not get).

routes/foo/secrets seems reasonable. The next problem is segmenting
routers and only allowing "owners". I think short term we could allow all
routers to see all routes/secrets and tighten it later.

—
Reply to this email directly or view it on GitHub
https://github.com/openshift/origin/issues/2162#issuecomment-189398648.

[deads2k]

I don't want to tie behavior to our implementation of an authorizer by checking role membership. Group membership would put power in the correct hands.

The admission check would still have problems on watches. Since watches are a list type operation, a subresource could support a filtered watch.

[deads2k]

I've talked it all the way through with @liggitt . the subresource can be done, but it would result in a watch fanout from the router that cannot be born.

I debated a route2 with a secret ref that fed a controller that reconciled route, but that ended with a reverse lookup problem and synchronization of watches.

I got tired on Friday and said, "screw it, we're making a SafeRoute type that hides the Key". Jordan says I may need to hide from the UI team, so I'm calling this a problem for Monday.

[smarterclayton]

I don't think this is a 1.2 thing at this point. I'm not sure route
secrets need a watch, the same way we don't watch pod secrets,
although I could see wanting to create route and secret out of order
(not sure that's required though).

We have to deal with linked secrets for ingress, so this is not really
a very important point here. Don't give viewers access to your
namespace, same statement as V2

[deads2k]

Don't give viewers access to your
namespace, same statement as V2

This has impact for registry users trying to discover the external name to locate the docker registry. It's not an uncommon question, but in the current situation we can't expose that information. It is compounded when your primary exposure is a web UI and you want to have a copy/paste docker login or docker pull line. The information can be a config a web UI component, but API exposure is cleaner.

[deads2k]

Oh, and I should clarify: SafeRoute resource dealing in the Route kind that strips the Key field.

[smarterclayton]

Regarding getting the public name it's better to just reflect it back to
the service (as we have planned all along), or expose a specific bit of
information for the registry. Looking up the route isn't the right way to
do it in general.

On Fri, Feb 26, 2016 at 8:14 PM, David Eads [email protected]
wrote:

Oh, and I should clarify: SafeRoute resource dealing in the Route kind
that strips the Key field.

—
Reply to this email directly or view it on GitHub
https://github.com/openshift/origin/issues/2162#issuecomment-189545572.

[ibotty]

Is there a plan to address this?

[smarterclayton]

It's dependent on integrating Ingress rules from upstream Kube, but the
timing is still a bit open.

On Tue, Jul 5, 2016 at 10:47 AM, Tobias Florek [email protected]
wrote:

Is there a plan to address this?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openshift/origin/issues/2162#issuecomment-230500006,
or mute the thread
https://github.com/notifications/unsubscribe/ABG_pzfI-dt8lBIR6guGG3P5U2fB9upLks5qSm6JgaJpZM4EWnYt
.

[aweiteka]

Ingress rules looking like kube 1.5 or later.

[mfojtik]

@enj this might be related to the BZ :-)

[lichyc]

As this issue is silent for a some time, while it's not solved I like to ask on timeline indication - cheers

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[enj]

/remove-lifecycle stale
/lifecycle frozen

[oskarkorczak]

/remove-lifecycle stale

[nickle-ye]

hi .. is there any process on this?

[enj]

hi .. is there any process on this?

No. It is not high enough priority for anyone to work on at this time. I do not expect that to change.

[Exordian]

This one blocks us some projects on our side to use openshift. Routes currently needs to be handled like secrets (or require even more protection in case of wildcard certificates), which is not quite handy. Almost every other component in OpenShift (including the PV subsystem) allows the seperation of configuration and secrets. Routes are the only execption to the best of my knowledge.

[relief-melone]

looks like still nothing on this topic. Storing Private Keys and Certificate information completely blocks any projects from being easily shared in git repos across multiple teams. Would be see this as an important feature.
Sad to see something this important to go stale for more than 4 years now.

[enj]

/unassign

@stlaz @sttts @mfojtik

[memo42]

Wow, no priority for improving security and usability. That is pretty disappointing.

[alexandrsemak]

5 years later ~||||~