Quarkus: ConfigMaps/Secrets prioritization when consuming more than one using kubernetes-config extension

[dsanchor]

Describe the bug
kubernetes-config extension supports setting more than one ConfigMap names under the property quarkus.kubernetes-config.config-maps.
What I have observed is that you cannot guarantee which ConfigMap is considered first, so there is no order or prioritization for overriding same properties set in different ConfigMaps

Same for Secrets.

Expected behavior

I would expect some kind of prioritization, for instance, aligned with the order defined in the quarkus.kubernetes-config.config-maps property, where any ConfigMap defined below the others would override any existing property from previous ConfigMaps.

Actual behavior

Order is now unknown.

To Reproduce

Application source code to test it: https://github.com/dsanchor/quarkus-hello-world/tree/77a32c8646e85eb3d4f276d0435dbf49f92c18a9

Configuration
application.properties

"%prod":
  quarkus:
    kubernetes-config:
      enabled: true
      config-maps:
        - quarkus-hello-world
        - global-ops-quarkus-hello-world

ConfigMaps

kind: List
apiVersion: v1
items:
- apiVersion: v1
  data:
    application.yaml: 'message: hola'
  kind: ConfigMap
  metadata:
     name: global-ops-quarkus-hello-world
- apiVersion: v1
  data:
    application.yaml: |-
      message: hello
      environment:
        name: dev
  kind: ConfigMap
  metadata:
    name: quarkus-hello-world

I would expect the message to be "hola".

Screenshots

Environment (please complete the following information):

Additional context

[quarkusbot]

/cc @geoand

[geoand]

Thanks for reporting!

So you expect the latest one you set to take precedence?

@Ladicek @iocanel what do you think is the least surprising behavior?

[kenfinnigan]

I'm not familiar with how the ConfigMap stuff is implemented, but does it require each config map to be its own ConfigSource? To then be able to set order/priority?

[geoand]

@kenfinnigan exactly!

The question is, which ones should have priority? The first ones, or the last ones?

[dsanchor]

Thanks for reporting!

So you expect the latest one you set to take precedence?

Probably yes, that was my intuition when trying it. Honestly, I would not care if the order is different but well documented.

@Ladicek @iocanel what do you think is the least surprising behavior?

[kenfinnigan]

@geoand I wonder if it shouldn't be a list, but instead something like:

"%prod":
  quarkus:
    kubernetes-config:
      enabled: true
      config-maps:
        - quarkus-hello-world
            - 200
        - global-ops-quarkus-hello-world
            - 300

I'm butchering the format, but hopefully you get my meaning

[geoand]

I kinda want to avoid that, because it could get dangerous - I really think don't think that ConfigMap properties should override system properties or env vars.

That's the reason the ConfigMap currently has a value of 270

[kenfinnigan]

True, but there's nothing stopping anyone doing that directly anyway.

We could add a big warning during build/runtime that says the priority means it is ahead of env vars.

The other option is to just define an ordering of config maps with that number and not the actual priority. That way the order is defined in the file, as opposed to based on ordering of the names only

[geoand]

That's true.

I was initially thinking of just making a rule about the order in which they appear in configuration, but I'll go ahead with your suggestion if others like it as well.

[dsanchor]

From a quarkus user point of view, I would not have to know about the meaning of that magic number (270)... why should I care?
From the docs it is clear that ConfigMaps properties would not override env vars nor sys properties.
Would not be enough to set the order in some way where that order would only affect to this list of ConfigMaps?
I may be losing track, apologize. :-)

[geoand]

Would not be enough to set the order in some way where that order would only affect to this list of ConfigMaps?

That is what I am advocating for as well :)

[Ladicek]

Inventing some syntax for the .properties config format (which we, unfortunately, advertise as the default) sounds ugly to me. Giving a meaning to the ConfigMap/Secret order sounds rather nice, and perhaps even natural. Big question is, is there a priority between ConfigMaps and Secrets? :-)

[geoand]

Good question about secrets and config maps - they currently have the same priority, but we should address the issue here as well.

I assume secrets should be of higher priority?

[dsanchor]

I thought about that as well and I would also assume that secrets would have a higher priority than configmaps. About the order in the list, whatever it is decided would make sense to users if it is properly documented.

[geoand]

I'll put something together based on the order later on tonight (or tomorrow morning) and hopefully we can get it into the next 1.7.x release

[Ladicek]

I would also expect that Secrets are higher priority than ConfigMaps, so it seems we've got a quorum here! :grin:

[geoand]

https://github.com/quarkusio/quarkus/pull/11796 should take care of this