Kustomize: ConfigMapGenerator: Merge config files in configmap without the filenames

[KarstenSiemer]

Hello,

i'd like to merge a splitted configuration file in a configmap to a single file again. So that all the resources i used produce a single file in the map.

configMapGenerator:
  - name: prometheus-config
    files:
      - config/global.yaml
      - config/scrape_configs/jmx_exporter_kafka.yaml
      - config/scrape_configs/kafka_exporter_kafka.yaml
      - config/scrape_configs/kubernetes-apiservers.yaml
      - config/scrape_configs/kubernetes-cadvisor.yaml
      - config/scrape_configs/kubernetes-nodes.yaml
      - config/scrape_configs/kubernetes-pods.yaml
      - config/scrape_configs/kubernetes-redis.yaml
      - config/scrape_configs/node_exporter_kafka.yaml
      - config/scrape_configs/prometheus.yaml
      - config/scrape_configs/wmi_exporter.yaml

This creates a configmap with such information:

apiVersion: v1
data:
  global.yaml: |
....
kubernetes-cadvisor.yaml: |+
....
kubernetes-nodes.yaml: |+
...

But i'd like all the configuration to be merged into the global.yaml or any other single file.
How can i achieve this?
I tried the "behavior: merge" and created a configmap in the base and tried to merge from the overlay, but that did create the same configmap for me

[Liujingfang1]

In configMapGenerator, one file is corresponding to one pair. The key is the file name and the value is the file content. It doesn't merge all files into one value. You need to cat all the file content into global.yaml and list only global.yaml in the configMapGenerator.

[Liujingfang1]

I'm thinking this request might be solved by #304. @monopole WDYT

[Liujingfang1]

We will add support for #304. Close as duplicate.

[yellowmegaman]

304 is closed. Probably should be reopened.

Got configmap with application.conf, would like to merge common/staging stuff in it, not using different filenames.
Example:

base:

configMapGenerator:
- name: my-configmap
  files:
  - application.conf

application.conf

param1=5
param2=6

staging:

bases:
- ../../base
namePrefix: staging-
nameSuffix: -v1
configMapGenerator:
- name: my-configmap
  behavior: merge
  files:
  - application.conf

application.conf

param3=2131r1fc
param4=asdsa

All i want is to concatenate staging values into the same file ( app can only work with 1 file - application.conf), so it look like this:

application.conf

param1=5
param2=6
param3=2131r1fc
param4=asdsa

AFAIK there's no way to do it right now.
https://github.com/kubernetes-sigs/kustomize/blob/master/examples/combineConfigs.md covers this with different filenames.

[ig0rsky]

I think this issue should be reopened. There's no easy way to create a configMap using the generator from the base dir. without ending up with complex keys at the moment. @Liujingfang1

[ItsLeeOwen]

For anyone else landing here from google searches...

Jeff Regan's presentation demonstrated configMapGenerator files rendering nicely merged configMap data output without filenames. https://youtu.be/WWJDbHo-OeY?t=15m30s

data:
  color: blue

However following his code example the configMap data output looks more like the following example, which is consistent with kubectl --from-file docs:

data:
  common.properties: |
    color=blue

To generate a configmap from a file with output that reads like a standard multiline key/value list you can use:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

configMapGenerator:
  - name: your-configmap-name
    behavior: merge
    env: your/path/to/.env

[jbraghu]

I tried following Jeff's presentation and I couldn't achieve a merged configmap. Here's what I am trying:

base:

configMapGenerator:
- name: app-configmap
  files:
  - application.properties

application.properties:

key1=value1
key2=value2
key3=value3

production:

bases:
- ../base
namePrefix: prod-
configMapGenerator:
- name: app-configmap
  files:
  - application-prod.properties
  behavior: merge

application-prod.properties:

key3=newvalue3
key4=value4

here's the kustomize output:

apiVersion: v1
data:
  application-prod.properties: |
    key4=value4
    key3=newvalue3
  application.properties: |
    key1=value1
    key2=value2
    key3=value3
kind: ConfigMap
metadata:
  annotations: {}
  labels: {}
  name: prod-app-configmap-mh44k9mfgh

what I was expecting:

apiVersion: v1
data:
  application.properties: |
    key1=value1
    key2=value2
    key3=newvalue3         <-----
    key4=value4                <-----
kind: ConfigMap
metadata:
  annotations: {}
  labels: {}
  name: prod-app-configmap-mh44k9mfgh

Essentially, a single "application.properties" that has the content merged.

@ItsLeeOwen @Liujingfang1 : is this kind of merging achievable?

[anshul-patel-infostretch]

@ItsLeeOwen @Liujingfang1 : is this kind of merging achievable?

Facing the same issue @jbraghu , Any solution?

[mr-karan]

^ +1 bump for visibility.

[Liujingfang1]

@mr-karan @anshul-patel-infostretch The ConfigMapGenerator is not planned to support this type of merging functionality. You are welcome to write a generator plugin for this. There is a list of example generators under plugin/someteam.example.com/v1. You can put your custom configmap generator here as well.

[davisford]

For anyone else landing here from google searches...

Jeff Regan's presentation demonstrated configMapGenerator files rendering nicely merged configMap data output without filenames. https://youtu.be/WWJDbHo-OeY?t=15m30s

When I try his example, this is the output I get, which isn't correct:

apiVersion: v1
data:
  common.properties: |-
    color=blue
    height=10m
  secret.properties: dbpassword=foo
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app: hello
  name: staging-myCMap-2t62g25htc