Splunk-connect-for-kubernetes: extension of Namespace to Index Routing

[Joerg-L]

What would you like to be added:
The option to add some prefix for the index name.
So as an example ProjectName_namespace

Why is this needed:
We are sharing the Splunk instance with several teams.
The indexes will have some naming strategy like ProjectName_main or TeamName_main.

In addition, when searching through multiple indexes we can use ProjectName_* for naming all the indexes of the project.

[matthewmodestino]

so you would like to have something like option for indexRoutingPrepend ?

      indexRouting:
      # indexRoutingDefaultIndex tells which index to use for the default kubenetes namespace. Used with indexRouting. Default is main.
      indexRoutingDefaultIndex:

Also just an FYI, you can already achieve this using the jq_transformer plugin instead of the index routing feature. We have found, while the index routing feature is easy to configure, there is a sweet spot for it's use. In large environments you may quickly get into hundreds/thousands of indexes. I do not recommend going down that path if you find the number of namespaces/projects gets into high triple digits

[chaitanyaphalak]

I would recommend using the jq transformer

[Joerg-L]

@matthewmodestino
If you understand indexRoutingPrepend as an option to define a string, which will be added to the namepspace name, then thats fine for me.

not sure if "jq transformer" will help me.
My problem is that we have support users, who shouldn't have access to log lines of "prod" namespace, but admins which should have access.

and after some reading the info was: "no row level security exists, use separate indexes"

[matthewmodestino]

the jq_transformer will definitely help...it can dynamically set your index field based on many things.

I will try and produce a sample for you when I get the chance. Likely on my flight home tomorrow.

[Joerg-L]

@matthewmodestino thanks a lot for your help

[leemmcc]

I think we've been trying to accomplish something similar. We have the same project name in multiple environments, but would like a prefix outside of cluster_name on our indexes. We've been (sort of) successful using a record transformer, and then setting the index_key on that param, but having some issues with it at the moment, I'm sure there's a better way.

   <filter **>
      @type record_transformer
      enable_ruby true
      <record>
        namespace_index prefix_${record["namespace"]}
      </record>
   </filter>

then in output:

    index_key namespace_index

This way we aren't actually changing the namespace name, just the index.

[chaitanyaphalak]

I think this can be easily solved with using annotations for the namespace. Closing as resolved with https://github.com/splunk/splunk-connect-for-kubernetes/pull/294