Victoriametrics: Support additional labels on prometheus import API

[ledmonster]

Is your feature request related to a problem? Please describe.

When I try to substitute VictoriaMetrics for Prometheus+Pushgateway, lack of additional labels feature on prometheus import API (/api/v1/import/prometheus) makes work harder. Pushgateway does support additional labels on API, and I want to use it in the import API, too.

Describe the solution you'd like

One idea is to support same API as Pushgateway.

/api/v1/import/prometheus{/<LABEL_NAME>/<LABEL_VALUE>}

or some query strings (e.g. labels) may also work.

/api/v1/import/prometheus?labels={<LABEL_NAME>%3D<LABEL_VALUE>,}

Describe alternatives you've considered

Now I'm using remote_write API instead. I parse Prometheus exposition format to WriteRequest and post it to the API with following code.

• https://gist.github.com/ledmonster/7d0de61a788b164a510c0359465a41f4

[valyala]

@ledmonster , this sounds like very useful feature request! I think it would be great adding labels={<LABEL_NAME>:<LABEL_VALUE>,...} query arg to all the supported data ingestion protocols that work over http.

[valyala]

In the mean time you may try adding labels via relabeling with -relabelConfig command-line flag - see these docs for details.

[ledmonster]

I think it would be great adding labels={:,...} query arg to all the supported data ingestion protocols that work over http.

That's great!!

I'm also using relabeling feature, thank you.

[valyala]

The commit f41b36bb9a162e5cc184da5e0236e3c441776c85 adds ability to add extra labels to data imported via the following http endpoints:

• /api/v1/import
• /api/v1/import/csv
• /api/v1/import/prometheus

Extra labels must be enumerated in extra_label=name=value query args. Multiple query args are supported. For example, {foo="bar"} and {baz="aaa"} labels must be added to all the imported metrics via /api/v1/import/prometheus?extra_label=foo=bar&extra_label=baz=aaa.

@ledmonster , could you build VictoriaMetrics from the commit f41b36bb9a162e5cc184da5e0236e3c441776c85 and verify whether the functionality for adding extra labels works as expected? See build instructions.

[ledmonster]

@valyala I'll try it later, thank you!!

[ledmonster]

Thank you. I confirmed /api/v1/import/prometheus, and it works.

$ curl --data-binary @data/node-exporter.prom  'http://localhost:8428/api/v1/import/prometheus?extra_label=job=curl&extra_label=instance=pc' 
$ curl -G 'http://localhost:8428/api/v1/export' --data-urlencode 'match[]=process_max_fds{job="curl"}' | jq
{
  "metric": {
    "__name__": "process_max_fds",
    "job": "curl",
    "instance": "pc"
  },
  "values": [
    1024
  ],
  "timestamps": [
    1599126287505
  ]
}

Also, it worked with urlencoded query parameter: /api/v1/import/prometheus?extra_label=job%3Dcurl&extra_label=instance%3Dpc

[valyala]

The ability to add extra labels has been added into v1.40.1. Closing feature request as done.