Victoriametrics: Recommended API library for Go?

[kmansoft]

This is a question, hope it's OK.

I'm looking to replace Influx with Victoria in my project. There is a client and so I will need a client library. The client both submits (writes) to the database and runs queries. It's written in Go.

The data uses series names like this:

process,item=systemd-resolve|systemd-resolve,node=56f202a6-0daebe93f0b0,sub=io_read
process,item=systemd-resolve|systemd-resolve,node=56f202a6-0daebe93f0b0,sub=io_write
process,item=systemd-resolve|systemd-resolve,node=56f202a6-0daebe93f0b0,sub=rss

The above are three data series (read, write, rss) for a process named systemd-resolve on a particular node (server).

Any recommendations for a client library?

Since Victoria supports Prometheus API, suppose that's the most obvious option. However, the official Prometheus GO library is in alpha and may not even be feature complete:

https://github.com/prometheus/client_golang

Also it's positioned as "an instrumentation library" not an API library, there is an API library part but I wonder if that even gets enough attention from developers.

I also notice that Victoria recently gained support of Influx API but only for the submit part. Let's say I wanted to use that, combined with Prometheus API for querying. Will I run into any trouble because of how data is mapped?

[hagen1778]

Hi @kmansoft !

Thanks for the question!

Prometheus itself supports only pull model of data scraping. That means there is no /write endpoint in Prometheus API. Golang library you mentioned was created to instrument Golang applications with metrics(counters, gauges etc.) and has no options to write them somewhere.

So it would be better to use Influx, OpenTSDB or Graphite libraries for writes. Please take a look on potential Influx compatibility issues here.

As for querying - I've never heard about PromQL libraries. Most of users query data via Grafana or Prometheus itself. However, it's just HTTP requests and it should be easy to implement. For parsing responses from Prometheus or VictoriaMetrics could be useful to take a look on this data model .

I hope other users could share their experience of migrating from Influx to VictoriaMetrics.

[kmansoft]

Thank you @hagen1778 for your response.

Does Victoria support tags - or a similar mechanism?

I'm looking to encode let's say "disk name" some way in Victoria, in Influx it can be done using tags or values, tags are preferred because even though they increase cardinality, tags support grouping and values don't.

https://docs.influxdata.com/influxdb/v1.7/concepts/schema_and_data_layout/

So I currently have:

disk,node=56f202a6-1248-4efe-8549-0daebe93f0b0,item=/dev/sda [time, value = ...]
disk,node=56f202a6-1248-4efe-8549-0daebe93f0b0,item=/dev/sdb [time, value = ...]
disk,node=56f202a6-1248-4efe-8549-0daebe93f0b0,item=/dev/sdc [time, value = ...]

and then I can make a query asking for values grouped by item

SELECT MEAN(value), sub FROM disk GROUP BY time(1m), sub

which returns time series data for each disk ("item") separately from other disks. Very nice.

Do you have any suggestions on how to map this out in Victoria?

[hagen1778]

Yes, VictoriaMetrics supports tags! Please take a look at this test and used testdata.
So basically when you send data like this:

measurement,tag1=value1,tag2=value2 field1=1.23,field2=123

Received tags will be converted to labels in VM data model. Then if you query VM with GET /api/v1/export?match={__name__!=\"\"} you supposed to get following response:

[
    {"metric":{"__name__":"measurement_field2","tag1":"value1","tag2":"value2"},"values":[123]},
    {"metric":{"__name__":"measurement_field1","tag1":"value1","tag2":"value2"},"values":[1.23]}
  ]

To filter data by tags(labels) just use them as key=value selector - GET /api/v1/export?match={__name__!="", tag1="value1"}

[kmansoft]

Thank you. Is it possible to get time series data fur all tags at once, grouped by tag value?

[hagen1778]

I'm not sure if got your question. Can you provide example?
In meantime, please check following article to see some PromQL query examples.

[kmansoft]

I think it's this example in prometheus docs

If the same fictional cluster scheduler exposed CPU usage metrics like the following for every instance:

instance_cpu_time_ns{app="lion", proc="web", rev="34d0f99", env="prod", job="cluster-manager"}
instance_cpu_time_ns{app="elephant", proc="worker", rev="34d0f99", env="prod", job="cluster-manager"}
instance_cpu_time_ns{app="turtle", proc="api", rev="4d3a513", env="prod", job="cluster-manager"}
instance_cpu_time_ns{app="fox", proc="widget", rev="4d3a513", env="prod", job="cluster-manager"}
...
...we could get the top 3 CPU users grouped by application (app) and process type (proc) like this:

topk(3, sum(rate(instance_cpu_time_ns[5m])) by

[valyala]

I'm looking to encode let's say "disk name" some way in Victoria, in Influx it can be done using tags or values, tags are preferred because even though they increase cardinality, tags support grouping and values don't.

As @hagen1778 already said, VictoriaMetrics supports tags. This chapter from README.md contains details about data model mapping between InfluxDB and VictoriaMetrics.

Is it possible to get time series data fur all tags at once, grouped by tag value?

The following PromQL query does just that:

avg(disk_value) by (item)

The query must be sent to /api/v1/query_range HTTP endpoint according to Prometheus querying API supported by VictoriaMetrics. This API has other useful endpoints - see docs for details.

[kmansoft]

Thank you

[kmansoft]

The query must be sent to /api/v1/query_range HTTP endpoint according to Prometheus querying API supported by VictoriaMetrics.

Is there something I need to do to enable Prometheus endpoint?

Just starting victoria-metrics with no parameters gives me this in "ss -tlnp":

LISTEN  0 128 0.0.0.0:8428 0.0.0.0:* users:(("victoria-metric",pid=18531,fd=8))        

But I don't see victoria-metric listening on any http ports (and Prometheus querying API is http, correct?)

[kmansoft]

@hagen1778

Then if you query VM with GET /api/v1/export?match={__name__!=\"\"} you supposed to get following response

http://localhost:8428/api/v1/export?match={__name__!=\"\"}

gets me this instead:

error in "/api/v1/export": cannot parse "{__name__!=\\"\\"}": cannot expand WITH expressions: missing "\\"\\"" value inside stringExpr

Running current Git code. what might be wrong?

[hagen1778]

Hi @kmansoft

I must be mistaken with copy&paste. Try to use following curl command:

curl 'http://localhost:8428/api/v1/export' -d 'match[]={__name__!=""}'

You can find some examples here

[kmansoft]

On the http listener - there is only one right? - which handles both InfluxDB and Prometheus APIs because these two use different http paths with none in common?

[valyala]

On the http listener - there is only one right? - which handles both InfluxDB and Prometheus APIs because these two use different http paths with none in common?

Correct.

[valyala]

But I don't see victoria-metric listening on any http ports (and Prometheus querying API is http, correct?)

VictoriaMetrics listens for http requests on the 8428 port by default. The port may be changed with -httpListenAddr command-line option. See victoria-metrics-prod -help for more details on the available config options.

[kmansoft]

On the http listener - there is only one right?
Correct

OK thanks.