Incubator-superset: [feature] Prometheus (or other TSDB) as data source

[alanmcruickshank]

Currently we have SQLAlchemy and Druid as datasources for Superset.

Would introducing a connection to a time series database like prometheus or graphite be compatible with the roadmap for Superset? There would be value for our use case to have superset as the central point for all our metrics whether or not they live in a SQL-like datastore.

A couple of considerations for implementation:

• Given the different method of querying (it's not SQL), it would have to be a top level connection type.
• There would be only one possible time column, just "time". There is no concept of different times fields.

Do people think this would be a good idea, or something that would actually get in the way more than anything?

[mistercrunch]

With the name Superset now we have to allow for querying all database :)

But yeah it should work ok and fits the roadmap. As you said it would behave essentially like a table with a single metric and single time column. It might not be the best experience as a bunch of the fields in the explore view become obsolete, but would allow to have dashboard that mix elements coming from multiple data sources.

[kelein]

The same need for prometheus data source support.

[xrmx]

Isn't grafana the de-facto ui for metrics?

[kelein]

@xrmx I have used grafana for a real-time monitor frontend, but I also want a statistics reporter with some metrics aggregation during week or month. Grafana seems not suitable.

[alanmcruickshank]

@xrmx I would agree for most cases.

I quite like the idea of superset being our one-stop-shop for metrics of all sorts together - which means I think it's valuable to support it - at least to the extent that we can get some metrics in and beside metrics from other sources. If those metrics point to something being up, then I imagine reverting back to Grafana for a deeper look would still be a pretty sensible plan.

[mistercrunch]

Notice: this issue has been closed because it has been inactive for 222 days. Feel free to comment and request for this issue to be reopened.

[gecube]

I'd like to revive thus issue

[raghu999]

+1 for prometheus support.

[tongsrv]

+1

[mistercrunch]

+1 for prometheus to support SQL :)

[QiXuanWang]

I liked the idea of superset as a central point while grafana being a TS monitor. Hopefully someone could bring up a turnkey solution :)

[nuharaf]

If you guys need timeseries database that support sql, maybe look at timescaledb

[tangyong]

+1

[gecube]

@nuhamind2

If you guys need timeseries database that support sql, maybe look at timescaledb

The TimeScaleDB is not the best option. The marketing specs of it are superior, but the real adoption faces with different struggles. So it is very interesting to get the native support for prometheus.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[andrew-howden]

please revive

[TimeStone]

+1 for this feature, its quite meaningful for me.

[flinz]

+1 Please revive!

[dpgaspar]

ping @villebro

[villebro]

I've been planning on writing a sqlalchemy connector for some time, and have been thinking specifically about targeting a TSDB (InfluxDB is my top contender, but I'm not an expert in this domain). Probably not something I will get around to doing right now, but hopefully sometime this year, unless someone beats me to it.

[hc128168]

+1

[akulkarni]

If you are considering a time-series database, I'd would (humbly) suggest TimescaleDB again (note I work on TimescaleDB):

• Provides a full SQL interface for time-series data

• Outperforms InfluxDB

• Handles high-cardinality well

• Can be used as long-term storage with Prometheus

In particular, it may even work right out of the box with Superset.

[willbarrett]

Pinning this issue as there's strong interest in an implementation and it will keep the idle bot at bay.

[SamuelHaleMN]

I'd love to see support for Prometheus/PromQL as well!

[bkyryliuk]

A short cut here could be to embed grafana charts as iframes to the superset dashboards. It doesn't solve the connector problem, but allows to have analytical and performance monitoring charts to be side by side.

[valyala]

It would be great to add VictoriaMetrics as a datasource for Superset. It provides Prometheus querying API - see docs. Additionally to this, it provides export API, which allows exporting raw data according to the given filters on time range and on labels.

[gecube]

Hi @valyala ! Thanks for your message! I will be glad to see VM as datasource for SuperSet too!

[pabrahamusa]

Prometheus is something used by many Kubernetes projects now a days. And people who are already using superset would love to see the metrics there alsong with their other data. So it will be great to create a prometheus driver for Superset..!