Influxdb: [feature request] Add support for --format=line-protocol to Influx CLI

[jonseymour]

Feature Request

Proposal:

Add support for a --format=line-protocol option to the influx CLI

Current behavior:

There is currently no easy way to export data from an influx database in a manner that can be easily imported into another influx database.

Desired behavior:

To be able export data from one influx database in influx line format.

Use case:

This mechanism would be useful for creating test cases to demonstrate issues with influxdb &/or kapacitor. Data from one database could be exported into line format and re-imported into another influx database with curl.

It is not intended that this mechanism would be used to mass data transfers - some kind of TSM dump facility would be better suited for that purpose (e.g. #5531 )

[cacack]

I would expand this to include support for exporting data in the line-protocol format to the API too so that it is available through curl. As it stands now, there isn't any direct means of exporting data and then re-importing it with out touching it. For a recent project, I used curl to export to JSON, then wrote a parser in ruby to transform it into line-protocol and finally re-imported using curl.

[MikeSchroll]

How close does this get? https://github.com/influxdata/influxdb/pull/7046

[martin-magakian]

Checkout my PR #7227

[jwilder]

Fixed via #7046. There is export support via influx_inspect.

[skladd]

@corylanou, @jwilder, @martin-magakian, is there still a chance to get line protocol as fully supported output format, i.e. in the influxd response writer as proposed in #7227?

influx_inspect export does not reflect recent data changes, esp. deletes:

writing out wal file data for mydb/rp_4w...WARNING: detected deletes in wal file.
Some series for "mydb/rp_4w" may be brought back by replaying this data.
To resolve, you can either let the shard snapshot prior to exporting the data
or manually editing the exported file.

My use case is data transfer between different influxdb instances. So I am using json now and have to transform the json data back to line protocol on the receiving machine. Besides a lot of work I am loosing the tags.

[1tgr]

Can we re-open this issue? influx_inspect export doesn't fit the requirement as it exports an entire database. Adding a line-protocol option to the influx CLI or the influxd response would allow for filtering.

[rbetts]

@1tgr If we implemented this via Kapacitor, would that fit your workflow?

[1tgr]

@rbetts Sure, what I'm looking to do right now is capture data for testing. The line protocol format is pretty compact and readable, and can be re-imported easily. I can also see this being useful for selectively migrating data between InfluxDB instances.