Influxdb: Support delete with predicate functionality via /api/v2/delete

[stuartcarnie]

What

To ensure feature parity with Cloud 2.0, InfluxDB OSS 2.0 requires an implementation of the _delete with predicate_ API via /api/v2/delete. This will require enhancements to the tsdb package and storage engine.

Background

The API was disabled per #19580 and returns 501 Not Implemented as there was no equivalent 1.x tsdb storage API to support this feature. The following DeleteSeries API was evaluated

https://github.com/influxdata/influxdb/blob/b917d8d9b069ffb416cb522c55bc1e7a82dcd34d/tsdb/store.go#L1266

however, due to differences between series keys in TSM 2.0 vs 1.x, it is unable to support fine-grained delete functionality for a predicate that targets a specific field:

_measurement = 'cpu' AND _field = 'usage_user'

[benbjohnson]

Update: I'm in the thick of it and there's more code to move over from idpe than I expected. Hopefully this coming week it will be done.

[8none1]

Per discussions with the 2.0 GA team this is being de-prioritised in favour of backup and restore:
https://github.com/influxdata/influxdb/issues/19316

[cgfoed]

I'm sorry to ask a seemingly stupid question, but im trying to delete data using CLI for nearly an hour now (and also tried the curl method).
How exactly is it possible to delete data right now in Version 2.0.0 (37cc047)?

My last tries where looking like that:
influx delete --org myOrg --bucket test_bucket --start '1970-01-01T00:00:00.00Z' --stop '2020-01-01T00:00:00.00Z' --token VeryLongTokenFromRootHere
resulting in
Error: Failed to delete data: Not implemented.

[timhallinflux]

@cgfoed For OSS, this issue states that it isn't possible to selective delete data. You have to drop the entire bucket until this issue is resolved.

[snenkov]

When is this issue expected to be resolved? Not managing to delete data is a big issue.

[zbot473]

I'm not even able to delete without predicate, and backup/restore is not really an alternative IMHO.

I understand that OSS is probably lower priority because we aren't paying, but please consider fixing this quickly.

[doivosevic]

While waiting, what we do is delete buckets :/

On Wed, Dec 2, 2020 at 4:23 AM Mihir Patil [e-mail] wrote:

I'm not even able to delete without predicate, and backup/restore is not
really an alternative IMHO.

I understand that OSS is probably lower priority because we aren't paying,
but please consider fixing this quickly.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/influxdata/influxdb/issues/19635#issuecomment-736963869,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABQVJDU4RG75WXVULR475R3SSWXL7ANCNFSM4RYSBZFA
.

[benbjohnson]

Pending PR for the fix: https://github.com/influxdata/influxdb/pull/20236

[zbot473]

Thanks!

[timhallinflux]

plan is to drop this into 2.0.3 early next week.

[agolovenko]

Thanks for fixing this. However, I must say that this is not the first "surprise" while using what is supposed to be "GA version". Seems like your marketing department runs far ahead of development. As a result: trying to migrate my project to Influx 2x looked like a premature decision in hindsight and I ended up wasting quite some time.