Influxdb: [feature request] DROP MEASUREMENT should accept regex

[beckettsean]

e.g. DROP MEASUREMENT /^foo/ should drop all measurements starting with "foo".

[beckettsean]

related to https://github.com/influxdb/influxdb/issues/4351

@dgnorton not sure if it makes sense to tackle both at once, probably not. However, once we support SHOW with regex, it's natural to want DROP with regex.

[huhongbo]

4501 has support show measurement ,how about drop support?

[beckettsean]

@huhongbo that PR only addressed the SHOW MEASUREMENT command.

[vchekan]

It would be nice to have a "dry-run" modifier for DROP command.

[bentatham]

+1

[bentatham]

As a workaround (for others that might need this now), I did this (to delete all measurements starting with spinach:

influx --execute 'show measurements' --database=db | grep "^spinach" | xargs -I{} influx --database=db --execute 'drop measurement "{}"'

Not particularly efficient of course, but gets the job done.

[jwilder]

This is currently supported via delete from /^foo/. A delete from without any filter criteria works the same as a drop measurement.

[Baiteman]

I do not recommend the previous suggestion when a lot of data is affected. Turns out it is a quick way to max out your machine's ram (from 2 to ~ 13 GB in about half a minute xD)

[ninjarocketman]

Also delete does not drop the series from the index. So deleting data with delete can lead to a polluted index. Regex support for drop measurement would be great to fully clean data.

[AvinKumar]

Any updates on this feature.? I am in badly need a fix. In my case lot of junk measurements got created and I'm struggling to delete them. I also tried the suggestion given by bentatham but that didn't work too.

influx --database='mydb' -host 'localhost' -port '8086' -username 'test' -password 'test' --execute 'show measurements' | grep "^java" | xargs -I{} influx --database='mydb' -host 'localhost' -port '8086' -username 'test' -password 'test' --execute 'drop measurement "{}"'

Influxdb version: 1.5.2

[TattiQ]

@AvinKumar why didn't it work? what did you see exactly when running this command?

I used to run

influx --execute 'show measurements' --database=db | grep "^measurement_name-*" | head -20 | xargs -I{} influx --database=db --execute 'drop measurement "{}"'

I deleted the data in portions since full compaction kicked in and this is a resource intensive process as it turned out.

Full Compactions - Full compactions run when a shard has become cold for writes for long time, or when deletes have occurred on the shard. Full compactions produce an optimal set of TSM files and include all optimizations from Level and Index Optimization compactions. Once a shard is fully compacted, no other compactions will run on it unless new writes or deletes are stored.

[TattiQ]

of course I agree that it's time to optimize influx for deletions and make it possible to drop measurements using regex. It's pretty easy to get a situation where a lot of them are created by some code and this needs to be handled somehow

[dmoebius]

In this SO post someone mentions that DROP SERIES supports regexps. I cannot find this in the official v1.6 docs. Is it true?
What's the difference between DROP MEASUREMENT and DROP SERIES?!?

[AvinKumar]

Hi Tatyana,
I am extremely sorry for late reply, somehow I missed your mail.

In my case multiple junk measurements were created like the ones shown
below.

Caused by: javax.servlet.jsp.JspTagException: Digest mismatch
java.lang.IndexOutOfBoundsException: Index: 1
... 91 more
TotalStackTree: java.lang.IllegalArgumentException: Illegal string value
\'job_listing\'A=0\'. Legal values are \'none\'

I tried deleting them the way you have given but with no luck.
Influx Command:
$influx -host localhost -username admin -password pass -port 8086 --execute
'show measurements' --database=analytics | grep "^Caused*" | head -20 |
xargs -I{} influx -host localhost -username admin -password pass -port 8086
--database=analytics --execute 'drop measurement "{}"'

I get no error in the console.

Am I missing something here.?

-Avin

On Fri, Jul 27, 2018 at 9:03 PM, Dirk Möbius [e-mail]
wrote:

In this SO post
https://stackoverflow.com/questions/38587898/in-influxdb-how-to-delete-all-measurements/38647418
someone mentions that DROP SERIES supports regexps. I cannot find this in
the official v1.6 docs. Is it true?
What's the difference between DROP MEASUREMENT and DROP SERIES?!?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/influxdata/influxdb/issues/4275#issuecomment-408455335,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AUN3gmHH1DtDcqkAibfdHiwZEZ-KOgcpks5uKzLIgaJpZM4GGIGq
.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had recent activity. Please reopen if this issue is still important to you. Thank you for your contributions.

[reloxx13]

In this SO post someone mentions that DROP SERIES supports regexps. I cannot find this in the official v1.6 docs. Is it true?
What's the difference between DROP MEASUREMENT and DROP SERIES?!?

Can confirm drop series accept regex

used
drop series from /^*\.POWER/

[rsaim]

+1 for this feature.

[mnachmia31]

+1 for adding this feature