Influxdb: DROP SERIES is not reflected in SHOW TAG VALUES

[arindamchoudhury]

Hi,

I am using influxdb 1.6.2

When I do
DROP SERIES FROM "measurement" WHERE "tag_name" = 'tag-value'

this gets reflected in
SHOW SERIES FROM "measurement"

But

SHOW TAG VALUES FROM "measurement" WITH KEY = "tag_name"

still shows the tag-value

[e-dard]

Hi @arindamchoudhury I was able to reproduce this on the tsi1 index, but no the inmem index. Thanks for the report.

> create database db
> insert cpu,region=west value=1
> insert cpu,region=east value=1
>
> DROP SERIES FROM "cpu" WHERE "region"='west'
>
> SHOW SERIES FROM "cpu"
key
---
cpu,region=east
>
> SHOW TAG VALUES FROM "cpu" WITH KEY = "region"
name: cpu
key    value
---    -----
region east
region west
>
>

[lpic10]

Any news on this?
I use SHOW TAG VALUES FROM system WITH KEY=host on grafana for host template, but that finds a lot of hosts already deleted from influxdb

[benbjohnson]

@lpic10 Sorry for the delay on this. We're looking into it.

@e-dard The issue stems from this line here:

https://github.com/influxdata/influxdb/blob/master/tsdb/engine/tsm1/engine.go#L1475

The inmem index has all data in-memory so it can easily compute the new measurement index data. However, Rebuild() on tsi1 is a no-op because that would be an expensive operation.

One option would be to check for the existence of all key/value pairs on all the series deleted. That also seems a bit expensive but I'm not sure how else we handle it.

[ushuz]

Same issue for me on 1.6.4 with inmem index. There are several other reports in #5092.

SHOW TAG VALUES WITH KEY = "some-tag" returns tags from dropped series, however SHOW TAG VALUES WITH KEY = "some_tag" WHERE another_tag = 'another_tag' won't return those dropped tags.

---

A workaround for me: SHOW TAG VALUES WITH KEY = "some_tag" WHERE some_tag != ''

[lpic10]

@lpic10 Sorry for the delay on this. We're looking into it.

@e-dard The issue stems from this line here:

https://github.com/influxdata/influxdb/blob/master/tsdb/engine/tsm1/engine.go#L1475

The inmem index has all data in-memory so it can easily compute the new measurement index data. However, Rebuild() on tsi1 is a no-op because that would be an expensive operation.

One option would be to check for the existence of all key/value pairs on all the series deleted. That also seems a bit expensive but I'm not sure how else we handle it.

Is there some workaround, eg. manually rebuilding the tsi indexes?

[cocampbe]

I just noticed this. Prior to running 1.7.3 these values would be removed after dropping all series.

> drop series where host = 'dhcp-10-96-109-139' >

> show series where host = 'dhcp-10-96-109-139' >

When I do a query to show tag values, I still see the host.

> show tag values from "system" with key = "host" name: system
key value --- -----
host dhcp-10-96-109-112 host dhcp-10-96-109-125
host dhcp-10-96-109-134 host dhcp-10-96-109-139

[cocampbe]

Well. Restarting the db ended up being a work around for me.

[juliencorsini]

Hi there,
Any planned fix on this issue? I am also seeing this behavior in 1.7.4.

[stuartmacfarlane]

Same issue here on influxdb-1.7.3-1.x86_64 running pretty much default settings in the config file.
Restarting the service clears down the lingering tags.

[arindamchoudhury]

Is there any manual way to fix it? without restarting.

[lishunan246]

The same with version 1.7.4 on Ubuntu x64.

[jpldev]

Any news about this?
Thanks

[Pshellvon]

Same thing with InfluxDB 1.7.4 on Debian 9

[dandv]

Confirmed in 1.7.7 on Ubuntu 18.04 with XFS. Tried to reproduce with a sample measurement, but it seems to happen only for larger databases.

Related: https://community.influxdata.com/t/commad-to-delete-a-particular-series-from-influxdb/5600/18?u=dandv

UPDATE: systemctl restart influxd fixed the problem - the zombie tags are gone.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[arindamchoudhury]

this still happens. please dont make it stale.

[sbm17]

Is there any resolution for this? Tried to drop series and restart the db. I still see the values.

[Esity]

@russorat is there a reason why the stale-exempt label was removed?

[nicolas17]

This issue is obviously present and easily reproducible. It's not requiring more information from users. Why do you need "activity" on the issue? It's not stale until it's fixed.

Don't make me post a comment every month to keep it open.

[russorat]

@Esity @nicolas17 we got rid of stalebot so there's no need for that label anymore.

[afenioux]

have you tried to rebuild tsi index?
(it worked for me)

https://docs.influxdata.com/influxdb/v1.7/administration/rebuild-tsi-index/

[doglex]

A workaround: clone to a temp database and then clone back.

create database temp
use udp
select * into temp..:MEASUREMENT from tail group by *
drop database udp
create database udp
use temp
select * into udp..:MEASUREMENT from tail group by *

[arindamchoudhury]

Hi,

just a question out of curiosity, it was included in milestone 1.8.0 which is already released. will it be reconsidered for another milestone?

[hanej]

Is there an update on this issue?

[dominikkv]

Yo. I'm asking for a friend.