Influxdb: ERR: no data received, TCP connection refused, high CPU and memory usage

[oliv3]

Running InfluxDB 1.7.4 on Debian buster/sid
Kernel: Linux 4.19.0-3-amd64 #1 SMP Debian 4.19.20-1 (2019-02-11) x86_64 GNU/Linux

All was working fine until now, a simple SELECT on a measurement fails like this:

> select count(*) from my_measurement
ERR: no data received
> select count(*) from my_measurement
ERR: Post http://localhost:8086/query?chunked=true&db=test&epoch=ns&q=select+count%28%2A%29+from+my_measurement: dial tcp 127.0.0.1:8086: connect: connection refused
> select count(*) from my_measurement
ERR: Post http://localhost:8086/query?chunked=true&db=test&epoch=ns&q=select+count%28%2A%29+from+my_measurement: dial tcp 127.0.0.1:8086: connect: connection refused
> select count(*) from my_measurement
ERR: no data received

At the same time, CPU usage reaches 200%, and memory usage also grows:

PID   USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND 
21836 influxdb  20   0 8022072   6.0g 828588 S 200.0  79.1   2:07.78 influxd

[oliv3]

After making a backup of all the databases, I tried droping the measurement, took some time with high CPU usage, still the measurement appears:

> drop measurement my_measurement
> show measurements
name: measurements
name
----
test
my_measurement
> select count(*) from my_measurement
>

Also high CPU usage for some seconds in the last SELECT. Trying to drop the measurement again, it's still there:

> drop measurement my_measurement
> show measurements
name: measurements
name
----
test
my_measurement

[oliv3]

Also, I don't have that much series (most likely < 1 million). After backing up all databases in ./influxdb_backup directory, here is the disk usage:

$ du -sh /var/lib/influxdb influxdb_backup
768M    /var/lib/influxdb
496M    influxdb_backup

There's also (I think) enough RAM:

$ free -h
              total        used        free      shared  buff/cache   available
Mem:          7.6Gi       3.7Gi       2.3Gi        81Mi       1.6Gi       3.5Gi
Swap:            0B          0B          0B

[oliv3]

Same problem exists after droping the database containing my_measurement and restoring the backup.

[yashpatels]

Did you found any solution ?
Actually I am also getting the same error.

[dandv]

Same problem after using SELECT ... INTO. Rebooting didn't help. Looks like the database is busted.

I've put up for a few years with InfluxDB's annoying limitations, but data corruption is too much. I will definitely NOT choose InfluxDB for my next time series project. TimescaleDB is based on PostgreSQL, which has a far better reliability track record.

[ksk438]

I have gotten the same exact issues, although with a larger table ~20 million rows (although I was under the impression it would be able to handle much more).

[fridayhoho]

same problem ,

I've write a program to transport counts of 379,773,838 records from MySql to InfluxDB1.7 OSS

`show databases
name: databases

name

_internal
collectd
mSensorDB
terminal_sensors
terminal_sensors_history`

terminal_sensors is old database ,everything is ok
` use terminal_sensors
Using database terminal_sensors

select * from history_sensor_t order by time desc limit 1
name: history_sensor_t
time alarmLevel dataValue sensorNo status tno
---- ---------- --------- -------- ------ ---
1563763931270000000 02 67 E11 1 LZSGAJ0056`

but the new database terminal_sensors_history

can't select , it always refused
`
SHOW MEASUREMENTS
name: measurements

name

history_sensor_t

select * from history_sensor_t order by time limit 1
ERR: no data received
`

During the transporting , the disk space is full, and influxDB got killed by system

`[root@VM_33_2_centos /var/lib/influxdb/data]# du -h -d1

• 1.5G ./terminal_sensors_history
• 189M ./terminal_sensors
• 251M ./mSensorDB
• 64M ./collectd
• 27M ./_internal
• 2.0G .

`

I did not config the log to write to file....

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had recent activity. Please reopen if this issue is still important to you. Thank you for your contributions.

[oliv3]

How do I reopen ?