Skywalking: how to erase oversize data in ES gracefully

Created on 15 Jan 2019  路  2Comments  路  Source: apache/skywalking

hello, we created many trace data in ES in production environment, when we delete all of the nodes data directly in ES we encountered UI not available, always pops up 500 error, util we set up a new skywalking collector and restart the agents. So do we have a graceful way to erase the useless data in ES without restart or redeploy the collectors and agents ?

question
Source
picture liu316484231

Most helpful comment

Deleting data in ElasticSearch is not very high performance. The suggestions are

  1. Set the ElasticSearch with more nodes.
  2. Set a very long TTL to make it disable, and write your own scripts/codes to implement TTL at night
  3. Similar with (2), contribute the codes to use, which support deleting data at several certain time points each day if you want to contribute to us.
picture wu-sheng on 15 Jan 2019
👍2

All 2 comments

the skywalking version we use is 5.0GA

liu316484231 picture liu316484231 on 15 Jan 2019

Deleting data in ElasticSearch is not very high performance. The suggestions are

  1. Set the ElasticSearch with more nodes.
  2. Set a very long TTL to make it disable, and write your own scripts/codes to implement TTL at night
  3. Similar with (2), contribute the codes to use, which support deleting data at several certain time points each day if you want to contribute to us.
wu-sheng picture wu-sheng on 15 Jan 2019
👍2
Was this page helpful?
0 / 5 - 0 ratings

Related issues

EvanLjp picture EvanLjp  路  5Comments
cyejing picture cyejing  路  5Comments
hanahmily picture hanahmily  路  5Comments
dingsongjie picture dingsongjie  路  4Comments
wu-sheng picture wu-sheng  路  5Comments