Krypto-trading-bot: Zombie [nation] - CPU usage over 100% and bot stops

[Corfucinas]

Hi,

For the past three weeks I been experiencing sudden and unexplained spikes in CPU usage by K.
I though I might need a higher CPU core to deal with a normal part of the process, so I migrated servers and got a 3.2 GHz-no avail.
Once it reaches 100% it just stops sending orders, the last orders submitted will stay in place ignoring FV until they get executed, then the position just hangs.

Three times only this week

Quick fix - something like Monit to monitor and if it goes over n%> allowed CPU just restart (by restarting the process it behaves normally and doesn't go over 6% usage).

But if somebody doesnt have access to the server for any reason (they go out for the weekend) then they would get basically find themselves with a stucked position which is dangerous.
Also, if you use the server for any other process they would basically crashed (mine just crashed and had to hard restart it on this occasion).

---

[ctubio]

what exchange are you using? bitmex? or hitbtc/bequant?

are you on linux? or macos? or winy?

[Corfucinas]

Bitmex, Linux (Ubuntu 19.04 x64)

[ctubio]

coOl'' i believe then is bitmex spamming buncha information (filled market trades) every now and then

lemme rereview bitmex data carefully for a few to see if some event is firing nonstop every millisecond (like market trades)

thanks for the report'''

[Corfucinas]

Today's event happened at between 7.30/8.30am UTC+8

Let me know if I can do anything to help your investigation 🕵️‍♂️

[ctubio]

i got all info already ty''' just i need to identify and amend the situation; many thanks for your help'¡

:koala: sorry ima leave this here to be used meanwhile debugging:
https://www.youtube.com/watch?v=Ayq_I1NU2Q8

[Corfucinas]

Happened again, just when BTC dropped, woke up with frozen orders since 1am (UTC+8)

[asfbhero]

This might be happening for me too. Seems to be during fast movements, so bitmex spamming us with fast orders and the bot getting overloaded somehow makes sense.

[asfbhero]

Yes this is 100% happening on my end. Anytime there is a large and fast move the bot freezes up. I usually end up having to restart the whole VM before it will work again.

When it locks up the current quote price at the top of the order book (the ones where it shows your orders) shows a price of 0. Not sure if that's helpful or not, but I don't get anything weird in the logs, so that's all I've got to go on.

[Corfucinas]

Hey, just following up on stale comments/issues from the past. Any ideas what might be causing this? ¿Alguna forma que pueda ayudar?

[nonkreon]

I have been experiencing this since forever, let me try to identify the cause using the symptoms; one question to everyone experiencing this: do you have things that focus on putting in multiple orders like super opportunities or safeties enabled or disabled? if you're using ak-47 safety, how many bullets did you allow?

[Corfucinas]

I have been experiencing this since forever, let me try to identify the cause using the symptoms; one question to everyone experiencing this: do you have things that focus on putting in multiple orders like super opportunities or safeties enabled or disabled? if you're using ak-47 safety, how many bullets did you allow?

@nonkreon

Nope. Never used that setting. Using ping pong with aggressive. No SOP either. Also, no EMAs, just manual.

[ctubio]

lets fix this at https://github.com/ctubio/Krypto-trading-bot/issues/1031 (i do belive is the same issue)
also will add rejected messages to the output so the reason is clearly visible

[Corfucinas]

@ctubio Gracias