Gekko: Trader fail on coinbase pro (gdax) and submitting wrong date ( 1970)

[meauxt]

Note: this is the technical bug tracker, please use other platforms for getting support and starting a (non technical) discussion. See the getting help page for details.

I'm submitting a ...
[ ] bug report
[ ] question about the decisions made in the repository

Action taken (what you did)
robot try to trade

Expected result (what you hoped would happen)
retry since the condition still met, or don't report profit.and output the reason of failing

Actual result (unexpected outcome)
report profit and did not made the trade

Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, etc)
2018-12-25 00:33:36 (INFO): Trader Received advice to go long. Buying BTC
2018-12-25 00:33:36 (DEBUG): Creating order to buy 0.10345737621582726 BTC
2018-12-25 00:33:41 (DEBUG): [ORDER] statusChange: SUBMITTED
2018-12-25 00:33:41 (DEBUG): [ORDER] statusChange: OPEN
2018-12-25 00:33:42 (DEBUG): [ORDER] statusChange: REJECTED
2018-12-25 00:33:42 (INFO): [ORDER] summary: { price: 0,
amount: 0,
date: moment("1970-01-01T00:00:00.000"),
side: 'buy',
orders: 1 }
2018-12-25 00:33:42 (DEBUG): syncing private data
2018-12-25 00:33:47 (WARN): WARNING: exchange did not provide fee information, assuming no fees..
2018-12-25 00:34:36 (INFO): (PROFIT REPORT) original balance: 421.30683345 USD
2018-12-25 00:34:36 (INFO): (PROFIT REPORT) current balance: 418.69639602 USD
2018-12-25 00:34:36 (INFO): (PROFIT REPORT) profit: -2.61043743 USD (-0.61960482%)

[meauxt]

i just looked into the json respond under evens.completed_trades:
first 2 two happened successfully, but start having issue after that... it's weird why it was working well and then stop after two trades. also
tradeCompleted: [{id: "trade-1", adviceId: "advice-1", action: "buy", cost: 0, amount: 0.09555038, price: 4185.72,…},…]
0: {id: "trade-1", adviceId: "advice-1", action: "buy", cost: 0, amount: 0.09555038, price: 4185.72,…}
action: "buy"
adviceId: "advice-1"
amount: 0.09555038
balance: 421.6011310074112
cost: 0
date: "2018-12-24T01:34:00.106Z"
effectivePrice: 4185.72
feePercent: 0
id: "trade-1"
portfolio: {currency: 21.3596574535042, asset: 0.0955503899813567}
asset: 0.0955503899813567
currency: 21.3596574535042
price: 4185.72
1: {id: "trade-2", adviceId: "advice-2", action: "sell", cost: 0, amount: 0.09555038, price: 4158.4,…}
action: "sell"
adviceId: "advice-2"
amount: 0.09555038
balance: 418.69639914189673
cost: 0
date: "2018-12-24T02:48:43.917Z"
effectivePrice: 4158.4
feePercent: 0
id: "trade-2"
portfolio: {currency: 418.6963576455042, asset: 9.9813567e-9}
price: 4158.4
2: {id: "trade-3", adviceId: "advice-3", action: "buy", amount: 0, price: 0,…}
action: "buy"
adviceId: "advice-3"
amount: 0
balance: 418.69639933363857
date: "1970-01-01T00:00:00.000Z"
effectivePrice: 0
id: "trade-3"
portfolio: {currency: 418.6963576455042, asset: 9.9813567e-9}
price: 0
3: {id: "trade-5", adviceId: "advice-5", action: "buy", amount: 0, price: 0,…}
action: "buy"
adviceId: "advice-5"
amount: 0
balance: 418.69639602072647
date: "1970-01-01T00:00:00.000Z"
effectivePrice: 0
id: "trade-5"
portfolio: {currency: 418.6963576455042, asset: 9.9813567e-9}
price: 0

when i looked into the advice object all trades have the right date...

[vqalex]

I have exact the same problem

[alex-ef]

Hello,
The same problem can be observed on my setup also. In addition the trader refuses to place further trades of the same type - if this is a sell and it fails, when we try to make new sells they are not executed.
This is a bit critical because, while wrong date is inconvenient, the real issue is that Gekko is not retrying the trade after this rejected order. It also keeps the wrong trade state and trader "assumes" that the order is completed, while it is not - this is causing the next orders to fail.

[alex-ef]

2067 - This and the linked to it issues are about the similar behavior but unfortunately all of them are closed due to inactivity and not really fixed.

[crypto49er]

Is it possible Coinbase Pro is returning with the wrong date? I seen the wrong date multiple times but it hasn't caused the trade to fail.

[crypto49er]

Ok. I just had a failed trade due to this date issue. I still think the date is coming from Coinbase Pro, but nevertheless, Gekko should continue to put the trade through instead of stopping. I will see what I can find. Here's what it looks like in terminal even though this sell trade never happened.

2019-01-08 15:56:13 (INFO): (ROUNDTRIP) entry date (UTC) exit date (UTC) exposed duration P&L profit
2019-01-08 15:56:13 (INFO): (ROUNDTRIP) 2019-01-08 14:16 1970-01-01 00:00 49 years -98.47 -87.39
2019-01-08 15:56:13 (INFO): (PROFIT REPORT) original balance: 114.59492718 USD
(PROFIT REPORT) current balance: 111.69692572 USD
(PROFIT REPORT) profit: -2.89800146 USD (-2.52890903%)
2019-01-08 15:56:13 (INFO): report { startTime: '2019-01-05 23:21:00',
endTime: '2019-01-08 15:56:00',
timespan: '3 days',
market: -2.8968638861946516,
balance: 111.6969257216308,
profit: -2.8980014618999803,
relativeProfit: -2.5289090303785002,
yearlyProfit: -393.3423356090645,
relativeYearlyProfit: -343.24585326460584,
startPrice: 154.65,
endPrice: 150.17,
trades: 13,
startBalance: 114.59492718353079,
exposure: -6653.483909677419,
sharpe: -9.880526708233624,
downside: -24.804105303045706,
alpha: -0.0011375757053286861 }

[vqalex]

Yea, happed to me today..order was partially filled, then Reject

[vqalex]

Looks like the problem is not with actual date. If not mistaken, it could be just delay between order submitted/retry and price move.

Post-only Mode means that Traders can only place an Order if it would be posted to the Order Book as a Maker Order. An Order which would be posted as a Taker Order will be rejected. No Market Orders may be placed and no Orders will be filled. Resting orders may be canceled in post-only mode.

I think gekko is not recognazing REJECT corectly

[vqalex]

I think we really need hotfix for it

[vqalex]

This is exactly what happens(from stackoverflow.com)Post Only" buy orders must have a price that is less than the current market price that people are willing to sell at.

1.8A Limit Order with ‘post-only’ selected will only be posted to the Order Book if it would not be posted at the same price as an existing Order on the Order Book.

Put differently, if you want the buy order to go on the order book, and not fill immediately, you must bid less than people are currently willing to sell for.

When you try to post an order at , there's a decent chance that price was at the lowest ask price, i.e., the price that people are currently willing to sell for.

And then certainly when you increase your bid, you're easily going into territory that people are already willing to sell for.

To solve this, simply bid less, not more.

The 1.8A section is saying exactly this, though it is ambiguous the way you're looking at it. They're saying you cannot post a buy order to the order book if someone is already on the order book saying they're willing to sell at that price. You certainly can post a buy order at the same buy price as other people are already willing to buy at.

[vqalex]

I think we just need to add logic, when returned reject to check price and bid again.
I wander what will happen if I would change (gdax.js)
line 220
From

} else if (status === 'rejected') {
return callback(undefined, { executed: false, open: false });

TO

} else if (status === 'rejected') {
return callback(undefined, { executed: false, open: TRUE });

any ideas?

[crypto49er]

I wrote a strat that buys immediately and sells after 5 minutes to attempt to get a rejected status.

I was able to replicate this issue after live trading for about an hour. The wrong date is connected to the trade being rejected.

@vqalex I am trying to find out what info is accessible within gdax.js. If I can get the price, which is also 0 when the the trade is incorrectly rejected, I can add logic like this:

} else if (status === 'rejected') {
    if ( price == 0 ) { // price is not actual variable, need to find the variable that actually holds the price
        return callback(undefined, { executed: false, open: TRUE });
    }
return callback(undefined, { executed: false, open: false });
}

If data.filled_size is returning the correct value (not sure if it will, since date, price, amount is already wrong), I would use that so the trade continues like an open order.

[vqalex]

per research it should return

reject_reason: ‘post only’,

If it is a case then we can use it instead of price, it really depends on what is exposed to gdax.js
https://github.com/coinbase/gdax-tt/issues/149

[vqalex]

I'm a beginner in javascript but what about this @crypto49er ?

var status = data.status;
var reject_reason = data.reject_reason;
if(status == 'pending') {
  // technically not open yet, but will be soon
  return callback(undefined, { executed: false, open: true, filledAmount: 0 });
} if (status === 'done' || status === 'settled') {
  return callback(undefined, { executed: true, open: false });
} else if (status === 'rejected' && reject_reason !== 'post only') {
  return callback(undefined, { executed: false, open: false });
} else if(status === 'open' || status === 'active' || (status === 'rejected' && reject_reason === 'post only')) {
  return callback(undefined, { executed: false, open: true, filledAmount: parseFloat(data.filled_size) });
}

callback(new Error('Unknown status ' + status));

};

I will try this:

if(status == 'pending') {
  // technically not open yet, but will be soon
  return callback(undefined, { executed: false, open: true, filledAmount: 0 });
} if (status === 'done' || status === 'settled') {
  return callback(undefined, { executed: true, open: false });
} else if (status === 'rejected' && reject_reason !== 'post only') {
  return callback(undefined, { executed: false, open: false });
} else if(status === 'open' || status === 'active' || (status === 'rejected' && reject_reason === 'post only')) {
  return callback(undefined, { executed: false, open: true, filledAmount: parseFloat(data.filled_size) });
}xu

[crypto49er]

@vqalex I'm not great at javascript either but it looks like what you are doing will cause it to kick out to line 229 "Unknown Status", as the reject_reason is probably equal to 'post only'. But you should certainly give it a shot to see what happens.

I would have the whole data variable output to console just to see what's actually inside it. Ran it live for 5 hours today and I wasn't able to get the rejected message. I will probably have to go into Gekko Broker and modify it so it always sends sell below bid price or buy above ask price just to force this error message.

[crypto49er]

@vqalex I was able to force Gekko to create rejected trades by going into sticky.js and changing line 111 to

return r(ticker.bid + 0.03);

The amount varies based on the crypto and the spread. On Coinbase Pro, 0.03 works for most fiat/crypto trade pairs.

So I tried my version (which is really your original where you just change "open" from false to true. That just force the callback to move the order but doesn't actually send it back to the exchange. It will say it filled the order but it didn't.

Your latest, which I also tried, forced it to drop to "unknown status rejected" so it still doesn't retry the trade. Will continue testing.

[crypto49er]

From what @askmike said here (https://github.com/askmike/gekko/issues/2552#issuecomment-423484503), we need to create a new error type in gdax.js and have that error handled in Gekko Broker (sticky.js) so the order is retried. I can kind of get the error passed over to sticky.js but haven't been able to actually get Gekko to retry the rejected trade.

[vqalex]

@crypto49er please post full reject response from coinbase if you have.

[crypto49er]

@vqalex

Object {id: "be146864-871c-48fa-af27-cd7082772313", price: "4.48000000", size: "2.15523191", product_id: "ETC-USD", side: "buy", …}
created_at: "2019-01-10T16:28:28.477462Z"
executed_value: "0.0000000000000000"
fill_fees: "0.0000000000000000"
filled_size: "0.00000000"
id: "be146864-871c-48fa-af27-cd7082772313"
post_only: true
price: "4.48000000"
product_id: "ETC-USD"
reject_reason: "post only"
settled: true
side: "buy"
size: "2.15523191"
status: "rejected"
time_in_force: "GTC"
type: "limit"
__proto__: Object {constructor: , __defineGetter__: , __defineSetter__: , …}

[crypto49er]

One interesting thing of note is retrying a rejected trade probably won't work. I noticed that insufficient fund trades, where Gekko incorrectly calculates the available balance divided by trade price are retried, but all 10 (that's the number set for insufficient fund trades) of those attempts fail. My guess is the order is not found on the exchange so retrying a non-existent order will always fail.

Since rejected orders are also not stored after they are rejected, there's no point to retry them.

[crypto49er]

That's not to say this issue shouldn't be fixed, but instead of retrying, a new trade needs to be re-submitted.

[crypto49er]

I came up with a messy fix for this. It's so messy that I wrote it up on Gekko forum instead of here.

https://forum.gekko.wizb.it/thread-57832.html

I can't wait for the official fix for this issue.

[scintillae]

@crypto49er

Jack, thanks for posting what you have so far. I have been dealing with the same issue. I am a relative beginner to Gekko, and only code as a hobbyist and very basic stuff for my work as an engineer, so please bear with me.

In your forum post you note:

In the config file, we are going to store the state of "rejected" so the other files can read/write to them. I just added the following before the last line. You can probably add them anywhere.

Code:
config.IssueState = {
rejected: false,
side: 'sell',
}

Will this solution only work using the command line (using '--config config.js' for instance), or can it be used in the UI? If so, where would the code above be placed? In the "web/vue/dist/UIconfig" file, if following the online gekko server installation instructions?

I also watched your video from last year about the limit vs. market
order issue; how do these issues interact with each other?

Again, I'm a bit of a newbie to Gekko, but I have some hobby background in Javascript; I think my misunderstanding is in the specifics of Gekko's internals; I'm hoping for some advice that will help me bootstrap my own solution...

@askmike I too am looking forward to an official fix for this and would like to help in any way I can, my coding skills aren't much but I could do testing or code simple things...

[crypto49er]

@scintillae

I don't think it can be used for UI. You can certainly try it if you want. But you need to modify the code a bit, to this:

IssueState: { rejected: false, side: 'sell', }

You might be better off just changing post_only to false in gdax.js so you don't get these rejected trades.

[crypto49er]

@scintillae @vqalex @alex-ef @meauxt

I just wrote up a much cleaner fix that should work in UI as well as in command line. You can check it out in this pull request #2721 . Hopefully this fixes the dreaded 1969/1970 date issue once and for all!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. If you feel this is very a important issue please reach out the maintainer of this project directly via e-mail: gekko at mvr dot me.