Gekko: [FEATURE] Visualize Indicators

[ansonphong]

Note: this is the technical bug tracker, please use other platforms for getting support and starting a (non technical) discussion. See the getting help page for details.

I'm submitting a ...
[X] feature proposal

Other information

There has been a mounting desire from Gekko users, myself included, to be able to visualize the result of indicators while backtesting strategies, as well as visualize other data on the primary graph as well as additional sub-graphs.

@pixelGithub has started a rough ptototype based on the initial push where @askmike exposed the indicator results to the UI: https://github.com/askmike/gekko/commit/f08e358852e074fdcf038f13b67d846965d27f83

>> Feature Proposal: Visualize Indicators

^^ You can click that link to see I've written a very rough top level overview of how I propose the visualizations could be implemented. I'd like to flesh this out with the insight of the community before doing an implementation, and in this way perhaps we can make it a community effort.

I can implement the classes and data model, the biggest help would be somebody who is familiar / comfortable with D3 to make quick work of it on the frontend. Also another pro JS developer with an appreciation for object oriented design patterns would be very helpful!

Here is an example of the current rough prototype implemented by PixelCrunch / @pixelGithub

@askmike Let me know if there's a better place where I should be posting this.

[askmike]

Let's keep this open, I'll slowly close other threads about this.

I am currently working full time on a new version of Gekko, development is early stage but the majority of my time now is working together with a designer to create a new interface. This interface includes a lot of new things, including indicator results.

As soon as I have some designs to show I will post them (probably on the forum).

The first step is preparing the codebase to expose these indicator results, see #1850 for a big refactor required to make this happen.

[ansonphong]

@askmike Wow that's really great news, thanks for making me aware of that PR https://github.com/askmike/gekko/pull/1850. Shall we pause on this for the meantime until there is a preview release available for the new Gekko?

[jamespfarrell]

@askmike I currently have a lot of spare time to invest in Gekko development, please let me know if I can help you in any way

[ansonphong]

@jamespfarrell Hey James, yes it would be great to discuss. I'm still waiting to hear back from @askmike for an indication of how to move on this, since I won't want to duplicate efforts that he's already making.

Are you on the Discord? https://discord.gg/26wMygt
You can find me on there as "phong", would be a good place to discuss development.

[askmike]

Apologies I am not able to move as fast and that I am unable to be super transparent about everything happening. But I have been very busy on my side, and I'm working on a complete new UI with a designer (who has been working full time on this). The reason I am so silent is because this new UI will be part of the open source project Gekko as well as my "gekko plus" platform (hosted cloud solution for people who don't want to manage gekko and servers themselves).

The new UI is different in a lot of ways:

• written in react
• completely new design
• will render all indicators results on default to the output screen (some indicators inside the price chart, others below in a separate mini chart)

This is a very early design, it might change:

---

As for your proposal: why do you want the strategies to be in control of what the charts look like?

there is default support for plain result.result values in indicators where it has not been implemented,

I agree here, but I think different designs should be part of different indicators. Not on the strategy level. What do you think?

[ansonphong]

@askmike Glad you're switching to React, that will be a much needed update to the framework! So far what I'm seeing here looks interesting, and knowing that you're in the works on this I'll pause efforts on this until this update is released.

In general, the more robust a framework is, the more people will do things with the framework that you couldn't have foreseen.

Definitely, every indicator will have different requirements and the ability to show multiple data-sets on a single sub-graph, such as what is required on a MACD for instance, with two lines and a histogram, etc.

That said, it's also important to be able to push data to any graph from the strategy. Why? There is data generated at the level of the strategy which I would want to visualize. For instance I have strategies that implement many different traditional and custom indicators, I want to be able to add data onto the graph about where they switch over, how they are "weighted" etc, including data derived from machine learning algos which is generated in the strategy. Color properties on the graphs also help play into revealing hidden patterns in the market.

It's a highly competitive space and it will only become increasingly competitive in terms of strategies and the sophistication required to make winning trades. The more tools you give strategy developers to maintain a strategic advantage, the more of an advantage Gekko will have as a platform. Being able to visualize higher dimensional data points generated by the strategy, and customize and push data to the existing indicators seems to me like a critical use case. By exposing every aspects of the graph to the strategy class would allow developers and data analysts to do things with it that we can't now imagine.

In my proposal for a data model & API for the graphs, it would be a simple matter of exposing the same methods to the strategy as to the indicators, and would not require any additional implimentation.

[stam]

@askmike Are you already working on the new UI? I have 3 years of react experience, maybe I can help.

[askmike]

Hey @stam! Yes I have started to work on it, I will push code as soon as I am ready for it (not yet).

I am now working hard on a lot of internal upgrades + order execution, I am very close to releasing that under the current (updated) vue UI. See #2216 for details. As soon as that is realeased I am going to shift all my attention to the new react UI (I've started it but not really anything yet) As for help: I am very open to this as soon as I pushed the skeleton of the react project to github. I'm quite picky about setup of the project (how to glue redux to react and such), as such I need a bit more time so I can be sure to stay in control. I am the one who needs to maintain everything in the end ;)

[stam]

@askmike Ah sure. As for project architecture, I would highly recommend mobx instead of redux for state management. Redux is quite a bit of boilerplate which forces you to use their way of working, while mobx only implements a very small observable which you can use however you want.

[askmike]

I don't like mobx, reactive objects is one of the reasons I am moving away
from vuejs. I am familiar with the boilerplate that comes with redux - but
I am not opposed to it.

After the (limited) discussion regarding frontend on this issue tracker it
became clear straight away that everyone has different ideas on how to
architect a nice frontend for Gekko. As for the main UI: I want to push for
something that I personally can and want to maintain. That said: I love
contributions and love it when people implement different ideas, as suchh I
think it's a better idea to come out with very good API docs (rest + ws) so
that others can build their own frontend for Gekko. What do you think?

Regards,
Mike

On Fri, Jun 29, 2018 at 7:57 PM, Jasper Stam [e-mail]
wrote:

@askmike https://github.com/askmike Ah sure. As for project
architecture, I would highly recommend mobx instead of redux for state
management. Redux is quite a bit of boilerplate which forces you to use
their way of working, while mobx only implements a very small observable
which you can use however you want.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/askmike/gekko/issues/2029#issuecomment-401346424, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA7MD2-wi9pi6r3R96nAb1K5zvvzse8Pks5uBiRPgaJpZM4SqI1D
.

--
PGP key at keybase.io/mikevanrossum
https://keybase.io/mikevanrossum/key.asc

[stam]

With redux I noticed on bigger projects that I lacked proper inheritance when implementing model logic with actions and reducers. I took a short look at VueX and it looks more like a reactive state tree than the observable MobX provides. But I agree that functional programming also has its own very good merits.

Regarding the bring-your-own-frontend, I think that only a very tiny % of the gekko-userbase has the incentive to personally develop on the frontend. When the default frontend is a seperate forkable repository with highly reusable components it already allows for a great deal of personalization.

Good API docs are always a great idea though!

[H256]

@askmike

I love contributions and love it when people implement different ideas, as suchh I think it's a better idea to come out with very good API docs (rest + ws) so that others can build their own frontend for Gekko. What do you think?

Yes, that's exactly what I was thinking. 👍
While building the quasar port for the ui, it was really a pain sometimes since it was hard to guess what data is passed on what channel between server and client etc. Also including indicator data was a real scientific project to find out. :D

So the idea to provide a very basic interface combined with fantastic, up to date documentation imho is the best way to go ;)

[mark-sch]

@H256 Do you still extend your quasar UI fork? It looks great and extends nice features. I have also some things in mind, but it is hard to decide which way to go. Waiting on v0.6 or old v0.5 UI or Qasar UI.

What I dislike on v0.5 UI is, that it aims to run on its own (process). In practice, my main Gekko is a bunch of several CLI Geckos running in background. So what I am missing right now is a UI which visualizes these several CLI instances (e.g. with some redis stuff). I want to check buy/sell advices from realtime runners. How do you use your Quasar GUI in practice?

[H256]

@mark-sch It frozen at the moment. I just want to wait for the 0.6 release to directly develop an UI around the new events.
In general the GUI listens for websocket-events by gekko.

[askmike]

@H256 awesome, I also can't wait! I am very close to releasing 0.6 (see

2216 for details), which means the event API is very close to be stable

(for 0.6 anyway). Before I release it I _will_ have API docs and will keep
a changelog to make it easier for you to develop against the gekkos web API.

On Sun, Jul 1, 2018 at 3:50 PM, Klemens Wittig [e-mail]
wrote:

@mark-sch https://github.com/mark-sch It frozen at the moment. I just
want to wait for the 0.6 release to directly develop an UI around the new
events.
In general the GUI listens for websocket-events by gekko.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/askmike/gekko/issues/2029#issuecomment-401593354, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA7MDyfQMD-AMkpl2bW0HhDoP_RdMDfWks5uCI1DgaJpZM4SqI1D
.

--
PGP key at keybase.io/mikevanrossum
https://keybase.io/mikevanrossum/key.asc

[askmike]

@H256 gekko server docs are here:
https://github.com/askmike/gekko/blob/pre-v0.6/docs/internals/server_api.md. They won't change
much anymore for 0.6 (except for maybe the gekko event tradeCompleted).

[H256]

@askmike Nice! Does the _/api/backtest_ - REST and _"gekko_event"_ - websocket event emit indicator results yet? (something like in this commit on my fork, _core/cp.js_ and _core/workers/pipeline/messageHandlers/backtestHandler.js_ https://github.com/H256/gekko/commit/55f983da0152b738331c45ab5b8fb7363c3e3f02)

[askmike]

Yes they both do, but note:

for performance reasons the backtest did not (and still does not) emit any WS events. However you can get all of them on finish, see:

• for live gekkos you'll get the stratUpdate event over the websocket. <-- I see GH doesn't render the docs properly. Sorry about that.
• for backtest rest call, set backtestResultExporter.data.stratUpdates to true and you'll get the same events as part of the REST call output.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. If you feel this is very a important issue please reach out the maintainer of this project directly via e-mail: gekko at mvr dot me.

[MoliseMG]

It has been year now. We have been waiting for the new react UI.

[kand617]

@askmike any update on this? Or is your new version not going to be open source?