Zenbot: Implement genetic algo for fine tuning strategy parameters
Take a look genetic-js
It is also present on npm.
Let's discuss here
gelotus
All 33 comments
hurr hurr hurr, i'm waaaay ahead of you. Unfortunately it's in python but i'm getting some good results #299
I have very little documentation currently, unfortunately.
Inside zen/ there is a python file main that will launch a genetic algorithm to train the parameters of a given strategy.
To launch:
docker-compose up
docker-compose exec server bash
fab backfill_local:<days>
cd zen
python -m scoop main.py <product> <days> <individuals> <strategy>
product can be either ETH-BTC or BTC-CUR where CUR stands for "currency"
I also wrote a thesis on parameter tuning with genetic algorithms, if you're interested:
Automatic Audio Encoder Tuning Using Evolutionary Algorithms.pdf
arpheno
on 16 Jun 2017
@arpheno How are you using your python strat with a JS based system? Does your python algo run theny ou take the outputted data over to zenbot?
ch0wdan
on 16 Jun 2017
Yes.
On Fri, Jun 16, 2017 at 6:57 PM Jordan notifications@github.com wrote:
@arpheno https://github.com/arpheno How are you using your python strat
with a JS based system? Does your python algo run theny ou take the
outputted data over to zenbot?—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/carlos8f/zenbot/issues/298#issuecomment-309078942,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABBzGh2FMd3a-oRbJMK2MYmQ0I2UUyeaks5sErQHgaJpZM4N8Xyg
.
arpheno
on 16 Jun 2017
@arpheno Great work, I'll take a look. Have you seen the algo that I have linked?
gelotus
on 17 Jun 2017
I've setting up your work. I'm still waiting for backfilling..
Some suggestion, can you make it agnostic in environment and selector? With a conf file? I've see that there is much stuff hardcoded, like specific commands for docker. And in zen/tests/test_integration.py this hardcoded selector main('gdax.BTC-ETH',120)
gelotus
on 17 Jun 2017
Finally i've succeeded in run your genetics.. After a fight with the new f-strings feature in python 3.6 and after.. I'm go to run some simulations and we will see
gelotus
on 18 Jun 2017
I will be glad to hear how it went. Currently 4 people are running it
including you and me.
On Sun, Jun 18, 2017 at 11:49 PM gelotus notifications@github.com wrote:
Finally i've succeeded in run your genetics.. After a fight with the new
f-strings feature in python 3.6 and after.. I'm go to run some simulations
and we will see—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/carlos8f/zenbot/issues/298#issuecomment-309305251,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABBzGhTrr80VtSOnyldDRc19d8gmKWcxks5sFZtxgaJpZM4N8Xyg
.
arpheno
on 19 Jun 2017
@arpheno sorry, i'm not expert in python, but seems to be a bug in evaluate_zen() that not pass the right values. If I print fitness var I obtain always [-100, -100]. Please take a look
gelotus
on 19 Jun 2017
Ok, with your fix all seems to work right.
Let's show some results:
Generation 181 {'avg': 1.1251920040357279, 'std': 6.1372615421965255, 'min': -15.320000000000269, 'max': 5.287425556773063, 'len': 20}
Current Hall of Fame:
[30m../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Randal Pitts[0m
[30m../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Stephanie Houchins[0m
[30m../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Patrick Smith[0m
[30m../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Thomas Tomlin[0m
[30m../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 James Alvarez[0m
[30m../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Cody Harvey[0m
[30m../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Jason Dixon[0m
[30m../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Lynn Hetrick[0m
[30m../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Tamika Vega[0m
[30m../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Jill Ogrady[0m
[30m../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Patrica Espinoza[0m
[30m../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Harold Richardson[0m
[30m../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Shirley Harten[0m
[30m../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.5633 --period 95m [5.62, 6.55] 5.287425556773063 Dustin Wright[0m
[30m../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 James Mckinney[0m
4 Elites will survive, they're currently the strongest:
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Randal Pitts
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Lynn Hetrick
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Tamika Vega
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Stephanie Houchins
8 Specialists will survive, they're the best in their domain:
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 10 --sar_af 0.0064 --sar_max_af 0.0778 --period 131m [-7.91, 13.4] -7.910000000556141 Celina Belzer
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0072 --sar_max_af 0.2336 --period 94m [6.5, 0.06] 0.058404866221893226 Scott Stancil
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0072 --sar_max_af 0.2336 --period 94m [6.5, -0.21] -0.21121792215630394 Felix Roy
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0072 --sar_max_af 0.2336 --period 94m [6.5, 0.06] 0.058404866221893226 John Cook
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0072 --sar_max_af 0.2336 --period 94m [6.5, -0.56] -0.5608584095551045 Harriet Martin
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 4 --sar_af 0.0078 --sar_max_af 0.2336 --period 78m [-8.53, 13.19] -8.530000000369082 Anita Farnsworth
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 4 --sar_af 0.0091 --sar_max_af 0.2336 --period 78m [-15.32, 13.63] -15.320000000000269 Jo Cleveland
../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 4 --sar_af 0.0078 --sar_max_af 0.2336 --period 78m [-8.53, 13.19] -8.530000000369082 Juanita Hurst
Some other have fought their way through:
../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Lynn Hetrick
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Randal Pitts
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Lynn Hetrick
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Lynn Hetrick
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m [5.62, 6.55] 5.287425556773063 Lynn Hetrick
...
With long simulations the simulations directory grows fast in size, needs a fix. In the hall of fame all the population have the same parameters.
gelotus
on 22 Jun 2017
can you pull the latest version? I think both issues should be addressed there.
I made it so one individual can only ever live once ( with the same parameters).
arpheno
on 22 Jun 2017
Well done, whit your last commit all work seamlessly.
But.. (philosophical thinking) it works on static data. What happens if the strong elites dies after n generation and the datas are refreshed with an auto backfill over the same n generations?
I think that if one keep the sim running in the same time that the trading bot working we will have a fresh and plus or minus the right parameters for actual period. There be also possible to create an automated bot refresh procedure that pass the new parameters automatically.
What do you think?
gelotus
on 25 Jun 2017
hey swozny what's happen in your repo?
Anyway I've tried to merge some of my ideas in your work. Environment configuration, selectors configuration and autobackfilling. The draw function is broken but i have create a little 'if' that point in conf for turning on or off. I've also setting minimum limit in parameters.
Now i'm working in linux env, need some good guy that try this in docker, because i'm the king of the ignorants on this env.
Will be a good thing to append a "date-time from to" to the hall of fame for improve analisys of results with autobackfill turned on.
If someone want to try this is my repo:
https://github.com/gelotus/zenbot
Take a look in conf.py,
command syntax varied a little bit:
for manual backfilling <zenbot_dir>$ fab autobackfill_local:"<selector>,<days>"
for starting sim
<zenbot_dir>/zen $ python3.6 -m scoop main.py <selector> <days> [<population>][<strategy>] [backfill]
Selectors are in conf, days are the days of trade, polulation in the number of live individuals after a generation change, min 10 default 10, strategy is the same of zenbot, default trend_ema, if backfill is present, it will turn on autobackfill timer during sim for updating the trade history.
Note that in my repo the docker files are absent. Pick those in carlos repo
gelotus
on 29 Jun 2017
I tried to merge your PR and it deleted the dockerfiles. I got a lot of
complaints from other people when I merged it, so I had to revert :(
On Thu, Jun 29, 2017 at 12:11 AM gelotus notifications@github.com wrote:
hey swozny what's happen in your repo?
Anyway I've tried to merge some of my ideas in your work. Environment
configuration, selectors configuration and autobackfilling. The draw
function is broken but i have create a little 'if' that point in conf for
turning on or off. I've also setting minimum limit in parameters.
Now i'm working in linux env, need some good guy that try this in docker,
because i'm the king of the ignorants on this env.
Will be a good thing to append a "date-time from to" to the hall of fame
for improve analisys of results with autobackfill turned on.If someone want to try this is my repo:
https://github.com/gelotus/zenbotTake a look in conf.py,
command syntax varied a little bit:
for manual backfilling$ fab
autobackfill_local:", "
for starting sim
/zen $ python3.6 -m scoop main.py
[][ ] [backfill] Selectors are in conf, days are the days of trade, polulation in the
number of live individuals after a generation change, min 10 default 10,
strategy is the same of zenbot, default trend_ema, if backfill is present,
it will turn on autobackfill timer during sim for updating the trade
history.Note that in my repo the docker files are absent. Pick those in carlos repo
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/carlos8f/zenbot/issues/298#issuecomment-311806903,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABBzGv84k3scHIEKmRZwiSKM6N-bHdPyks5sIs-JgaJpZM4N8Xyg
.
arpheno
on 29 Jun 2017
ok, let's try to implement a windows+docker env and test with it..
gelotus
on 29 Jun 2017
I think I'll get to work on this on the weekend :)
I'm absolutely amazed that someone is building on my work, not gonna let it
go to waste.
On Thu, Jun 29, 2017 at 9:13 PM gelotus notifications@github.com wrote:
ok, let's try to implement a windows+docker env and test with it..
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/carlos8f/zenbot/issues/298#issuecomment-312065191,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABBzGo4P6BMB1gmgH-DupmY24cGNmCorks5sI_GegaJpZM4N8Xyg
.
arpheno
on 29 Jun 2017
Just droppin my 2 cents: would be cool if trading would be performed by agent with best score, while in the background there is still evolution going on data of past day or so (quickly iterate) and if found better agent, use that. (While keeping backups in case of unpredicted disasters)
Maximus-CZ
on 29 Jun 2017
I think you're very courageous if you're not double checking the strategy
the optimizer comes up with before throwing it at real money.
On Thu, Jun 29, 2017 at 11:02 PM Maximus-CZ notifications@github.com
wrote:
Just droppin my 2 cents: would be cool if trading would be performed by
agent with best score, while in the background there is still evolution
going on data of past day or so (quickly iterate) and if found better
agent, use that. (While keeping backups in case of unpredicted disasters)—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/carlos8f/zenbot/issues/298#issuecomment-312103202,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABBzGqFtuWJh5i04MVIt0Wl29exXNHcUks5sJA2PgaJpZM4N8Xyg
.
arpheno
on 30 Jun 2017
Well before agent would trade with real money, it would first need to run trough simulations to verify its profitability, If I had 2 agents with simulated +10% and +30% ROI respectively, I wouldnt really see a problem switching to better agent at runtime.
On top of this, having Zenbot run several parallel agents in papertrading mode can also provide indication of which agent would be best choice for given market/day.
Maximus-CZ
on 30 Jun 2017
A little question.. Who uses docker? And on windows or on linux? I've see that the last docker for windows works only on windows 10, both stable and edge versions. For windows 7-8-8.1 there is docker-toolbox. That's very fragmented, on linux i don't see the convenience of using docker.. But i don't know, need an advice
gelotus
on 30 Jun 2017
@arpheno are you vanished? :) Take a look in my repo, docker was reinserted.
gelotus
on 26 Jul 2017
Hey Guys, Just want to say thanks for all your efforts to implement genetic algorithms into zenbot. It has been the first introduction to the topic for me and i think im hooked :+1:
For the last few weeks I have been trying to add covariance matrix adaptation evolution strategy, which i believe will automatically adjust cx and mut probabilities to help with convergence and to prevent getting stuck on local maximums allowing us to develop and optimise more complex strategies.
I think this should be possible and relatively easy using the tools from the python deap package a la:
http://deap.readthedocs.io/en/master/examples/cmaes.html
https://github.com/DEAP/deap/blob/a5d77d1e30f7674948bb6a2c4381fa4e247ac89d/examples/es/fctmin.py
Unfortunately this is my first introduction to both python and evolution strategies, and to be honest, i'm a bit out of my depth. @gelotus, @arpheno I was wondering if you guys, (or anyone else) would be able to have a look (im sad to see development seems to have stopped on this fork). I think this would be very beneficial to zenbot development and for me personally very rewarding (emotionally... maybe financially too !) =)
So far I have basically tried to frankenstein the example into the code by swonzy but it is not working with the compiling error:
TypeError: sequence item 0: expected str instance, Individual found
I think I need help to correctly register fitness and individual functions into the deap toolbox..
````import sys
import numpy
from functools import partial
from timer import ThreadTimer
from deap import algorithms, base, benchmarks, cma, creator, tools
from deap.tools import cxTwoPoint, mutGaussian
from scoop import shared
from blessings import Terminal
from subprocess import run, DEVNULL
from conf import indpb, sigma, partitions, selectors, path, bkfint
from evaluation import evaluate_zen, Andividual
from evolution import evolve
term = Terminal()
creator.create("FitnessMax", base.Fitness, weights=(1.0,))
creator.create("Individual", list, fitness=creator.FitnessMax)
toolbox = base.Toolbox()
N=30
def main(instrument, days, popsize, zstrategy='trend_ema', nobf=None):
print(term.blue("Starting evolution...."))
evaluate = partial(evaluate_zen, days=days)
toolbox.register("evaluate", evaluate_zen, days=days)
print(term.blue("Evaluating ") + term.green(str(popsize)) + term.blue(" individuals over ")
+ term.green(str(days)) + term.blue(" days in ") + term.green(str(partitions)) + term.blue(" partitions."))
Andividual.path = path
Andividual.instruments = selectors[instrument]
Andividual.mate = cxTwoPoint
Andividual.mutate = partial(mutGaussian, mu=0, sigma=sigma, indpb=indpb)
Andividual.strategy = zstrategy
print(term.blue("Mating function is ") + term.green(str(Andividual.mate)))
print(term.blue("Mutating function is ") + term.green(str(Andividual.mutate)))
numpy.random.seed(128)
strategy = cma.Strategy(centroid=[5.0]*N, sigma=5.0, lambda_=20*N)
toolbox.register("generate", strategy.generate, creator.Individual)
toolbox.register("update", strategy.update)
hof = tools.HallOfFame(1)
stats = tools.Statistics(lambda ind: ind.fitness.values)
stats.register("avg", numpy.mean)
stats.register("std", numpy.std)
stats.register("min", numpy.min)
stats.register("max", numpy.max)
algorithms.eaGenerateUpdate(toolbox, ngen=250, stats=stats, halloffame=hof)
res = evolve(evaluate, Andividual, popsize)
return res, hof[0].fitness.values[0]```
lachlnloaded
on 7 Sep 2017
I too am interested in learning more about live ML/genetic algo integration for realtime trade-bot strategy optimization, but my knowledge here is limited at the moment. I'm more than happy to start digging into the code myself, and probably will, but am also curious if @gelotus and/or @arpheno have any news regarding their efforts 🔥
jaketblank
on 18 Sep 2017
Has anybody encountered this problem? Im using a custom strategy and the only option it has is a --period option. It works fine for regular simulations and trading. I'm getting this error:
Generation 0 {'avg': -21.975665246121547, 'std': 4.6351407503136413, 'min': -27.740000000369083, 'max': -11.259453225628276, 'len': 10}
It's breeding season, we're expecting new members of the tribe...
Traceback (most recent call last):
File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.6/site-packages/scoop/bootstrap/__main__.py", line 302, in <module>
b.main()
File "/usr/local/lib/python3.6/site-packages/scoop/bootstrap/__main__.py", line 92, in main
self.run()
File "/usr/local/lib/python3.6/site-packages/scoop/bootstrap/__main__.py", line 290, in run
futures_startup()
File "/usr/local/lib/python3.6/site-packages/scoop/bootstrap/__main__.py", line 271, in futures_startup
run_name="__main__"
File "/usr/local/lib/python3.6/site-packages/scoop/futures.py", line 64, in _startup
result = _controller.switch(rootFuture, *args, **kargs)
File "/usr/local/lib/python3.6/site-packages/scoop/_control.py", line 253, in runController
raise future.exceptionValue
File "/usr/local/lib/python3.6/site-packages/scoop/_control.py", line 127, in runFuture
future.resultValue = future.callable(*future.args, **future.kargs)
File "/usr/local/lib/python3.6/runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "/usr/local/lib/python3.6/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "main.py", line 68, in <module>
res = main(INSTRUMENT, TOTAL_DAYS, popsize, strategy)
File "main.py", line 26, in main
res = evolve(evaluate, Andividual, popsize)
File "/app/zen/evolution/__init__.py", line 18, in evolve
return algorithm(cls, popsize, futures.map, evaluate, select, breed, mutate, stats, history, hof)
File "/app/zen/evolution/core.py", line 22, in algorithm
offspring = breed(population)
File "/app/zen/evolution/core.py", line 43, in breed
child1, child2 = parent1 + parent2
File "/app/zen/evolution/individual_base.py", line 36, in __add__
child1, child2 = self.__class__.mate(*couple)
File "/usr/local/lib/python3.6/site-packages/deap/tools/crossover.py", line 44, in cxTwoPoint
cxpoint2 = random.randint(1, size - 1)
File "/usr/local/lib/python3.6/random.py", line 220, in randint
return self.randrange(a, b+1)
**File "/usr/local/lib/python3.6/random.py", line 198, in randrange
raise ValueError("empty range for randrange() (%d,%d, %d)" % (istart, istop, width))
ValueError: empty range for randrange() (1,1, 0)**
brucetus
on 23 Sep 2017
The randrange() function is being called on a range with no depth (1,1), hence the error.
Looking up the trace, we see the function is calling randrange(a, b+1). On mobile right now so debugging is limited, but I would check to see what variable "b" is supposed to represent, and figure out why it's being passed through as 0
jaketblank
on 23 Sep 2017
@jaketblank
I can't find the random.py file in my computer. I'm running on Docker. Where would I be able to read random.py? Also I've been able to use zen on other strategies like trend_ema so I don't think that zen itself is broken.
brucetus
on 25 Sep 2017
The problem isn't with the randrange() function - if it was there would be a major problem with all python code that uses it :)
(That file should be located in "/usr/local/lib/python3.6/" on your system, just for reference)
I'd start looking in "individual_base.py" and look at the "mate()" method. See, then track forwards to see where values are being generated/sent to randrange()
jaketblank
on 25 Sep 2017
@jaketblank
random.randint(1, size - 1)
self.randrange(a, b+1)
randrange(1,1, 0)
that means b = 0 and if b = size - 1 then size = 1
From crossover.py:
size = min(len(ind1), len(ind2))
It's the minimum length of the two individuals being crossed over
Wouldn't that mean that at least one of the lengths of the individuals being crossed over is 1?
I'm not sure what that means but I think the problem is that the strategy I'm using only has one option: period size. I think this makes the length of the individuals too similar, but I'm not sure. I think a strategy with one option would be better optimized using the backtester script. Thanks though.
brucetus
on 26 Sep 2017
I get a strange error:
python3.6 -m scoop main.py BTC-CUR 10 10 macd
When genetic_algo starts it prints:
Starting evolution....
Evaluating 10 individuals over 10 days in 2 partitions.
Mating function is
Mutating function is functools.partial(
Sampling an initial valid population, this may take a while...
Currently 0 valid individuals
Then after a while:
xxxxxxxxxxCurrently 1 valid individuals
The algo never stops, repeatedly printing:
xxxxxxxxxxCurrently 1 valid individuals
What could be wrong? The DB was correctly backfilled.
albabosh
on 27 Dec 2017
Should I worry about this thing:
/usr/local/lib/python3.6/dist-packages/deap/tools/_hypervolume/pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
"module. Expect this to be very slow.", ImportWarning)
albabosh
on 27 Dec 2017
@albabosh I'm having the same output of you
xxxxxxxxxxCurrently 1 valid individuals for each cycle.
enricocas
on 28 Dec 2017
Have a look at pull request #1078. Now this issue doesn't come up any more for me (because a set can hold more than 1 individual now, after deleting the overwritten version of __eq__ .
Heerpa
on 7 Jan 2018
Can this be closed?
LoneWolf345
on 6 Mar 2018
For me it's ok
Il 06 mar 2018 10:13 PM, "LoneWolf345" notifications@github.com ha
scritto:
Can this be closed?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/DeviaVir/zenbot/issues/298#issuecomment-370930228,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA9DL5I96Va_CsxDMxZyyaYjLvE8UZsrks5tbvwMgaJpZM4N8Xyg
.
enricocas
on 6 Mar 2018
Related issues
MCrypto
·
5Comments
bbusche
·
5Comments
tabbek
·
3Comments
ituhin
·
3Comments
michaelr524
·
3Comments
Most helpful comment
Hey Guys, Just want to say thanks for all your efforts to implement genetic algorithms into zenbot. It has been the first introduction to the topic for me and i think im hooked :+1:
For the last few weeks I have been trying to add covariance matrix adaptation evolution strategy, which i believe will automatically adjust cx and mut probabilities to help with convergence and to prevent getting stuck on local maximums allowing us to develop and optimise more complex strategies.
I think this should be possible and relatively easy using the tools from the python deap package a la:
http://deap.readthedocs.io/en/master/examples/cmaes.html
https://github.com/DEAP/deap/blob/a5d77d1e30f7674948bb6a2c4381fa4e247ac89d/examples/es/fctmin.py
Unfortunately this is my first introduction to both python and evolution strategies, and to be honest, i'm a bit out of my depth. @gelotus, @arpheno I was wondering if you guys, (or anyone else) would be able to have a look (im sad to see development seems to have stopped on this fork). I think this would be very beneficial to zenbot development and for me personally very rewarding (emotionally... maybe financially too !) =)
So far I have basically tried to frankenstein the example into the code by swonzy but it is not working with the compiling error:
TypeError: sequence item 0: expected str instance, Individual foundI think I need help to correctly register fitness and individual functions into the deap toolbox..
````import sys
import numpy
from functools import partial
from timer import ThreadTimer
from deap import algorithms, base, benchmarks, cma, creator, tools
from deap.tools import cxTwoPoint, mutGaussian
from scoop import shared
from blessings import Terminal
from subprocess import run, DEVNULL
from conf import indpb, sigma, partitions, selectors, path, bkfint
from evaluation import evaluate_zen, Andividual
from evolution import evolve
term = Terminal()
creator.create("FitnessMax", base.Fitness, weights=(1.0,))
creator.create("Individual", list, fitness=creator.FitnessMax)
toolbox = base.Toolbox()
N=30
def main(instrument, days, popsize, zstrategy='trend_ema', nobf=None):
print(term.blue("Starting evolution...."))
evaluate = partial(evaluate_zen, days=days)
toolbox.register("evaluate", evaluate_zen, days=days)
print(term.blue("Evaluating ") + term.green(str(popsize)) + term.blue(" individuals over ")
+ term.green(str(days)) + term.blue(" days in ") + term.green(str(partitions)) + term.blue(" partitions."))
Andividual.path = path
Andividual.instruments = selectors[instrument]
Andividual.mate = cxTwoPoint
Andividual.mutate = partial(mutGaussian, mu=0, sigma=sigma, indpb=indpb)
Andividual.strategy = zstrategy
print(term.blue("Mating function is ") + term.green(str(Andividual.mate)))
print(term.blue("Mutating function is ") + term.green(str(Andividual.mutate)))