Conductor: Workers Not Executing with standalone redis db

[johnlongo]

Trying to setup an conductor instance using a standalone redis db, I'm able to create workflows, start workers polling without any issues. But when I schedule a workflow none of the workers are getting triggered. BTW when I use the exact same config with a MYSQL db they are getting triggered.

Any suggestions?

Below is my Server Properties File (NOTE: CHANGED SERVER DOMAIN TO XXXXXXXX.XXX for this Post)

Servers.

conductor.jetty.server.enabled=true
conductor.grpc.server.enabled=false

environment=prod
STACK=prod

Database persistence model. Possible values are memory, redis, and dynomite.

If ommitted, the persistence used is memory

#

memory : The data is stored in memory and lost when the server dies. Useful for testing or demo

redis : non-Dynomite based redis instance

dynomite : Dynomite cluster. Use this for HA configuration.

db=redis

Dynomite Cluster details.

format is host:port:rack separated by semicolon

workflow.dynomite.cluster.hosts=app-redis-1.XXXXXXXX.XXX:6379:us-east-1e

Dynomite cluster name

workflow.dynomite.cluster.name=conductor

Namespace for the keys stored in Dynomite/Redis

workflow.namespace.prefix=conductor

Namespace prefix for the dyno queues

workflow.namespace.queue.prefix=conductor

No. of threads allocated to dyno-queues (optional)

queues.dynomite.threads=10

workflow.dynomite.connection.maxConnsPerHost=31

Non-quorum port used to connect to local redis. Used by dyno-queues.

When using redis directly, set this to the same port as redis server

For Dynomite, this is 22122 by default or the local redis-server port used by Dynomite.

queues.dynomite.nonQuorum.port=6379

Elastic search instance type. Possible values are memory and external.

If not specified, the instance type will be embedded in memory

#

memory: The instance is created in memory and lost when the server dies. Useful for development and testing.

external: Elastic search instance runs outside of the server. Data is persisted and does not get lost when

the server dies. Useful for more stable environments like staging or production.

workflow.elasticsearch.instanceType=external

Transport address to elasticsearch

workflow.elasticsearch.url=app-es-2.XXXXXXXX.XXX:9300

Name of the elasticsearch cluster

workflow.elasticsearch.index.name=conductor

Additional modules (optional)

conductor.additional.modules=class_extending_com.google.inject.AbstractModule

Load sample kitchen sink workflow

loadSample=false

[johnlongo]

I've added a redis monitor log, when I start a workflow.
What I see is a few things:
1) I see the new workflow & task being added to the redis DB (i.e MESSAGE._deciderQueue, WORKFLOW_DEF_TO_WORKFLOWS, PENDING_WORKFLOWS, etc.)
2) I see a few "GET" for WORKFLOW, TASK. etc.
3) I'm also seeing the POLLING Request getting SET.
But that's it, nothing else.

NOTE: I do see a bunch of request for QUEUE with a suffix of ".c" that dont exist in the DB

monitor.log

[Mentis]

@johnlongo just faced the same issue myself. You need to specify following properties in your config or via system property:
EC2_AVAILABILITY_ZONE=us-east-1e
EC2_REGION=us-east-1

They should match region in workflow.dynomite.cluster.hosts
Otherwise default values will be used (us-east-1c) and DynoQueue won't work.

[cyzhao]

@kishorebanala this is a faq candidate.