Horizon: [proposal] Horizon to manage multiple apps

[andrewmclagan]

In a modern docker based environment (come on guys if your not using docker you should be) people often run multiple Laravel applications (micro-services) with their own queues.

Is it possible to use Horizon to manage multiple applications wiht multiple queues? We use Redis and have each redis queue using a different name.

If its not we would be happy to donate some resources to write a PR.

[ellisio]

I'm actually about to embark on getting Horizon configured on Kubernetes myself via Helm chart installs. I'm curious to hear any thoughts you have on the subject as I can't find anything super helpful anywhere.

[andrewmclagan]

Well we use it by directly creating a container within a pod for horizon.

So a Laravel pod would have the following containers:

• App (php-fpm, nginx)
• Horizon (php-fpm, supervisord, horizon-worker, nginx)
• Schedule (crond, php-fpm)

This is the supervisor config for a Horizon container

[supervisord]
logfile=/var/log/supervisord.log
pidfile=/var/run/supervisord.pid
nodaemon=true
loglevel=debug
user=root

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/artisan horizon
autostart=true
autorestart=true
numprocs=1
redirect_stderr=true

[program:php-fpm]
pidfile=/var/run/php-fpm.pid
priority=5
command=php-fpm
autostart=true
user=root

[program:nginx]
pidfile=/var/run/nginx.pid
priority=10
command=/usr/sbin/nginx -g 'daemon off;'
autostart=true
user=root

You dont need a helm chart... its not what helm is for.

[ellisio]

@andrewmclagan This is exactly what Helm is for.

Helm helps you manage Kubernetes applications — Helm Charts helps you define, install, and upgrade even the most complex Kubernetes application.

Our company is developing microservices using Laravel at the moment; and will soon be opening the doors for developers to pick a language of their choice to build services in that best fit the service's need. Because of this we are combining Jenkins and Helm for CI/CD so we can give our developers a way to deploy applications without having to worry about infrastructure needs.

Our solution, for Laravel projects, was to create a sm/laravel chart. We add the sm repo during a Jenkins build so their build can call a simple helper we wrote into a Jenkins library like so:

sm.deployLaravelChart()

Each Laravel project has a charts/ directory, which contains the following files:

• values.yaml: Global file for all environments.
• values-pr.yaml: For ephemeral Pull Request environments (removed after 4 hours of no use)
• values-qa.yaml: For the static QA environment.
• values-prod.yaml: For the static Prod environment.

This helper detects if the app was previously installed on {env} to determine if it needs to run helm install or helm upgrade. Appended to those commands are --name ${GIT_ORG}-${GIT_REPO} --version ${GIT_COMMIT}. Which will install/upgrade commit (version) abcd123 of the my-org-hello-world application via Helm.

Now for our sm/laravel chart. This contains the following:

• http-svc.yaml
• http-ingress.yaml
• http-deployment.yaml
• horizon-svc.yaml
• horizon-ingress.yaml
• horizon-deployment.yaml
• queue-svc.yaml
• queue-deployment.yaml

The http files are for Laravel and all web requests. The horizon files are for Horizon's dashboard http service. The queue files are for all workers. This allows us to define multiple workers, and have a single deployment file to handle their auto scaling, or replica.

I spent the last week implementing this flow and Helm is an absolutely needed tool for this kind of deployment. It keeps developers focused on "configuration" and not on "implementation" when it comes to the infrastructure.

[ellisio]

Also, for anyone stumbling upon this and thinking about going the pod route (which is totally fine), just make sure you deploy your pods using Deployments, or ReplicaSets. This way they will be rescheduled if the k8s node your pod is running on fails.

More info: https://kubernetes.io/docs/concepts/configuration/overview/#naked-pods-vs-replicasets-deployments-and-jobs

[andrewmclagan]

ellisio the issue I see is your storing environment config inside your service-repositories? Thats bad practices IMO. Code configs should be external to the code.

[andrewmclagan]

And yeah we totally use Deployments. We have a folder in all our laravel services /operations and that contains services.yaml and deployments.yaml k8 artifacts. We use Drone a pure docker CI that updates the deployments via kubectl

(we use Laravel/PHP and Golang backend services and React frontend)

[ellisio]

The only envvars we store are for environment specific, that can be stored in clear text. APP_KEY, etc. are stored via secrets and set by the Helm charts for the Deployments.

Trust me, security is important! I tend to verbally slap anyone who stores sensitive info in VCS.

On May 26, 2018, 6:53 AM -0600, Andrew McLagan [e-mail], wrote:

And yeah we totally use Deployments. We have a folder in all our laravel services /operations and that contains services.yaml and deployments.yaml k8 artifacts. We use Drone a pure docker CI that updates the deployments via kubectl
(we use Laravel/PHP and Golang backend services and React frontend)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[andrewmclagan]

:-) cool. Configuration is a tough nut to crack generally I find. We use Vault and K8s integrations. Its pretty good although probably a little much for our use-case...

Sounds like you have a tight setup :-)

[driesvints]

Going to close this as this can best be asked on a support channel. Please only report issues with Horizon on this issue tracker. Thanks!