Caprover: [Question] Having multiple leader on the cluster and cron job

[PolMrt]

Hi,

First of all, thanks for this awesome project. It helps me a lot to reduce my hosting cost.

While starting to deploy some of my side projects, I was wondering if it’s possible to have multiple leaders on the cluster in order to keep all the app working even if one of the server need to be stoped to do some maintenance on it ?

Also, I did some research in order to perform cron job. I saw a good answer on #158 referring to this stackoverflow question (I didn’t try it for now) but someone seems to say that it’s not longer working. So what’s the best way to do cron job ?

Thanks for your help!

[githubsaturn]

I was wondering if it’s possible to have multiple leaders on the cluster in order to keep all the app working even if one of the server need to be stoped to do some maintenance on it ?

Other than the main leader node, you can take down any node from your cluster and docker swarm automatically reassigns the services to other available nodes.
The leader node is different. CapRover stores the configuration in /captain directory on the leader node. If you take down this node, you won't be able to start your CapRover from another manager node because CapRover doesn't have access to that configuration directory.

Having said that, there is a way to achieve this. You can manually rsync this directory to another node and move the CapRover service to that node by changing it's placement constraint. So there is some scripting and knowledge of Docker swarm involved in that process.

---

Also, I did some research in order to perform cron job. I saw a good answer on #158 referring to this stackoverflow question (I didn’t try it for now) but someone seems to say that it’s not longer working. So what’s the best way to do cron job ?

cron jobs in Docker isn't really a CapRover specific topic. People have different views on what the practice is to run cron jobs in Docker. Depending on the use-case, some people suggest to run the cron job in the same container, some suggest that it should be a separate container. Depending on your use case, either of these solutions may be good for you.

[MitchellCash]

@githubsaturn is this something that could be built into the UI in the Clusters tab?

When you see your list of cluster nodes, any manager node that isn't a leader could have a "Promote to Leader" button that automates the task of reassigning the leader?

If you think this is something that you would build into CapRover I can turn this comment into its own feature request?

[githubsaturn]

The issue is not the fact that what node is the leader node in swarm. The underlying issue is the data that sits in /captain directory. So even if you change the leader node to another node, it doesn't make your cluster be resilient.

[MitchellCash]

@githubsaturn yes, sorry didn't explain it well enough, I was thinking because the Manager (leader) node holds SSH keys for the other cluster nodes that when you "Promote to Leader" it could use something like scp to copy the /captain dir to the new leader.

Essentially automating the task that you would usually do manually.

[digitalsanity]

I have successfully used lsyncd in the past for scenarios like this. It uses inotifywatch to monitor directories for changes and synchronizes those changed files using rsync to other members of the cluster. Very simple to setup and uses rsync.

It's been around forever. When Chef was barely a thing, I implemented a simple bash-based role-based system that used lsyncd, hostname checking and a 'heartbeat' script to force-overlay configuration files onto nodes, with transition states to initialize or transform (no APIs for spinning up instances..) between roles.