Azure-devops-docs: Health-Check for containers

Created on 23 Jan 2020  Â·  10Comments  Â·  Source: MicrosoftDocs/azure-devops-docs

Hey guys,
how can I configure my Healthcheck to check if my containerized build agent is still connected to azure devops? So that I can configure my orchestrator to spin up new instances of my container, when they lost connection.

I think this is even more intresting, if someone is using the --once parameter in his docker file in a kubernetes. This garantees new agents after every job...
https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/v2-linux?view=azure-devops#run-interactively.

Your current linux agent isn't running the ./run.sh script anymore, it's using a node-based agent software. Is there any documentation for it yet?
Is this also able to run with a "-once" argument?
How can I health check this in kubernetes , when I run it with the "-once" parameter.

Is running a container with "-once" shutting down the container after the job completely?

Kind regards,
Bjego


Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

Pri1 devops-cictech devopprod doc-bug

Most helpful comment

@WilliamAntonRohm @steved0x thank you for the hints. I think this should be part of the official docu to help your customers to run multiple agents in a kubernetes cluster.
Maybe a section "How to run in kubernetes" with a common configuration would be nice. This should cover scaling and unregistering stopped agents.
Kind regards,
Bjego

All 10 comments

@Bjego -- Thanks for your question. You may find answers here:​

As well, you might also consider using the Report a Problem tool to report the issue.

@steved0x -- please look into this issue.

@WilliamAntonRohm @steved0x thank you for the hints. I think this should be part of the official docu to help your customers to run multiple agents in a kubernetes cluster.
Maybe a section "How to run in kubernetes" with a common configuration would be nice. This should cover scaling and unregistering stopped agents.
Kind regards,
Bjego

second Bjego's comment above - this would be most useful!

@Bjego How did you make your linux agents forcing restart after each job ? did you find any documentation ? I was able to use --once for windows one but not in linux.

@Nirav-Bhadradiya hey, sorry vor the very late response.
I think it must be

exec ./externals/node/bin/node ./bin/AgentService.js interactive

See the shellscripts ending:
https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops#linux

Hi @Bjego ,

currently i have the same problem.
Did you found a nice solution for a livenessProbe and redinessProbe?
I would be very interested in it.
Cheers
Christian

Hi @Bjego ,

currently i have the same problem.
Did you found a nice solution for a livenessProbe and redinessProbe?
I would be very interested in it.
Cheers
Christian

Hi Christian,
unfortunally not as I had other topics to work on and this has not the highest priority at the moment. We are just preparing Azure Devops for our organisation and we still have to do quite a lot of paper work... But I thought the documentation for the agents could be better. Not sure when I do have time to fiddle around with the kubernetes based agents.
Cheers
Bjego

Hi Bjego,

to go on i solved this issue with a shell script. I just added a script like the following into the container and set a probe to it.

if [[ $(/bin/ps -ef | /bin/grep "/bin/AgentService.js" | /bin/grep -v grep | /usr/bin/wc -l) -ge "1" ]]; then exit 0; else exit 1; fi
It is just looking if there is a process.
Not a great and ideal solution, but i had no other idea.
Better would to communicate with the process to see if it really running.

@Lacerus regarding to #8416, i've changed the last line of the start.sh script now to use not the AgentListener.js via node. I'm using now the official documented wrapper and shut down the container after every build (not yet tested in my on prem cluster):
exec ./run.sh --once

old:

# `exec` the node runtime so it's aware of TERM and INT signals
# AgentService.js understands how to handle agent self-update and restart
exec ./externals/node/bin/node ./bin/AgentService.js interactive

cc @steved0x

Was this page helpful?
0 / 5 - 0 ratings