Marathon: Add ability to launch one-off tasks
Many apps require the ability to launch a one-off task (for example for database migrations). Add an easy way to do this in Marathon.
guenter
All 41 comments
This branch has a POC: https://github.com/mesosphere/marathon/tree/one-off
It needs cleanup.
guenter
on 17 Jul 2014
+1
chendo
on 14 Oct 2014
We've hacked up a super dirty nsinit solution that lets us shell into a Docker container, however we're worried about the extra memory usage that might trigger the container into being killed
chendo
on 14 Oct 2014
+1
ccverak
on 15 Oct 2014
:+1:
mo-mughrabi
on 21 Oct 2014
+1; any progress on this?
andytinycat
on 16 Nov 2014
+1
defender
on 28 Dec 2014
+1
gregory90
on 13 Jan 2015
+1
doctapp
on 16 Mar 2015
+1
j1n6
on 17 Mar 2015
For now, the only way we found to run one-off tasks is to use the suicide method:
{
"id": "myapp",
"cmd": "somecommand && curl -X DELETE marathon:8080/v2/apps/myapp"
...
}
doctapp
on 26 Mar 2015
+1
On Thu, Mar 26, 2015 at 4:15 PM Martin Tapp [email protected]
wrote:
For now, the only way we found to run one-off tasks is to use the suicide
method:
{
"id": "myapp",
"cmd": "somecommand && curl -X DELETE marathon:8080/v2/apps/myapp"
...
}
Reply to this email directly or view it on GitHub
https://github.com/mesosphere/marathon/issues/344#issuecomment-86697361.
ccverak
on 26 Mar 2015
+1
xjewer
on 24 Apr 2015
+1
mikljohansson
on 10 Jun 2015
Any progress on this?
doctapp
on 12 Nov 2015
Indeed, is it something the team may look into in the future?
Lawouach
on 30 Nov 2015
Hey @Lawouach @doctapp and previous commenters,
Yes, we're aware that this makes sense in the context. There is an ongoing internal discussion on integrating one off job support into Marathon, but we do not yet have a clear direction.
Feel free to share use cases: migrations, setting up prerequisites, general cron jobs – which features would you need in order to have decent benefit?
meichstedt
on 1 Dec 2015
One use case is running an ad hoc job such as running a batch job (e.g.
ETL) or training a machine learning model.
The principal benefit is that we can perform these tasks whenever they are
required (daily basis) without extensive planning.
We are using the 'suicide hack' that I posted for these jobs. We are using
marathon since it's API is better suited to our needs than a framework like
Chronos.
On Mon, Nov 30, 2015 at 9:19 PM, Matthias Eichstedt <
[email protected]> wrote:
Hey @Lawouach https://github.com/Lawouach @doctapp
https://github.com/doctapp and previous commenters,
Yes, we're aware that this makes sense in the context. There is an ongoing
internal discussion on integrating one off job support into Marathon, but
we do not yet have a clear direction.Feel free to share use cases: migrations, setting up prerequisites,
general cron jobs – which features would you need in order to have decent
benefit?—
Reply to this email directly or view it on GitHub
https://github.com/mesosphere/marathon/issues/344#issuecomment-160826800
.
doctapp
on 1 Dec 2015
Another use-case, though it's more likely a workaround at this stage, is to run a task to post-configure a task. Unless I have missed something, Marathon doesn't offer me a way to run post-configuration scripts from the host where those tasks are running so I resort to one-shot tasks.
I grant you this is more a workaround than a real use-case mind you.
Lawouach
on 1 Dec 2015
What do you all think about having a separate service for one-off jobs with some Marathon integration, e.g. for life cycle hooks?
Or another way of asking: What would be missing compared to a solution where it all is integrated into Marathon? What does "integration" mean for you?
kolloch
on 1 Dec 2015
The idea of running a Marathon job to handle one-off jobs beats the purpose of a scheduler handling such a task. Plus, we don't need another task taking cluster resources for that purpose; that's the framework's job.
We need a flag or some way of telling Marathon not to restart a job if it finishes.
doctapp
on 7 Dec 2015
@doctapp I should have made it more clear: My idea was creating a separate service (= framework) that is able to run any number of one-off tasks. So you would need to talk to a different end-point to achieve this.
In Marathon itself we could include hooks for life-cycle events that would end-up creating one-off tasks using the separate service.
Would that satisfy your requirements? If not, what exactly is missing.
kolloch
on 14 Dec 2015
The problem with another framework is that we would need to support two
types of APIs. From my perspective, we need Marathon's api with one-off
task launching and handling. That's why we're not using Chronos or others.
Why can't you simply extend Marathon for this purpose? The suicide hack is
the better alternative to another framework in my opinion...
On Mon, Dec 14, 2015 at 7:47 AM, Peter Kolloch [email protected]
wrote:
@doctapp https://github.com/doctapp I should have made it more clear:
My idea was creating a separate service (= framework) that is able to run
any number of one-off tasks. So you would need to talk to a different
end-point to achieve this.In Marathon itself we could include hooks for life-cycle events that would
end-up creating one-off tasks using the separate service.Would that satisfy your requirements? If not, what exactly is missing.
—
Reply to this email directly or view it on GitHub
https://github.com/mesosphere/marathon/issues/344#issuecomment-164428197
.
doctapp
on 14 Dec 2015
@doctapp "Why can't you simply extend Marathon for this purpose?" We probably could extend it. I question the "simply". I'd rather focus on making single components of software as simple as possible.
In my opinion, the API for one-off-jobs SHOULD be different. For one-off-jobs, it is common to wait for them to complete (which does not make sense for long-running services). There is more...
Also the scheduler could be optimized for this different use case. For long-running tasks, you expect to have relatively few task starts/terminations. For one-off-scheduling that is the norm.
That said it should also use the same abstractions where possible (e.g. how to specify the command to run, resource constraints, ...).
kolloch
on 14 Dec 2015
Agree and totally understand this. I don't mind having a distinct marathon
API for this purpose, as long as it's aligned between both type of tasks.
One-off tasks only need a subset of what long-running tasks expect.
That's why having one common API and endpoint (being a single Marathon
endpoint from the client's perspective), but with different
frameworks/implementation would be very interesting from my perspective...
We just don't want to deal with different framework APIs to launch tasks
that share common parameters mainly ports, resources...
On Mon, Dec 14, 2015 at 9:32 AM, Peter Kolloch [email protected]
wrote:
@doctapp https://github.com/doctapp "Why can't you simply extend
Marathon for this purpose?" We probably could extend it. I question the
"simply". I'd rather focus on making single components of software as
simple as possible.In my opinion, the API for one-off-jobs SHOULD be different. For
one-off-jobs, it is common to wait for them to complete (which does not
make sense for long-running services). There is more...Also the scheduler could be optimized for this different use case. For
long-running tasks, you expect to have relatively few task
starts/terminations. For one-off-scheduling that is the norm.That said it should also use the same abstractions where possible (e.g.
how to specify the command to run, resource constraints, ...).—
Reply to this email directly or view it on GitHub
https://github.com/mesosphere/marathon/issues/344#issuecomment-164451917
.
doctapp
on 14 Dec 2015
Has there been any recent progress on this?
assumednormal
on 26 Feb 2016
Good news everyone - this is now actively in design phase.
air
on 14 Apr 2016
@air This is a much needed feature. Is there any high level estimate when can this be ready in the main branch?
ghost
on 15 Apr 2016
Is there a work around to have one-off docker containers using Marathon?
@doctapp Provided a nice workaround with suicide method for the commands. Is there anyway to achieve this for docker containers?
@chendo Can you please provide more ideas for the hack you tried?
ghost
on 15 Apr 2016
+1
ryanolson
on 4 May 2016
+1
database migration
batch configuration(pre,post,etc) for multi apps
junneyang
on 10 May 2016
+1 much needed feature imho.
Run arbitrary command in the same "context" as the defined app (same env, same docker image etc),
a.) allow to specify if the N instances of the deployed task can be spawned in parallel with this command or shortly after it finishes.
b.) Configured timeout on the command would be a nice addition too.
Use cases - cache warming, pre-start cleanup of some distributed services, database schema migrations and much more.
strzelecki-maciek
on 10 Jun 2016
+1
nathanjackson
on 24 Jun 2016
@doctapp Provided a nice workaround with suicide method for the commands. Is there anyway to achieve this for docker containers?
kartikjoshi07
on 10 Aug 2016
All, we have built Metronome _on top of the Marathon core_ to provide this functionality. Please direct any requests to that project.
meichstedt
on 15 Aug 2016
Has there been any recent progress on this?
yuandy
on 26 Aug 2016
+1
grogsaxle
on 30 Sep 2016
+1
rmrfself
on 11 Oct 2016
+1
ozayduman
on 19 Dec 2016
By popular request, we're happy to actually work on this. #5009 will provide more details soon.
Yes! It's true! It's happening!
meichstedt
on 25 Jan 2017
Excellent, finally no more suicide hack!
doctapp
on 25 Jan 2017
Related issues
alenkacz
·
5Comments
medzin
·
5Comments
bydga
·
8Comments
pgkelley4
·
11Comments
timcharper
·
8Comments
Most helpful comment
Good news everyone - this is now actively in design phase.