Azure-pipelines-agent: [Feature Request] .Net Core Execution Engine

[sandorfr]

Now .net core is quite mature and the agent is actually based on .net core. I think this would be a great addition to have a .net core execution engine. Currently the only xplat engine is Node which comes with it's own challenges.

I seems that implementing such support shouldn't be too hard as the architecture is already built with engine extensibility. As far as I understand it, it would consist of an handler implementation similar to the node one https://github.com/Microsoft/vsts-agent/blob/f7e9a7ee42d0c30f0d31d449bb51d95bd6e9e0fb/src/Agent.Worker/Handlers/NodeHandler.cs, and a helper task lib similar to the node one.

Given the complexity, this could even be an up-for-grabs issue.

[bryanmacfarlane]

Yeah, we want this!

The complication is tasks are self contained and carry a task-lib. We would need a net core task lib (not hard). https://github.com/Microsoft/vsts-task-lib

The problem is the agent auto updates, therefore we build a self contained agent (build for win, osx, redhat, ubuntu) and it carries .net binaries and runtime. We can't have net core as a pre-req and the agent auto updates and eventually drifts (starts using 1.1 apis) then after auto update, the agent breaks until you act and get on the box. So, we ship everything contained.

The problem that drives is since it's one task that runs all platforms, the task can't carry binaries built for all platforms and runtimes.

Does the task carry C# as a script and compiled JIT? maybe. Should the task declare what version of the net core runtime it needs and agent is capable of pulling on demand?

So, these are the discussions we're having. The core problem is the agent and tasks are updated independently so we need to think through what that means.

Any thoughts or opinions are appreciated.

[sandorfr]

I'm so glad to hear you are already thinking about it :). thanks for sharing the technical challenges :) so quickly.

Does the task carry C# as a script and compiled JIT I have a bad feeling about this approach as I see most of node.js issue arising (too many files/dependency hell, runtime compilation issues, node version incompatibility and quirks). Btw, I think the job of bringing dotnet-ish scripting ability, would be better addressed by extending powershell support to other platforms when its xplat support will be mature enough.

Should the task declare what version of the net core runtime it needs and agent is capable of pulling on demand? this feels like a better approach to me. Maybe hosted agents could also have LTS versions preloaded, so if one task relies on an LTS version it won't trigger any download and slow down. Maybe support for extensions should be restricted to LTS versions? As far as I can tell the node handler does not support any node version and that's quite understandable.

The core problem is the agent and tasks are updated independently As far as I understand, there is nothing wrong with having several .net version installed on the system and since the task would run in a child process (like node engine does), it should not be an issue?

Let's see other people ideas.

(Hint : If we can come to a clear specification of what should be done, I might be willing to contribute).

[sandorfr]

@bryanmacfarlane Now .net core is a bit more mature, have you or your team made any progress on thoughts about this?

[jabbera]

I don't see why you can't tree the core runtime as powershell or node. My tasks need powershell 5. I can't self contain that, so it's just a prereq. I've coded them to fail if it's not installed.

[bryanmacfarlane]

It's all about the OS dependencies. net core has a bunch of OS dependencies the user needs to install (unlike node). We're also working on running tasks in a docker container so that factors in. In the case where OS dependencies are not there, you won't even get a runtime check - it will blow up. With node, we map node into the container and run the task.

Net core 2.0 has self contained linux (similar to node) - just release. So first step is agent on core clr 2.0 (in progress), one linux distro, self contained apps. Next step is task lib in core clr 2.0. Docker and yaml in parallel but all related.

We've started the design doc but it's early. I'm half way into writing it.

https://github.com/Microsoft/vsts-agent/blob/master/docs/design/coreclr.md

[jachin84]

Just wondering if this is still on the cards?

[damccorm]

@TingluoHuang @bryanmacfarlane are we still considering adding a C# handler? Got some additional interest for this in the task-lib

[stephenmichaelf]

For reference, this is also discussed here.

[vtbassmatt]

We are not planning to build a native .NET Core handler.