Vscode: Explore future direction of the task runner system.

[dbaeumer]

THIS IS WORK IN PROGRESS

This issue will be used to discuss the future direction of the task runner system in VS Code.

History

The task runner system in VS Code got introduce in one of the first public releases of VS Code to allow calling out to an external program to do some work (building, compiling, running tests, upload, publish). Besides running the external program the task running system parsed the output to produce problem markers (errors, warnings and info messages).

When introduced there was no extension API available nor a terminal.

Current Limitations

The task running framework has currently the following major limitations:

• can only execute one task at a time
• can only handle one command definition (all tasks share the same command (e.g. program)). This is being worked around using a script to execute different programs.
• no contribution mechanism for problem matchers
• no contribution mechanism for build tasks. There can only be exactly one.
• no input. If the external program reads from stdin the task simply hangs.
• no support for output encoding (this is currently hard coded to utf8)
• no support for ANSI control characters
• minimal status UI (spinning progress)

Open Questions

Do we need more than one build task?

There are two major reasons for this

• problem matcher must be very specific (for example match on the file name extension) to not create false problems. Consider an example where a compiler A and B produce the same error format. If there are executed in one task with both problem matchers for MA and MB attached then matcher MB might produce a marker although the error was generated by compiler B. Since problem formats don't vary much this in not a uncommon case. Making the problem matcher file extension specific requires quite some customization (users need to define the problem matcher in the task.json or we support variables in problem matchers)
• watch task in gulp run in parallel which can result in interleaved output from different compilers if two files are saved using a Save All action. Consider the following gulp file as a demonstration:

var gulp = require('gulp');
var watch = require('gulp-watch');

gulp.task('watch-less', function () {
    return watch('**/*.less', undefined, function(vinyl) {
        let counter = 0;
        let interval = setInterval(function() {
            console.log('Compiling less file ' + counter);
            if (++counter === 5) {
                clearInterval(interval);
            }
        }, 10);
    });
});

gulp.task('watch-ts', function () {
    return watch('**/*.ts', undefined, function(vinyl) {
        let counter = 0;
        let interval = setInterval(function() {
            console.log('Compiling ts file ' + counter);
            if (++counter === 5) {
                clearInterval(interval);
            }
        }, 10);
    });
});
gulp.task('watch', ['watch-less', 'watch-ts']);

Having one build task which trigger gulp watch and then saving changes to a ts and less file with Save All produces the following single output.

[20:51:36] Using gulpfile P:\mseng\VSCode\Playgrounds\gulp\gulpfile.js  
[20:51:36] Starting 'watch-less'...                                     
[20:51:36] Starting 'watch-ts'...                                       
Compiling less file 0                                                   
Compiling ts file 0                                                     
Compiling less file 1                                                   
Compiling ts file 1                                                     
Compiling less file 2                                                   
Compiling ts file 2                                                     
Compiling less file 3                                                   
Compiling ts file 3                                                     
Compiling less file 4                                                   
Compiling ts file 4                                                     

Assigning errors to the correct problem matcher is again problematic may be even impossible if a problem matcher is a multi line matcher (e.g matcher for example the lines ["Compiling less file 0","Compiling less file 1"]

To handle this the watch-less and the watch-ts task are best execute as two separate tasks. Without an integration in VS Code dev would usually start the two gulp task in two different terminals to keep the output separate for readability as well.

Proposal

Build and Co.

First we should decouple task executing from build, rebuild, clean, run tests. The task runner should be able to participate in these commands however it shouldn't be the main driver of these commands. Using the Build command as an example this should work as follows:

1. Users presses execute the Build command
2. VS Code activates all extensions that want to participate in build
3. These extensions register a build participant like they register a completion item provider today.
4. The task framework is a build participant as well
5. The Build command calls all participants to execute a build. When all participants are done the build is complete.
6. To help parsing the output of an external program into problem markers VS Codes internal problem matcher library will be made available as an npm module.

Dependency management

If we have more than one build participant or even two independent build tasks the question raises if we need to define the order of the executed participants or build tasks. We have the following options:

• we execute them in random order. Usually extension participating in a build don't depend on other extensions. So this might work from an extension perspective. However for external build tools this might be problematic and creating a single build task that does the sequencing might raise problem matcher challenges as described above.
• we allow users to script the order. If done in a generic way this will require quite some development effort since VS Code has no scripting interface right now. We could support a way to allow the scripting via a JS file in the extension host however this might be confusing for dev that don't know JS at all (for example php or C# devs)
• we allow users to define the order using a json structure. This could be done using a comparable approach to gulp task dependency. Something like:

{
   "main": ["minify-js", "compile-less"],
   "minify-js": ["compile-ts"]
}

Open question:

• do we need to do something special for 'watching builds' ?
• how do we collect output. Will we have one build output that participants can write two ?

Executing an external tool

External tool tasks should be executed in a terminal instead of being executed hidden with its output made available in an output channel. This will allow us to support ANSI control characters, input, ...

[DanTup]

Some things I'd love to support in Dart Code:

Builds

If running a 'watching build' (similar to TypeScript - something transpiling Dart to JS) would like to know when the user hits F5 so we can wait for any active build to finish, then launch a browser (if the last one launched isn't still active, if it is we might refresh it). Possibly the extension development stuff already does something very similar to this though it might be special.

Tests

• Multiple test configs - Allow the user to easily run different sets of tests (essentially just different commands/arguments). I think this exists for tasks, but not sure if it extends to tests.
• Run tests on save - Allow the user to have Dart tests run automatically in the Dart VM after saving changes to a .dart file
• Run tests on successful build - Allow the user to have tests run in Node/browser after build/transpile finishes

These last two can probably be handled by something similar to a "watching build", but it would need to know when build finishes to reliably run the JS tests at the same time (I guess this would be the same if you supported TypeScript run-on-build tests).

I'd also really love to be able to output test results in some structured format so Code code display them nicely (not just terminal text output) and allow clicking to navigate to the test (or correct location if it crashed with a stack trace). This is out of scope for this case (and possibly for Code entirely) but maybe worth having thought about if changing how testing is configured.

[Tyriar]

I created an issue for moving tasks to use the terminal instead of an output channel https://github.com/Microsoft/vscode/issues/15584 which solves many of these issues.

[rkeithhill]

I think a big issue with tasks is that they aren't so easily discoverable given the importance of tasks. I think the ViewBar needs another tab for Tasks. This would make tasks much more discoverable.

[62mkv]

May I also suggest that current limitations be more visibly disclosed in the documentation?

I for one was struggling for a couple of hours yesternight trying to have background tasks functioning "as expected" :(

[62mkv]

Also, it's not quite clear why is it impossible (or seems impossible) to be able to run AND return (without waiting forever) for a spawned background processes. Should I create a separate issue on this ?

[dbaeumer]

Quite some work has happened for January here. It is:

• option to execute tasks in the terminal "_runner": "terminal"
• commands per task.

[rkeithhill]

BTW another issue is when extensions want to contribute tasks. Each extension having to modify tasks.json isn't ideal. It would be better if extensions could add a file like <extension-name>-tasks.json e.g. ms-vscode.PowerShell-tasks.json that VSCode could merge at runtime into the task runner.

[dbaeumer]

@rkeithhill this is on the backlog as well. The idea is to allow extensions to specific problem matchers and task in the package.json as contributions.

[dbaeumer]

More work has happened in February but still not 100% done.

[michael-hawker]

Not sure if I should file a new bug or list it under here since you're working on an overhaul.

If I run a PowerShell script as part of a task which wants to report progress it won't show that progress in the output window, even though it would if it was running in the terminal (though maybe the new terminal runner option is a workaround).

[dbaeumer]

@michael-hawker the way to go here is to use the new terminal runner framework.

[dbaeumer]

The March release will have two additional things implemented:

• extensions can contribute problem patterns and matchers statically via package.json
• extensions can contribute tasks dynamically using code. The task definition is now proposed API in March.

[rkeithhill]

Cool!

Any chance we will get a Side bar view for Tasks? For such an important part of VSCode, tasks are kind of buried. I can imagine a UI that shows tasks, allows you to run them, edit them, add them, delete them. And perhaps view the output of the last run of each. And it would indicate a running task and provide a way to terminate execution of the task.

At the very least, how about a predefined kbd shortcut for workbench.action.tasks.runTask? Perhaps Ctrl+Alt+t?

[gep13]

@agc93 fyi...

[dbaeumer]

A lot of progress has happened and we will provide a polished task 2.0.0 in June.

[DanTup]

@dbaeumer Is this something we can start looking at in Insiders? Is there a good place for info on how it works? Thanks!

[dbaeumer]

@DanTup the doc has not been written yet but the best place will be the 1.14 release notes. For the API you can already start looking at the vscode.proposed.d.ts in the source tree.

[DanTup]

@dbaeumer Great; thanks!

[dbaeumer]

Will keep the item open for August. Most of the stuff discussed in here is now in the 1.14 build.

[gep13]

@dbaeumer woot! Have started playing around with the new stuff for integrating Cake tasks. Looking forward to getting this pushed out. :+1:

[dbaeumer]

Moving to on deck. There is still some final polish work to do here.

[dbaeumer]

I am closing the issue. Please open new issues if there is something wrong or missing with the task 2.0.0. Having a single large item is very hard to act upon.