Gulp: task not waiting for dependency to finish

[tcurdt]

Shouldn't this work?

var gulp = require('gulp');
var del = require('del');

...

gulp.task('clean', function(callback) {
  del([ site.build + '/**' ], callback);
});

gulp.task('build', [ 'clean' ], function() {
  return gulp.start('html', 'less', 'js', 'png', 'jpg');
});

gulp.task('deploy', [ 'build' ], function() {
 ...
});

Unfortunately deploy runs before build finished.

gulp.task('clean', function(callback) {
  del([ site.build + '/**' ], callback);
});

gulp.task('build', [ 'clean', 'html', 'less', 'js', 'png', 'jpg' ], function() {
});

gulp.task('deploy', [ 'build' ], function() {
 ...
});

The above does not work either as clean is run in parallel.

So what's the suggested way of doing this?

[jehoshua02]

@tcurdt I really hope you don't get shut down on this one. I've been wondering the same thing.

[tcurdt]

Some progress. This seems to work

var run = require('run-sequence');

gulp.task('build', [ 'clean' ], function(callback) {
  run(['html', 'less', 'scss', 'coffee', 'js', 'png', 'jpg'], function(){
    callback();
  });
});

gulp.task('deploy', [ 'build' ], function() {
  ...
});

although I am wondering whether it should return or not.

[robrich]

Duplicate of https://github.com/orchestrator/orchestrator/issues/15. Sadly using gulp.start inside a task and using the "when it's done" function together with dependencies does bad things in gulp 3. This is one of the main things we're resolving in gulp 4.

[tcurdt]

@robrich Thanks for the pointer. Do you know anything about the state/release plan for gulp 4?

[robrich]

Sorry, I don't.

[blvz]

@tcurdt maybe you can already give gulp4 branch a spin. See: https://github.com/gulpjs/gulp/issues/557#issuecomment-73769701

[davisford]

I'm seeing a similar issue with tasks like this:

gulp.task('one', function () {
   return gulp.src('/path/file')
      .pipe(somePlugin())
      .pipe(gulp.dest('/path/file1'))
})

gulp.task('two', ['one'], function (done) {
   // async code here reads output from task one and writes to new file
  var read = readline.createInterface({ input: fs.createReadStream('/path/file1'), output: process.stdout, terminal: false}),
    file = fs.createWriteStream('/path/file2');
  read.on('line', function (line) {
    // process line
    file.write(line + '\n');
  }).on('close', function () {
    file.end();
    done();
  });
});

gulp.task('three', ['two'], function () {
   // tries to read file output from task two
   return gulp.src('/path/file2')
      .pipe(somePlugin())
      .pipe(gulp.dest('/path/file3'))
})

When I run gulp three, task three starts just slightly before task two finishes writing and closes; the file contents aren't quite there yet, and, as it reads an empty file, task three outputs an empty file.

I can open /path/file2 and look at the contents and it is not empty after the gulp process finishes.

The docs indicate that this scenario is supported. I'm wondering if there's some kind of regression?

$gulp --version
[18:32:55] CLI version 3.8.10
[18:32:55] Local version 3.8.10

[contra]

Use gulp4 this is not going to be fixed on the 3.x branch

[davisford]

Ouch -- I appreciate the prompt response (and the open source contribution), but I briefly tried the 4.0 branch, and don't really have the luxury to rewrite all my build scripts right now. I'm just dropping this in here in case anyone else is in the same boat. I'll just re-write the task to ensure it executes in sequential order in a different fashion.

My mistake: in my example above, I was calling done() on the read stream close; it needed to be changed to wait and call done on the write stream close. Terribly sorry for the bother.

file.on('finish', function () { done(); });
read.on('close', function () { file.done(); });

[contra]

@davisford :palm_tree: Not meant to be harsh, just letting you know we are no longer taking issues for the 3.x branch unless it is something that is a major issue we can backport easily (like node 0.12 and io.js support) :palm_tree:

[thomashigginbotham]

Even with run-sequence, I was having the same problem. Here's how I worked around it:

gulp.task('build', function(callback) {
  gulp.src('abc.xyz')
    .pipe(someTask())
    .pipe(gulp.dest('dist'))
    .on('end', callback);
});

gulp.task('deploy', [ 'build' ], function() {
  /* The .on('end', callback) above ensures that the "build" task has fully completed. It's
     now safe to use "dist/abc.xyz". */
});

[phated]

@thomashigginbotham your issue is completely different. You should just be returning your streams, always.

[thomashigginbotham]

I agree that it's best practice to always return a stream, and my example was simplified for readability -- I just forgot to add the return keyword when typing it out. My point still stands, as it resolved the issue of guaranteeing the file operation was complete before continuing with the next task.

[leonliao]

Met same issue here. Let's say there are 2 tasks, First and Second. Second runs after First.
The First task generates some files, which are to be read by the Second task. Using dependency doesn't make sure the Second task can find the files generated.

I have to explicitly using the done callback on the pipeline to let Second only starts after First truly done.

//This approach works!
gulp.task('First', function(done)) {
   var subFolders = fs.readdirSync(somefolder)...
   var tasksForFolders = subFolders.map(function(folder) {
       return gulp.src('folder/**/*').sthtogeneratefiles();
   });
   tasksForFolders[tasksForFolders.length-1].on('end',done);
   return tasksForFolders;
}

gulp.task('Second', ['First'],function() {
    return gulp.src('generatedfolders/**/*').doth();
}

Without the done trick, the Second never finds the files generated by First. Below shows what I tried, the Second task can find the files generated by calling gulp First by hand, and then calling gulp Second subsequently.

//This is the WRONG approach, just for demonstration!!
gulp.task('First', function()) {
   var subFolders = fs.readdirSync(somefolder)...
   var tasksForFolders = subFolders.map(function(folder) {
       return gulp.src('folder/**/*').sthtogeneratefiles();
   });
   return tasksForFolders;
}

gulp.task('Second', function() {
    return gulp.src('generatedfolders/**/*').doth();
}