Gulp: Add sourcemaps into core

[contra]

cc @floridoo

Basically src/dest will have a sourcemap option. Now people don't need to use gulp-sourcemaps in their own files. This will DRY up a ton of stuff

[ilanbiala]

Awesome!

[addyosmani]

:+1:

[sindresorhus]

:+1:

[TrySound]

:+1:

[insidewhy]

Does this mean I'll finally be able to merge two streams along with their source maps?

[contra]

@nuisanceofcats Example?

[insidewhy]

@contra Like:

var gulp       = require('gulp')
var traceur    = require('gulp-traceur')
var concat     = require('gulp-concat')
var merge      = require('merge-stream')

gulp.task('default', function() {
  merge(
    gulp.src(['app/*.js', 'app/*/*.js']).pipe(traceur())
    gulp.src('bootstrap.js')
  )
  .pipe(concat('output.js'))
  .pipe(gulp.dest('dist/assets'))

})

With full source maps all the way through, the "merge" (or an alternative) would merge the streams and their source maps.

[contra]

That should still be possible with the current implementation though, just not as clean

[insidewhy]

@contra doesn't seem to be... I tried it like this:

var gulp       = require('gulp')
var traceur    = require('gulp-traceur')
var sourcemaps = require('gulp-sourcemaps')
var concat     = require('gulp-concat')
var merge      = require('merge-stream')

gulp.task('default', function() {
  merge(
    gulp.src(['app/*.js', 'app/*/*.js']).pipe(sourcemaps.init()).pipe(traceur())
    gulp.src('bootstrap.js').pipe(sourcemaps.init())
  )
  .pipe(concat('output.js'))
  .pipe(sourcemaps.write())
  .pipe(gulp.dest('dist/assets'))
})

But ended up with a bogus source map. I asked on github for help but none was forthcoming so I assumed it wasn't possible.

[contra]

That should work... @floridoo any thoughts?

[johnpapa]

:+1:

[boogerlad]

@nuisanceofcats Any luck fixing that? I'm trying to get the same thing working.

[insidewhy]

Have been waiting for gulp v4 and experimenting with my own stuff. All node based asset pipelines have major flaws or restrictions, v4 will probably sort it all out.

[boogerlad]

Sadly I'm on v4 alpha. Many things still seem to be not perfect. If I am compiling a less file to a css file, the sourcemap will refer to the outputted css file if I'm using multiple sources. Oh well.

[phated]

@boogerlad sourcemaps have nothing to do with v4 alpha. This was an issue to discuss implementing the boilerplate into gulp.src/dest. That has pushed off until gulp 4.1 anyway.

[boogerlad]

Sorry for going off topic.

[pikeas]

Based on my reading of this thread, there's no way to merge sourcemaps in Gulp v3, and this won't be added until v4.1, which is months away?

[contra]

@boogerlad if there is an issue with gulp-sourcemaps then open an issue on gulp-sourcemaps, no point in reporting it on an unrelated issue

[contra]

Moving this back to 4.0 since this is pretty easy to do

[contra]

@floridoo any thoughts RE merging streams of sourcemaps?

[floridoo]

@contra the source maps are per file, so merging streams should not affect them in theory. maybe the sourcemaps property is dropped by the merge plugin? i will have a look when i find the time.

[insidewhy]

@floridoo Combining source maps needs special handling that currently isn't in gulp. Here's an implementation:

https://github.com/sighjs/sigh-core/blob/master/src/sourceMap.js#L26

[floridoo]

@ohjames merge does not combine files or sourcemaps, that's what gulp-concat is for. merge just merges streams.

[insidewhy]

Yep, unfortunately if you merge streams before passing them into concat (e.g. the input streams each use different transformations) then the output source maps are incorrect. I think there is/was a github issue floating around for this.

[insidewhy]

My previous comment is incorrect/misleading though, you're right gulp-concat does exactly that... I think.

[contra]

@ohjames before we bring this into core I want to make sure that any edge cases are handled. If you have any other concerns/oddities feel free to raise them here

[contra]

Added into src on vinyl-fs master, going to do dest later today then make it available for testing on 4.0 :+1: https://github.com/wearefractal/vinyl-fs/commit/60685eb89e57f61c4462855532f43d75f0a616db

[insidewhy]

Cool, anyone else think the default setting for source maps should be true rather than false?

[insidewhy]

@contra I tested the merge + concat again with the git master commit, the same test I ran back in my comment from December. It still fails but I found the issue, essentially it's because one of the sources is not transformed (the bootstrap.js which should just be appended raw) and hence has an empty source map attached by this line of code:

https://github.com/floridoo/gulp-sourcemaps/blob/master/index.js#L107

gulp-concat should detect empty source maps and replace them with identity source maps. Either this or that line there could be changed to attach an identity source map instead of an empty source map.

sigh takes the former approach as it's slightly more efficient and abstracts it behind the source map API. With that small change then gulp-concat will produce correct source maps :)

[contra]

@ohjames I thought about making it true by default, but I think it makes more sense to opt-in to having more files pop up in the output and more CPU usage

[phated]

Looks like this is 50% done (included in gulp.src but not gulp.dest). Marking as "help wanted" incase anyone wants to send a PR to https://github.com/wearefractal/vinyl-fs that adds the dest support.

[contra]

@phated Just added it into .dest this morning, needs tests and docs though

[joshribakoff]

Inlining the sourcemaps like this breaks lots of older browsers like IE. If we want to support IE, is there some way to disable this? The behavior I seem to be seeing is that its enabled by default now, which means after updating gulp, lots of older browsers suddenly broke with no obvious fix.

Edit Actually not sure what's going on here. I had my calls to gulp-sourcemaps commented out, and was getting intemittent sourcemaps at random. I removed the lines altogether & got no sourcemaps in output. I'm not looking for help debugging, just pointing out a possible headache that would be caused if something like this were to be merged in & enabled by default.

[contra]

@joshribakoff Not sure why you are saying it's on by default when that was never said and certainly isn't true

[contra]

Also FYI for any interested parties, the sourcemaps option is now available for src and dest on vinyl-fs master. Please test out the changes and I'll cut a release if everything looks good

[contra]

im going to close this since it has been implemented, but it still needs testing

[eddiemonge]

Whats the api for this?

[Dan503]

// How to use native Sourcemaps in Gulp 4 using ES5

var sass = require('gulp-sass');

gulp.task('sass', function() {
  return gulp.src('src/styles/main.scss', { sourcemaps: true })
    .pipe(sass().on('error', sass.logError))
    .pipe(gulp.dest('site/css', { sourcemaps: true }))
});

In short you need to set { sourcemaps: true } on _both_ gulp.src and gulp.dest for it to work.

Here is a slightly sleeker ES6 way of writing it:

// How to use native Sourcemaps in Gulp 4 using ES6

import sass from 'gulp-sass';

const sourcemaps = true;

gulp.task('sass', () => {
  return gulp.src('src/styles/main.scss', { sourcemaps })
    .pipe(sass().on('error', sass.logError))
    .pipe(gulp.dest('site/css', { sourcemaps }))
});

It outputs the sourcemap _inside_ the CSS file though. Is there a way to output .map files instead without resorting to the gulp-sourcemaps plugin?

[Dan503]

Ahh I found the bit of documentation covering how to do external source maps :D

https://gulpjs.com/docs/en/api/src#sourcemaps

// How to use native external Sourcemaps in Gulp 4

var sass = require('gulp-sass');

gulp.task('sass', function() {
  return gulp.src('src/styles/main.scss', { sourcemaps: true })
    .pipe(sass().on('error', sass.logError))
    .pipe(gulp.dest('site/css', { sourcemaps: '.' }))
});

'.' = a relative path to the directory you want the sourcemap to go in. '.' = same directory as the output file.