gulp.src does not copy subdirs within symlink

[k-funk]

$ gulp -v
CLI version 1.2.2
Local version 4.0.0-alpha.2

Directory structure of images and symlink'ed directory (shared-assets):

├── images
│   └── branding -> ../shared-assets/branding/     (this is a symlink)
├── shared-assets
│   ├── branding
│   │   ├── img1.png
│   │   ├── img2.png
│   │   └── dir1
│   │       ├── img3.png
│   │       └── img4.png

gulp.src('..../images/**/*.{.jpg,.png}').pipe(gulp.dest('dist/images'))

Produces:

├── images
│   ├── branding
│   │   ├── img1.png
│   │   └── img2.png

gulp.src('..../images/**/*').pipe(gulp.dest('dist/images'))  // matches all files

Produces:

├── images
│   ├── branding
│   │   ├── img1.png
│   │   └── img2.png
│   └── dir1  (empty dir)

I expected dir1 and its contents to be in the output. To further clarify, I _do_ want gulp to copy the contents (vs copying the symlink) which it does correctly with img1.png, img2.png, just not with subdirs or their contents img3.png, img4.png.

Is this expected behavior? Bug?
I don't feel like it's a fault of my glob string, since I can use /**/*.{jpg,png} for any number of subdirs of any depth that aren't symlinked.

[k-funk]

Anything I can do to push this along? I'm new to node, but willing to help.

[mabar]

There is an option follow, which can help you.
gulp.src('..../images/**/*', { follow: true })

But be aware of cyclic links. That kind of mistake maybe cause delete of all files.

Optionally, you can use workaround too

var vfs = require('vinyl-fs');

gulp.task('myTask', [], function() {
  vfs.src'..../images/**/*')
  .pipe(vfs.dest('dist/images'))
)};

[phated]

The option mentioned by @MasterOfZozo is actually followSymlinks but it is defaulted to true already. Please provide a reproduction repository for people to debug against.

[k-funk]

@phated https://github.com/k-funk/gulp-symlink-subdir-bug

I included followSymlinks: true so that @MasterOfZozo could see it happen.

[k-funk]

I think that the bug is a naming between glob-stream, and glob. followSymLinks is being passed as an option to glob.Glob, instead of follow here: https://github.com/gulpjs/glob-stream/blob/master/index.js#L28

If I add ourOpt.follow = true; right before calling var globber = new glob.Glob(ourGlob, ourOpt);, all is well. Could you ensure that this option gets passed correctly ASAP?

For future minor/major releases:
I'm not entirely sure why one-level-deep of files within a symlink were read into the stream, but 2+ levels were not. My guess is that wrap-with-vinyl-file.js is handling more than is necessary... but I could be wrong.

[phated]

@k-funk we pass all options through to node-glob; I think that's what @MasterOfZozo was talking about with the { follow: true } option. Our followSymlinks option actually resolves symlinks to non-symlink files, which is separate than the follow option you want to use with node-glob. It seems to make sense to keep this behavior.

[k-funk]

Not quite understanding the use-case/benefits of having followSymlinks be true, and follow being false. Seems to me that if someone wants to follow symlinks to non-symlinked files, they'd want to include subdirectories and their contents as well. In addition, gulp.src('path', {follow: true, followSymlinks: true}) feels a bit awkward to me.

[phated]

followSymlinks turns symbolic links into hard files (reads the contents of the linked file and creates duplicate files in the output, etc). It might be a poor name for what it actually does, but that's what we have right now. follow does something different, in which it follows symlinks throughout a tree, and isn't enabled by default in node-glob because it breaks on circular links. Those 2 behaviors are mutually exclusive so we can't force follow on if followSymlinks is set to true.

It's a frustrating state of naming but imagine followSymlinks as convertToFile and you'd have { follow: true, convertToFile: true } (notice how they are very different with better naming).

I'm going to close this and chalk it up to a documentation problem (which have multiple issues open), but I'll open an issue on vinyl-fs to revisit the naming of the followSymlinks option. Thanks for bringing this up!