Mvc: Problems with bower grunt gulp workflow on MVC5 (or less) and MVC6

[Bartmax]

I'm having some problems to work with a bower-gulp workflow with asp.net
I encourage you to create a new project and do a complete gulp/grunt workflow where you have dev files while debuggin, build files on build and use some of the common workflow utils like useref and wiredep. You know, source files on dev, build files on prod with bumping, min, auto update .cshtml references and publishing the required bits.

I will enumerate some problems (got way more)

MVC5 only:
Dynamic created files are not included in csproj so they won't get published.

This is also a problem for simple copy from bower components to folders like Fonts because MVC5 doesn't have a watch on folders so you need to keep refreshing solution explorer and adding files to solution and like previous point, if you fail to do that, they wont get published. And of course errors when you perform a simple clean folder task because of the now missing files.

MVC (all versions) :
tools like useref (usefull for concat-min on release) doesn't recognize "~" character so you will need to avoid them using only "/" which will give you problems if you are debug on local IIS (because you end up with localhost/appname/ .. or you must have to do some complex configurations on hosts files and ports on IIS not easily shared via source control.

so if you avoid using ~ character, you can wiredep and useref, but actually doing it on layout is a pain. MVC doesn't work with .html template pages, and this tools are not built to work on .cshtml,you can do 'make it work' with a lot of imagination and custom configurations

wiredep directly on App_Start/Bundles.cs (I know it's not intended to be used like this) is currently the only valid approach but it's kinda lame, and you end up using a mix of tasks and optimization library.

MVC6 only:
wiredep on wwwroot, you need to configure a lot to get some decent behavior (and it looks like cheating anyways) or you end up publishing your whole bower_components

Task Runners on visual studio are great for SPA. :+1:

On MVC6 needs some big improvements (like I said, forget about wiredep or you end up with really complex customization or publishing all your bower_components) and, it is unusable on MVC5 projects. :-1:

maybe I'm missing some key points, actually I hope so, on how it's the workflow supposed to be, but it ain't looking good so far.

If the solution is to don't wiredep, useref, (min, uglify only on prod), then what's the point of using a task runner in the first place?

note: don't take this in a rush, I did a lot of research/learning and worked on it for more than 2 days already. This is not a couldn't get done in 5 minutes rant.

[davidfowl]

I'm trying to extract something concrete from the listed pain points for Mvc 6:

• wiredep and useref don't work well with razor files because of ~

Is that the biggest problem?

/cc @madskristensen @sayedihashimi @DamianEdwards

[Bartmax]

I will list what I think are the biggest problems that I found for MVC6 in order starting with the worst one:

• useref don't work because of ~.
• There's no clear/clean way to use processed .cshtml build files. (see below)
• wiredep and inject need customization to generate ~ in paths.
• wwwroot cannot access bower_components unless you save them inside wwwroot. (see below)

More info on build .cshtml files:
Altering _Layout.cshtml (or any .cshtml file) with useref needs a new file output (right now required to be inside View folder) with a coditional on which one to choose. ViewStart example :

 if (!HttpContext.Current.IsDebuggingEnabled)
    {
        Layout = "~/Views/Shared/build/_Layout.cshtml";
    }
    else
    {
        Layout = "~/Views/Shared/_Layout.cshtml";
    }

This can get really complicated if you have multiple layout called within views, because you will need that logic for all views with it's Layout variable defined.

bower_components folder not accessible while debugging:
You can save them inside wwwroot or make your packages available inside wwwroot somehow (copy task) referencing the shadow copy while developing, this increase workflow complexity and those files end up being published unless you handle smart clean scenarios (increasing workflow complexity even more)

[Eilon]

I'm not super familiar with these components, but CSHTML is its own file format, so if a tool for processing just HTML is run on it, I wouldn't be surprised if it mangles the format. It's like running a JavaScript lint or minify tool on a TypeScript file. They're _kinda_ similar, but not at all the same. Such a tool would no doubt ruin the TypeScript file. Or is this some other kind of problem?

[Bartmax]

TL;DR:
It's not about if those tools should work with .cshtml or not, it's about having a nice experience on doing the minimal required client-build steps from dev to publish.

@Eilon It's not a surprise that these tools doesn't align well with .cshtml file. That's not really a concern here, the more important issue is that when you do web development, the client-side workflow of minifing, bundling, cache busting, compiling (.less, .sass, etc) must be taken into consideration on every major web development framework. ASP.NET had it's own workflow on Microsoft.AspNet.Web.Optimizations package, which support bundling and mostly all described above. For MVC6 they did a huge change and dropped support for the currently implemented workflow in favor to use already community existing workflow that worked great on multiple other languages with a very mature tools and way more extensibility, customization, and possibilities. This is a GOOD thing and a huge step forward for ASP.NET.

The current implementation supports Bower with Grunt or Gulp, which are an unopinionated package manager that works closely to a Task Runner for automation of the required tasks from dev to publish (some mentioned above).

The minimun that must be supported for a framework to be 'developing ready' for website is to support this kind of workflow:

Get and Update client side packages (DONE!)
Integrate those packages into the final html files respecting dependency order somehow (either manually or automatically)
Ability to use the development files on localhost while developing (KINDA)
Bundle (concat multiple files into 1 file) (DONE)
Minification (make files smaller by removing whitespace, comments, etc) (DONE)
Reference post-processed files on non-debug environment settings (NOT WORKING)
Reference cdn files with fallback (DONE)
Publish the bare minimun required files (NEEDS IMPROVEMENT)

Also, as a Framework, this process (used on every single web application project) should be straight forward, if this gets too complex, some web developers will find this frustrating.

Previous version of MVC created a great workflow for this minimun scenario but the idea is to drop it in favor of bower / grunt or gulp. Currently there are already people arguing if this was a step forward because right now is way more complex that it used to be before, and when you need just the minimum, configuring all the workflow is a lot of work you weren't usually aware it exists.

I think ASP.NET need a concrete and smooth way to cover what they did before with the Optimization framework. Maybe create it's own NPM modules, extend existing ones, or redesign something, I'm not sure. I just wanted to flag an alert sooner than later.

[davidfowl]

/cc @DamianEdwards

[ctolkien]

Do we still need uncompressed/uncombined files for local development? Can just use sourcemaps which simplifies the entire process (no need to rewrite any template files).

[BrianVallelunga]

I'm with @Bartmax here. I've been researching this for a few days and am trying to find a way to move forward using Gulp. I'm mostly hung up on two of the issues he points out: 1) How do I reference my bower components during development if they're not in wwwroot. and 2) When/how do I modify my views for optimization? Obviously I don't want to accidentally use Gulp and useref on my Views in development and then have to undo the changes before I check-in code, but I don't see much of an alternative. Some guidance would be nice on these two subjects.

The best plan of action I've come up with is to advise people keep wwwroot entirely free and not checked in to source control. Then have Grunt or Gulp populate it with a before or after build task. This ensures that scripts and images don't get modified by the tasks and then checked into source control accidentally.

[Bartmax]

I was playing around with brunch.io instead of grunt/gulp and looks promising!
Just playing around for 20 minutes, but got app.{css,js} and libs.{css,js} from their source folders (css,less,js,etc) to their corresponding build folders with sorcemaps as @ctolkien suggested and looks more likely what a developer expects.

While still have no access to bower_components, it's build process makes it not that much needed. This strategy can also work with gulp (i guess)

Still need to test some more complex/advanced stuff.
No html processing needed because brunch uses different builds on same files. (not exactly the same, but I'm ok with it for now)

I'm not sure how a publish from Visual Studio (or build for that matter, depending on configuration {release|debug}) could trigger a brunch build --production, that would be awesome.

[ctolkien]

This strategy can also work with gulp (i guess)

Yes that's what we're using for gulp. Here's a sample:

gulp.task('less', function() {
    gulp.src('./css/**/main.less')
        .pipe(plumber({ errorHandler: onError    }))
        .pipe(sourcemaps.init())
        .pipe(less({compress: true}))
        .pipe(sourcemaps.write('maps'))
        .pipe(gulp.dest('./wwwroot/css/'));

We don't ever care about uncompressed "development" assets. Skips the whole HTML rewrite step entirely.

[BrianVallelunga]

My current in-progress setup is:

1. Keep wwwroot out of source control.
2. Create a Gulp task called "Build" that does the following:
   
   
   
   1. Clean wwwroot
   
   
   2. Use bower-main-files with Gulp to have all bower dependencies copied to wwwroot
   
   
   3. Write other simple Gulp tasks to copy images, and process LESS/SASS, putting the files (unminified/unbundled) into wwwroot
   
   
3. Have my CSHTML reference files in their final wwwroot development location. For example: /assets/css/reset.css

The above should work great for development. For production/deployment, my plan is to run the same Build task, followed by an Optimize task that modifies my views and wwwroot files in-place, replacing where needed. I'm still working on this one though.

---

Update

I've given up on the above process since it is too difficult to modify the views and to combine assets already in the wwwroot folder. Instead, I'm combining and minifying with source maps in development like others here.

With regard to the Bower libraries, using gulp-main-bower-files plus gulp-filter worked well for me. Here's what I'm doing now for Bower libraries:

gulp.task("lib-scripts", function () {
    log("Copying library scripts");

    var scriptFilter = filter("**/*.js");

    return gulp
        .src(mainBowerFiles())
        .pipe(scriptFilter)
        .pipe(sourcemaps.init())
        .pipe(uglify())
        .pipe(concat('lib.js'))
        .pipe(sourcemaps.write('maps'))
        .pipe(gulp.dest(config.wwwRoot + "scripts"));
});

And for my css files:

gulp.task("app-styles", function () {
    log("Compiling SASS -> CSS and copying");

    var revManifest = gulp.src(config.temp + "rev-image-manifest.json");

    return gulp
        .src("./Assets/Styles/*.*")
        .pipe(plumber())
        .pipe(sass())
        .pipe(autoprefixer())
        .pipe(sourcemaps.init())
        .pipe(revReplace({ manifest: revManifest }))
        .pipe(concat("app.css"))
        .pipe(minifyCSS())
        .rev() // TODO: figure out how to point to this renamed file in the view
        .pipe(sourcemaps.write('maps'))
        .pipe(gulp.dest(config.wwwRoot + "styles"));
});

I still think MS has a major issue introducing this new workflow to existing MVC users. Instead of a couple of Nuget package and a few easy steps, I had to install 17 Gulp plugins to get the same results.

Actually, the results aren't entirely the same either. I'm still unable to version the whole concatenated and minified CSS file though. While I can use rev to rename the css file to app-HASH.css, I'm going to have to write a server-side helper to enable the view to point to the right file. This isn't a huge deal, but it doesn't come out of the box. _UPDATE:_ It looks like this feature is coming soon, which is great: https://github.com/aspnet/Mvc/issues/2141

On a positive note, I'm now able to version assets using gulp-rev and gulp-rev-replace. This adds hashes to the file names like background.png -> background-BC34C65.png. That's going to be very useful.

[timgit]

I experienced similar issues integrating gulp with ASP.NET MVC 5. As some previous posters mentioned, I created a folder called public, then a subfolder called build which was the output of all of my gulp build processes (both for dev and minified for prod). This build folder was subsequently ignored from source control.

The last step was to write a little HTML helper called FileTagger to support this integration so I wouldn't have to keep changing my cshtml files and such. It's great to see this capability being built into the script and link tag helpers in MVC 6!

[Bartmax]

@timgit I think the script and link tag helpers already handles those scenarios with the addition to the environment tag helper, and also provides cdn, busting and globbing patterns. I guess that everyone that tackles this ends in the same scenario, I think that means we can have an official workflow. We need to doc somewhere and we are set :D

[bastienJS]

@BrianVallelunga

I have not all answer down after your question but =>

"...1) How do I reference my bower components during development if they're not in wwwroot..."

A gulp task must copy all npm/bower dependencies to the wwwroot then you can reference them.

[rynowak]

This is about grunt/gulp and isn't super actionable for MVC.

@DamianEdwards is there any action item for the team here? Or close?

[DamianEdwards]

This issue was from when we shipped Gulp in our default templates. I don't think we intend to do anything specific regarding this anymore.

[rynowak]

Great, I'm closing this issue then.

If anyone from this thread thinks that your feedback on this (ancient) issue still has meaning now that we don't ship Gulp anymore, please open a new issue and CTRL+C CTRL+V.