Sp-dev-docs: Integration of custom gulp tasks - formally build tasks

[StfBauer]

I did a quick check on the gulp tasks. From my point of view it is pretty much locked down and supports only some few options.

Let's say I have I like to use precompiled Handlebars during development I'm unable to extend the functionalities, because I don't have direct access to the gulp file.

My guess right now is that extensions like these are currently not supported. In future it will be great to have somehow an interface to inject custom build tasks in the overall build process.

Currently the only way is to use additional libraries via CDN.

[patmill]

Are you looking for customized tasks, or just improvements in how bundling libraries should work?

We are planning on doing a bunch of work documenting the build pipeline. It is a plug-able system. Adding in @nickpape-msft and @iclanton - who have done most of the work in this area.

[StfBauer]

I'm looking more into customized tasks. Let's stick with the handlebars example. I like to define a watch that compiles the HTML to a JavaScript add it, let's say in the Web Part Folder.
For the rest the normal bundelling should work.

Sure in additon those libraries should be bundled too.

So it's a mixture of both.

[patmill]

While it is possible to create your own gulp tasks, we don't have a way to inject steps into the build in tasks (ie, as part of gulp serve, do a, then build, then b).
We have some ideas on how to accomplish this, possibly by having specific phases that you can plug in to. In all honesty though, it will likely be a while before we get to that step.

[StfBauer]

Meanwhile probably use another side project and gulp the compiled JavaScript source in. Just a workaround but should work.

[iclanton]

Are you specifically trying to use *.hbs files in your bundle, and have them compile to JS as part of your build?

The easiest way to accomplish this is with a Webpack loader. There are a few existing Handlebars loaders. This one appears to be the most popular.

To use this approach to loading a *.hbs template, you need to do a couple of things:

• Ensure *.hbs files are copied to the lib folder. To do this, you need to configure the copy-static-assets build task to copy *.hbs files. This is pretty simple - just add a copy-static-assets.json file to the config directory with the following contents:

{
  "includeExtensions": [
    "hbs"
  ]
}

• Install the loader. Just run: npm install handlebars-loader --save-dev.
• Reference your templates. This is all you need to do to reference a template:

const template = require('handlebars!./path/to/template.hbs');

To understand what's going on in this line, you need a little bit of background on how Webpack loaders work. Here's an article that goes into depth on the subject. Here's the TL;DR version - In a require statement, everything before the ! describes which loader Webpack should use to load the file. We predefine loaders to match certain extensions in our OOB config (like *.JS, *.PNG, *.JSON, *.CSS, and several others - this is something you'll be able to add to the Webpack configuration soonish. Keep an eye out for documentation), but you can specify the loader directly in the require call. Webpack allows certain shorthands for loader names (the most popular one is to call your loader <something>-loader) so webpack will take the name you specify (in our require('handlebars!./path/to/template.hbs') example, the loader name is handlebars) and look for modules with names that match a number of these patterns. It'll find the handlebars-loader and use that to load your template.

I realized that explanation rambled on for a bit, but hopefully it made sense.

Does that solve your problem?

[vvolodin]

So far the existing gulp build poses more problems than it solves.
It's all somewhere deep in node_modules, so in order to modify something I need to dig through 10 or 15 modules used during the build.
It would be much more useful to have a large gulp file which one could comprehend.
Otherwise it's not too different from what we had before with Visual Studio, it does everything for you but if you want to customize something - you're on your own.

[waldekmastykarz]

The problem with modifying any of the existing files is that your changes will be overwritten should the tools update in the future. Also, given that node_modules is not stored in source control it would be hard to share the changes with a team.
It would be helpful to be able to plug into the build pipeline to include additional operations.

[wictorwilen]

@vvolodin I don't agree, this way (and once the source for the generator is available) you have more customization options. But just as @waldekmastykarz says - DO NOT overwrite or modify any default files, and do not even consider fiddling with the files in the node_modules directory. If you want to add a change to one of those modules, then get the correct module repo on github, propose a change (as a PR?) and wait for it to be correctly handled.

[StfBauer]

Well there are two points here to sum up.
@wictorwilen and @waldekmastykarz were at the dev kitchen and had early access to the SPFx. I guess this project had a proper gulp file that could be edited and extended with additional gulp tasks. Correct me if I'm wrong. :)

What @vvolodin is referring to that the current developer preview of SPFx that calls the build process directly from the node_modules. In order to inject something, the node_modules needs to be modified, which is never an option because the things in the node_modules shouldn't touched.

No offend here but but from my point of view the current state of the preview is a ready built visual stuido project generator based on nodejs and a pseudo yeoman generator that provision some files and install node_modules.
From my perspective it's no big deal right now because most people that currently use the generator are not that much into nodejs. It also gives a great insight on how we are able to customise build SharePoint Web Parts in future.

Overall the whole setup is a quite unusual for a yeoman generator. Normally you have a single gulp file that is editable and you are allowed to add custom build task to the already existing ones.

As Pat Mill mentioned on my inital comment I'm keen to see how this injection of custom gulp tasks will work in future. Right now lets get ourself familiar with the new kind of web parts and the work bench.

Send PR to a github repository to get your custom task in will never be a real useful option.

[waldekmastykarz]

Totally with you on it @StfBauer with one little difference: to me it doesn't matter if I get to change SPFx standard build tasks or not as long as I can have the build process do everything I need to do. I can imagine, from the support point of view, it's easier to encapsulate standard build tasks to ensure that they are working as intended. To make it flexible, providing extension points that we could use would be sufficient for me. For advanced scenarios and opinionated developers, no matter which approach you choose, it won't be enough and you will end up building your complete pipeline yourself.

I agree that I haven't seen another project using Gulp the way its used in SPFx but it doesn't necessarily mean it's wrong. I'd love to hear the reasoning behind it. @chakkaradeep, @patmill care to chime in? 😄

[vvolodin]

@StfBauer thanks for expanding on my comment.
I don't mean to be overly critical but it just doesn't feel like I'm in control over my project and simple tasks take a lot of digging around transpiled typescript files to do. How to disable tslink, how to opt out of css-modules, how is webpack configured, how exactly my app part files are loaded by the module loader, can I use react to build my settings panel. I realize of course that the project is at the early stage but this actually is a better time to make the build system more flexible in the future.

[waldekmastykarz]

@vvolodin I guess it's twofold: on one hand there wouldn't be any harm in Microsoft exposing more of the build process and mechanics of the build process to us (like Webpack configuration). On the other hand if some of these tasks correspond to how the SharePoint Framework works in SharePoint (like loading modules) then it wouldn't make any sense in changing any of it in the dev tools, given that it wouldn't work after deploying your Web Part to SharePoint which would still use the original logic.

[vvolodin]

Oh, I just meant it would be useful to have some docs/understanding on how the stuff we're supposed to build is getting loaded. So far it's a bit too much of a black box thing - you just prepare your inputs, build it and observe the artifacts like manifest.json, stats.json, etc.
I like where this is going - local workbench, modules, webpack, live reload, but I'd like to have a possibility to build it without the generator as well.

[waldekmastykarz]

Documenting the manifest and the different pieces would definitely be beneficial.

[StfBauer]

Injection of custom task currently works without any problem in one of my samples. Will write a blog post about it.

[LoungeFlyZ]

As someone who has lived in NodeJS for the last 9 months I just wanted to throw it out there that the way the current build system works is very _non-NodeJSy_ and I think anyone coming at this from a non-VS world will look at it and be like "wha?!"

Pros: It does it's job and its simple.
Cons: AFAIK no one else (in Node at least) does things this way and it will cause confusion.

My main concern is with how the gulp build pipe is all hidden away and for all intents and purposes not designed to be changed/modified (or at least that is how it looked to me). This is a problem for some of the reasons mentioned above. If you want to add a much of pre-steps that integrate into the gulp build pipeline like adding source maps for example, or changing the TSC options etc... its not obvious how/where to do that.

Personally, I think if you have to document how your gulp build pipeline works and how to change it then there is something wrong. (no offense)

But that aside, I think the biggest issue is how people will view this when first looking. They will be expecting to open the gulp file and see how the project is built and be able to follow it easily.

Currently it looks like the approach has been like you would see in Visual Studio by hiding all the complexity, which isn't the norm in node projects. Like i said, people expect to edit their build pipelines and add things they do as standard etc...

I personally think the current gulp build process could do with a design review with someone from the Node world who knows their stuff and would be able to consult on how to make it more approachable for that community. My 2c.

[StfBauer]

Problem solved by just using native gulp behaviour.

Before gulp executes all registered tasks will be evaluated and brought in the right order.
This fact allows to register task to any of the existing tasks.

I think it is suitable for many simple scenarious that precompiles files and puts them during a development process to a temp folder. In case fo SPFX it just put is to the web part folder.

Does it fell like a hack? Yes it does because of the missing gulp task in the gulp file.
On the other hand thats the way gulp actually is intended to work.

Wrote a more in depth blog post about it.
Use a custom gulp task in the new SharePoint Framework

[tomresing]

I won't pretend I understand this, but I'm glad smart people are working on it. Maybe by the time I get around to finishing the "Hello World" walkthrough you'll have solved all the hard problems! :)

[patmill]

Closing this down now.

[vvolodin]

Is it possible to integrate custom build tasks now?

[patmill]

See here - https://github.com/Microsoft/web-build-tools/tree/master/core-build/gulp-core-build#defining-a-custom-task

[msft-github-bot]

Issues that have been closed & had no follow-up activity for at least 7 days are automatically locked. Please refer to our wiki for more details, including how to remediate this action if you feel this was done prematurely or in error: Issue List: Our approach to locked issues