Sp-dev-docs: Alternative for webpack & gulp

[leon-andria]

Category

• [x] Question
• [ ] Typo
• [ ] Bug
• [ ] Additional article idea

Do we have a documentation that explains the packaging workflow without using Webpack & Gulp ?
Ideally, to do it manually rather than relaying on the "black-box" process from the generated template from yo.

[iclanton]

Currently, no we don't have any documentation around that.

At a high level, you need to create a JavaScript bundle that conforms to the same template as what's produced using the OOB build system, so that's something that looks like this:

define('<componentId>_<version>', [<array of dependencies>], function (dependency1, dependency2, ...) {
  // Factory function that takes dependencies as arguments
});

And you need to create a manifest that also matches what's produced from the OOB build system. The manifest format is documented in @microsoft/sp-module-interfaces, and the schema is published to https://dev.office.com/json-schemas/spfx/client-side-web-part-manifest.schema.json

There are a lot of important details in the build, however. The way dependencies are resolved and excluded is particularly delicate, so I caution you to be careful. We also can't guarantee that what you build will work correctly if you're not using the supported tools.

[leon-andria]

Thank you for your answer but i'm confuse.
The format you are suggesting is more AMD with requirejs and what I see on the running sample with yo, bundles are done with CommonJS. I hope so...

The main reason I asking this is that, webpack & gulp are slow and not only that, you have to do a lot of encapsulations without counting of libs that are not maintaining on npm package.
To setup the dev env, it tooks like 5 minutes to downloads all dependencies and dev dependencies.

Choosing React looks not taking benefits of reactivity system. Every time, the render method is called. Sure, React is making diffing behind the scene but I don't see added value using it if we can't see real benefits, i think using other lib like VueJS will defeat the purpose and it's a shame.

I believe that there is a better experience dev flow without having all encapsulated defacto. I think having a well defined architecture of what should be exposed in facade should be good, because the community will contribute to use the best nodejs lib around there and not to wait any bugs or updates that are blocking.

This being said, I'm going to investigate on the overall workflow to define another dev workflow without webpack & gulp. So, if you guy can point me to a documentation or a high level architecture or poster, i'm interested so i can contribute to the community.

Thanks in advance

[octogonz]

@VesaJuvonen FYI

[leon-andria]

Hi guys,

As there is no answers, I started to investigate and reach out that there are package starting with @ms/xxx are missing... telemetry, a11y,...

The funny part is that if these missing package are resolved, we got a high chance that it's working just with:

let workbench = await import("xxxx");

Seriously, I can't believe that a big product like Sharepoint doesn't have support for the community, this client-side is an opportunity to change the game after 11 year starting with 2007, good by UserControls and ASP.NET Life Cycle. Does the Client-Side will do the same mistake like in the server-side again ? I really don't believe that and Sharepoint should not be reserved for elite.

[octogonz]

I believe that there is a better experience dev flow without having all encapsulated defacto.
I think having a well defined architecture of what should be exposed in facade should be
good, because the community will contribute to use the best nodejs lib around there and
not to wait any bugs or updates that are blocking.

If you have complex custom requirements, then you could certainly replace our tool chain with an alternative implementation, as long as you:

• Create bundles with the same format as our webpack bundles
• Create manifests with the same file format as our manifests
• Conform to the API contracts implied by the TypeScript language, and avoid accessing any unsupported or internal parts of our system

However, this is an advanced scenario. Our tool chain performs a lot of processing beyond simply invoking tsc and webpack, and a nontrivial amount of work will be involved to mimic it. I would not recommend to implement your own tool chain simply because your build times are a little slow. We use this tool chain to build 100+ first party projects internally, so it goes through a lot of testing and debugging. This is the main reason why its evolution is sometimes a little slow: We don't release a new version of the tools until all of our first party projects have been migrated and are building without errors.

Ideally, we would provide complete formal standards docs that precisely describe the tooling contracts (instead of merely "make sure your tool chain does the same things that ours does"). We are working towards this, for example we publish JSON schemas describing our manifest formats. However frankly our first priority is to improve the implementation of the standard tool chain that most developers are using. There's still a lot of work to do there.

To that end, we are very aware of the feedback that the way we invoke tools such as tsc and webpack is not very transparent. We've been investigating some architectural changes that would simplify this and eliminate some of the wrapper layers.

Choosing React looks not taking benefits of reactivity system. Every time, the render method
is called. Sure, React is making diffing behind the scene but I don't see added value using it
if we can't see real benefits, i think using other lib like VueJS will defeat the purpose and it's a shame.

You're free to use whatever rendering library you like. SPFx uses React internally, but our API contracts don't require you to use React. We wouldn't consider it "a shame" if you choose to use VueJS. You should use whatever library makes sense for your project. React is older but has a huge community. They obviously saw real benefits. :-)

As there is no answers, I started to investigate and reach out that there are package
starting with are missing... telemetry, a11y,...

These are internal package names. You would normally encounter these errors if you try to consume the *.js files from the "lib" folder of Microsoft's packages. That's not supported. You must use the bundles from our "dist" folder. The "lib" folder is for informational purposes only.

[leon-andria]

Thank you very much @pgonzal for your answers, really appreciate it.
Anyway, I spent 2 full days to debug step by step the all processes. I can tell you that I have a full overview of all dependencies. It's so scared but I think it's due to some legacy code and constraints, I don't have any explanation.
Just to give you a sample picture about lazy loading the _sp-webpart-workbench_:

I'm not using Webpack but FuseBox (http://fuse-box.org/)
This picture means a lot and no need to have a sentence to explain it.
Trust me, if you go to FuseBox, you will change your mind on your toolchain workflow. And I'm sure that lib folder is the result of Typescript (There is no doubt that MS don't use it internally :-)).
During my debug process, I found out that many code (95%) can be applied using lazying loading with code-splitting as the requirejs or systemjs try to do.
If you want to discuss more, drop me a message (leon[at]coolflow[dot]io), I think I can contribute to make it better in this eco-system.

HTH,

[octogonz]

This picture means a lot and no need to have a sentence to explain it.
Trust me, if you go to FuseBox, you will change your mind on your toolchain workflow.

Thanks for sharing this, @leon-andria . We took a look at FuseBox and found that it is very easy to configure. However, it doesn't have the same level of community support or advanced features as webpack, so it's not really appropriate for our tool chain scenario. Also @iclanton found that the output was very verbose compared to webpack's. We'll keep an eye on this project, however. It's evolving very rapidly and maybe these issues will be addressed in the future. Thanks again!

[leon-andria]

Hi @pgonzal,

Thank you for your answer. To be frankly, I found your answer so politely and telling the all the truth. Let me tell you exactly what it is, we are professionals on this fields and when I will said WE, it's about all professional people that invest on SharePoint and companies which using SharePoint.
The reality is that you guys, made a design of architecture that is very close, and not so much room for customization. Proof, you are talking about your tool chain scenario, but let me you, we don't care about your tool chain, let's us our own way of development experiences. here, SPfx forces us that it should like this and no way. We as a consumer, it's really frustrating because it doesn't simplify the development and doesn't encourage developers to use it. Trust me, it's not about criticizing your choice, it's about we as a IT decision markers, how we can convince our businesses if we feel that it's complicated and having the same cliche that development with SharePoint is always out of budgets and delays.
The tool chain you are providing is too complicated, trust, i debug all pieces of the workflow execution until that the webpart is displayed on the workbench. The reality is beyond your imagination, it's very slow, it's only one webpart, so think about more than one webpart.
I'm so frustrating about this and I think as a customer voice, I will think about to write an article on Medium to highlight my points. I think it's a good to share that to the SharePoint Community and there are a lot here in Switzerland, having MVP's involving on this will help.

best regards,
Léon

[octogonz]

@VesaJuvonen FYI

From the beginning, one of our design goals for SPFx was that Microsoft developers should use the same platform, APIs, and tools that we ship to third parties. This strategy would make our first-party developers more aware of the third-party experience. It would ensure that we're testing in depth.

You've pointed out an important downside of this strategy: Microsoft's own resources and requirements are often very different from third parties. We have large teams, elaborate pipelines, and internal onboarding for new developers. For someone who doesn't have all these requirements, our process may feel alien and overwhelming.

You're of course not the first person to voice this frustration. We are working to simplify the SPFx tooling, and to make it more flexible. However we also have a goal to provide a complete end-to-end solution for starting developers, and to offer support if something isn't working. If some people prefer to use e.g. Babel+FuseBox+Vue.js instead of TypeScript+Webpack+React, we should certainly make it easier to do that. But we can't support it in the same way. We simply don't have enough resources to invest in every imaginable configuration. The gap between meeting one person's needs versus a reliable end-to-end solution is very expensive. As an example, we recently tried swapping out Jest for Karma. On the surface, yes, all we had to do was replace one Gulp task with another -- but it took months of effort to arrive at something that was actually supportable for the internal teams who use it.

In short, we do want to give you want you are asking for. The reality is that it's more work than it seems. It will unavoidably require some compromises and constraints.

Thanks again for sharing your experience. I'll look forward to reading your article. :-)

[leon-andria]

Hi @pgonzal, @VesaJuvonen

I really appreciate your answer because of it's honesty.
If you guys want to talk, i'm open. I know exactly how to tackle this after a week of demystification about this topic. Seriously, i'm not joking, i can draw you a picture or diagram of what is needed and not to make it compatible with existing run-time. I know that you have a big challenge from SharePoint Online requirements but this doesn't mean that you enforce your tool chain to us.
Where I failed by using fusebox is that there are some dependencies are hidden, it's like cryptographic. And only your process with webpack resolves the sourcemap of these hidden deps, so i can understand all the tricks and custom development on webpack, i check all of them, it's over engineer, most of them are not needed and i can understand why you said webpack has advance features (that's is not true, it's not about manipulating AST). What you did guys, it's a salad with Gulp & Webpack, so i'm not surprise that it requires time and efforts to sustain that (should never exist by the way).
Let me help you if you want, the process should be follow:

1. Skeleton for the workbench
2. SPWebPartManager, the way you load the manifest and activate the webpart, here you play with requirejs or systemjs, i saw the condition doing, you can add a custom implementation to support other stuff, so we can use dynamic import.

All the code i had walk-trough, i'm sure that we can remove 40%. I'm not joking but it's the reality when you debug step by step, i think none of you guy goes deep like this. But, i did it, because of passion and it's my business for promoting SharePoint my customers.

So, any time to talk if you want.

[msft-github-bot]

Issues that have been closed & had no follow-up activity for at least 7 days are automatically locked. Please refer to our wiki for more details, including how to remediate this action if you feel this was done prematurely or in error: Issue List: Our approach to locked issues