P5.js: Modularize source so parts can be used independently

[brysonian]

I've been working on a project that started in p5 and has now had to migrate to three. I've found that I miss a few of the handy bits of p5, pImage, random, etc and have torn those bits out to use in this project. One issue is that the project is build on AMD and I've got a solid working relationship with browserify. If every module were UMDified however, it would be possible to just require the bits you wanted in either build system. Does anyone have any thoughts/objections to me doing this? It won't affect the current build process or anything.

[lmccart]

I think this sounds good, it is in line with an early dream we had of being able to make customizable builds with just the necessary modules. I'm no pro with the umd stuff though, so I don't know if this introduces any potential problems or complications? but if not, I'd say go for it.

[kevinbarabash]

I think switching to commonjs + browserify for the build system is a good idea. I'm not sure about UMDifying every module. This will introduce lots of boilerplate. If all the modules were in commonjs you'd still be able to pull in the parts you want browserify those.

[brysonian]

it will introduce a lot of boilerplate, that's for sure, though all of that would be stripped out in the build process, and only there for the users who want to do this kind of per-module use.

Moving everything to commonjs sounds great to me and i generally find managing large projects far easier using commonjs, but then we'd be in the same boat for people who want this use case and are using AMD. Though it would more cleanly allow folks on node/io to access the modules for backend apps if the project was commonjs-first.

Another route might be a grunt task that exports all modules in a user-defined format, or just wraps the current source with UMD boilerplate. Then folks who care could do something like grunt export umd -o someplace/somewhere

[kevinbarabash]

I'm no pro when it comes to UMD, what part of the build process strips out the boilerplate?

I wonder if there are any people who are using AMD directly from the source?

I'd rather have a grunt task that exports modules to a certain format as use suggest and then use commonjs in the source to keep things clean.

[brysonian]

Right now the requirejs task, with some help from amdclean, generate a single UMD build of the entire library and strips out the defines etc in each module.

The bigger issue i think is that most of the modules add their code to the p5.prototype which might make it a bit of a bear to track dependencies between modules, and add more complexity to the boilerplate to make sure everything ends up where it is supposed to go.

[Spaxe]

@brysonian Could you outline the steps to do commonjs + browserfy for p5js?

[benhinchley]

making them all proper commonjs modules would be awesome, on another point would moving the codebase from straight js to typescript be beneficial ??

[kadamwhite]

(De-lurking)

I would strongly caution against moving off of straight JavaScript. The only potential "compilation required" option I feel might be viable would be ES6, as that is a standard that will be implemented natively in browsers; (coffee|type)script limits potential contributions.

[Spaxe]

I support @kadamwhite in that p5js should be standalone and have no other JS dependencies.

My comment comes from a practical point of view.

Currently, p5js checks for window.draw and window.setup, making embedding use on your every day web design difficult, because it goes against the convention for diluting global scope.

My question is, without changing how p5js behaves and its no-dependency setup, is there a way to include p5js in websites with scope protection?

[brysonian]

@Spaxe, here is a simple example with browserify.

var p5 = require("p5");
var myp5 = new p5(function( sketch ) {
  sketch.setup = function() {
    var cnv = sketch.createCanvas(500, 500);
    sketch.colorMode(sketch.HSB, 360, 100, 100);
    sketch.noStroke();
  };

  sketch.draw = function() {
   var h = sketch.frameCount % 360;
   sketch.fill(h, 100, 100);
   sketch.rect(0, 0, sketch.width, sketch.height);
  };
}, 'sketchdiv');

Which also keeps your global scope clean.

I agree with @kadamwhite that sticking with pure JS is the right move for the project. I can see some merit in moving to es6 (via babel). The project is currently organized using AMD, and many prefer CommonJS but there is a case to be made that since es6 modules are the future of JS it would make sense to go that route.

[kadamwhite]

The major pro for AMD is that it does permit development and testing in the browser with the unbuilt code. I haven't worked with the P5 codebase in some time, but I remember being pleased that was possible. Source maps and build processes with a watch task definitely make that a minor benefit, though: I'd opt for whatever module system is going to be the most stable for the longest period of time. In that context, I do think ES6 and CommonJS do offer some benefits over AMD in terms of ease of use as a developer.

Regarding ES6, however, I'd caution against adopting too much of the new syntax _yet_ even if we do put in a babel build. It'll be a while before ES6 is common knowledge and I think a transition/rewrite of what we have would be more distracting than helpful before 1.0.

[brysonian]

yeah i fully agree; the babel rabbit hole could be a huge distraction. And there is no reason that as-is people can't use the library in babel projects on their own. I've done it via with browserify and the babelify transform and it works great.

[Spaxe]

Thanks @brysonian, that's exactly what I needed to get me started.

ES6 is fully backwards compatible with ES5 anyhow. My personal opinion is that if it doesn't benefit the API / design principles / other use cases, it's not top priority.

[brysonian]

Been thinking more about this and also using babel more and more to support es6. I'm thinking that if we make a change, that moving towards es6 modules and @kadamwhite suggested makes the most sense. We would lose the unbuilt-in-browser-development which is nice now, but we would potentially gain quite a bit in terms of flexibility with each module. If there is interest, I can start working on it.

[workergnome]

We had a long discussion about this at the conference this week—here's what we decided about _publishing_ as UMD.

For the maximum utility to users of different backgrounds (beginning Processing user to accomplished Web Developer), The library itself should be published as:

• a UMD module
  
  
  
  • To be used with AMD
  
  
  • To be used with Requirejs
  
  
  • To be used as a drop in file
  
  
• It should be published on Bower (lower)
• It should be published on NPM (higher)

All of this is orthogonal to how we _develop_ p5.js: We could use browserify, amdclean, e6modules, or just concat-ing files to handle our modularization.

We should reevaluate the module structure of p5 so that parts of it can be used independently. To do so, we have reorganized the code into functional directories, so that a future project could potentially convert these into modules. We'd have to figure out how to do something like dependency injection to keep the current API.

There was a lot of support for converting to CommonJS modules, but nobody has embraced actually _doing_ that project yet. The general feeling was that we're not ready for E6 support.

There was also some discussion about just moving to a concatenation strategy for p5.js—we're not currently using modules much, and everything gets appended into a single object anyway, so are we making things complicated just for the sake of doing things "right"?

[kadamwhite]

Thanks for the summary! My two cents are that modules give us three major advantages over concatenation:

• explicit dependency tree. Module systems have a more formalized way of describing dependencies, which makes it easier to reason about application architecture than with concatenation.
• simpler testing infrastructure. Having code modularized gives you a consistent way to reference the code under test from within the test files without having to accommodate any globals or unrelated code.
• flexibility. If we get a module system in place that we're all happy* with (*: assuming here that the current system does not fulfill that criterion) then we have more structure for any future changes. E.g., if we decided to move some of the now-consolidated code back out into separate files, there's a very specific structure for doing so

Tl;dr : I'm in favor of modules and interested in helping out so we can get to a satisfactory system!

[workergnome]

Personally, I think that doing this with CommonJS and Browserify would be the way to proceed. But I'm not an expert on this, and I'm _not_ volunteering to do it! :grin:

[kadamwhite]

Dredging this one up for a poll: I'm interested in taking p5 and seeing how big an effort it would be to rebuild it more or less in-situ using browserify. Anybody object to me making that investigation, before I begin? (prob will start this weekend/next week)

[brysonian]

I'm into it. Are you going to go for commonjs modules and straight browserify or es2015 with browserify+babel?

[workergnome]

I think it's definitely worth an investigation.

would recommend commonjs & straight browserify—I think that es2015 is still a bit too new, at least for the developer-friendly codebase we're trying to create. It would be cool, but if I remember @kadamwhite's comments from last month, the commonjs conversion is 80% of the work, anyway, and it would be a reasonably straightforward conversion to go to es2105 in the future if we decide to do so..

[lmccart]

sounds great, go for it!

[kadamwhite]

Awesome! Yes, I'm planning to try straight CommonJS; Babel is awesome but I don't want us to get distracted :)

[dhowe]

+1

[benhinchley]

:+1:

[lmccart]

I believe this one has been resolved now with @kadamwhite's PR converting to browserify (https://github.com/processing/p5.js/pull/776), is this correct?

[kadamwhite]

@lmccart Not yet. The next step is to make each component module return itself, and let core aggregate those together; currently every "leaf" module has a dependency on the "core" trunk. It's probably 1-2 days of work, and I'm up to tackle it, but I won't have bandwidth until a few weeks from now.

[lmccart]

ah got it, thanks for the update. if you're down to work on this one in a few weeks, that'd be great!

[kadamwhite]

@lmccart Is this still something you want to do? I can work on a PoC for e.g. color or some other individual component, so that the team can see how it would play out in practice; then we could move forward if the approach looks good.

[lmccart]

that'd be great!

[sakshamsaxena]

@kadamwhite I can pursue this on similar lines to 43274cb as above. I think it does look spelendid. We'll have a very clean and organised prototype of p5 :+1:

[lmccart]

I think we can close this one now 🎉 and continue work on the interface over on this issue thread: https://github.com/processing/p5.js-website/issues/138

[sakshamsaxena]

I thought we'd close #94 for this (?). This was more about the source rather than the build, I think (?)

[lmccart]

@sakshamsaxena oops you're right, I got the issues confused :) I think we leave the other one open until the website part is ready though

[lmccart]

I think our current solution for custom builds addresses this to some extent. There hasn't been much energy pushing for more in this direction in the past couple years, so I'm going to close this for now. We can reopen if this becomes a bigger priority in the future.