Three.js: NodeMaterial visual editor

[pailhead]

The discussion in https://github.com/mrdoob/three.js/issues/7522 became convoluted, with several different topics overlapping.

One topic is:

NodeMaterial is verbose when coding, i can't make heads or tails with it, especially when compared to just writing GLSL.

To which the answer is mostly:

You're not intended to code, there will be a visual editor.

I actually didn't understand if @sunag's visual editor will be open sourced and will be part of three.js.

I'd like to discuss the visual editor component to NodeMaterial. When, how, why, etc. :)

For example:

I saw this editor https://shaderfrog.com/ mentioned in #7522, how does the three.js community decide which closed/open source tool it should depend on? What's the significance of SEA3D in this context and how does it relate to three.js?

[donmccurdy]

NodeMaterial is verbose when coding, i can't make heads or tails with it, especially when compared to just writing GLSL.

You're not intended to code, there will be a visual editor.

I would disagree with those assumptions, actually:

1. If NodeMaterial went into src/, we should keep backward-compatible MeshFooMaterial classes that construct appropriate node materials for users. If users are doing things three.js already supports, they may as well keep using those.
2. For a good subset of customizing existing three.js materials (e.g. per-vertex alpha, per-texture transforms, additional UV sets, texture swizzles, ...) writing NodeMaterial code is not hard. We should write docs, of course, but these features will be fairly easy to accomplish in code.
3. For more custom shaders, or users who simply have a particular workflow preference, using a visual editor or writing a custom ShaderMaterial in a code editor are both viable options.

I think that the visual editor will help those who aren't comfortable writing shaders, and some artists and developers coming from other workflows may just prefer it. But I certainly do not think coding with NodeMaterial is by any means impractical.

About the particular visual editor — I think the examples given in #7522 are just examples. We'll want to build something specifically for three.js, based on existing work if possible...

[pailhead]

Thanks for the input

1. will this take away the ability for me to use onBeforeCompile as i was using it now? Ie. will my app that has onBeforeCompile break?
2. no visual editor involved? Devs would write GLSL that would have to be wrapped with nodes?

What i've heard just now is that the visual node editor will come after NodeMaterial ie. there will be a state of the world where three.js users are (somewhat?) compelled to write code using various Node classes, without a visual editor?

If so, do you know of any framework that currently works like this (nodes, without a node flow visual editor)?

But I certainly do not think coding with NodeMaterial is by any means impractical.

To each their own :)

We'll want to build something specifically for three.js, based on existing work if possible...

I'd be curious to hear a guess on this time line? I could be cocky and ask 2020 :) but in all seriousness, it sounds like it will come after NodeMaterial becomes first class citizen?

[donmccurdy]

will this take away the ability for me to use onBeforeCompile as i was using it now?

No, or at least I don't think so.

no visual editor involved? Devs would write GLSL that would have to be wrapped with nodes?

you can continue to use ShaderMaterial without wrapping anything in nodes, if you prefer to write GLSL

What i've heard just now is that the visual node editor will come after NodeMaterial ... I'd be curious to hear a guess on this time line?

This is up for debate, I think.

[pailhead]

Well, let's start the debate :)

The timeline on this stuff is very long. If NodeMaterial makes it in tomorrow, waiting another 2.5 years for the editor would be too much.

I think NodeMaterial is not as useful for coders, and useless for non GLSL users since they still have to understand how shading works.

Should NodeMaterial be coupled with a visual editor? Why, why not?

[makc]

I saw this editor https://shaderfrog.com/ mentioned in #7522

emm, it does not seem to be node-based editor?

[pailhead]

I realized that when i started playing with it, some nodes exist but they're for overlaying different materials it seems, so definitely not the tool that's needed?

[makc]

Lemme bump this with http://victhorlopez.github.io/editor/

this is for litegl but I bet some code could be borrowed to bring this kind of function in 3js editor

[mrdoob]

@sunag used to have his editor live at http://sea3d.poonya.com/flow/
Seems like the page is no longer working...

[pailhead]

I think I need to figure out what sea3d is.

[quacrobat]

@makc that uses https://github.com/jagenjo/litegraph.js so maybe a good place to start for someone to implement a threejs version?

[pailhead]

That looks like an interesting resource I was trying to find a framework for making a generic graph editor.

[mrdoob]

@pailhead why was this closed?

[pailhead]

I think i actually misclicked :)

[pailhead]

@mrdoob any thoughts of how a tool built with a dependency like this would fit the big picture? I plan to dig into his some.

[makc]

@pailhead there is already loads of files in editor/js/libs so why not add another one? in for a penny - in for a pound.

[pailhead]

I had a brain fart. This wouldn’t have to be a dependency of three

[AndrewRayCode]

emm, it does not seem to be node-based editor?

I should make the UI more obvious ;) https://shaderfrog.com/app/view/2826 click "edit" on this shader

I personally use ShaderFrog to make custom Three shaders, but frankly it's difficult / impossible for other people to do the same, the system does not support it well and I have to download and manually hack the output to suit my needs.

ShaderFrog's node system doesn't use Three's NodeMaterial stuff (nor do I understand the nodematerial stuff) - it's meant to be something more general to glsl.

[looeee]

Here's an article from Babylon.js on the state development of their node material and visual editor. It's a bit light on details but you can read more details on their implementation and progress on GH and the babylon.js forum.

[trusktr]

Some visual node (or flow-based programming) tools for ideas:

1. http://idflood.github.io/ThreeNodes.js/index_optimized.html#example/geometry_and_material1.json - MIT
   
   
   
   • built with Three.js in mind
   
   
2. https://rete.js.org - MIT
3. https://www.totaljs.com/flow - MIT
4. https://codepen.io/xgundam05/pen/KjqJn - bare-bones node editor UI (implicitly MIT)
5. https://nodered.org - Apache 2.0
   
   
   
   • Three.js example
   
   

I like the first one because it is already built on Three. It doesn't have node materials yet, but that could be added.

[pailhead]

Dang, wish I could have seen some of these examples before starting this from scratch,

https://dusanbosnjak.com/test/nodes/

[wonglok]

Hi Guys, omg i've been dreaming about the APIs of node based material >v< just found this page from google. OMG im so blessed. i've been working on the proof of concept of something that works like a node based system for a few years.

https://age.wonglok.com/
https://effectnode.com/
https://igraph.effectnode.com/

Thank you so much for all your hard work guys >v

[wonglok]

I will definitely try to make a GUI + JSON + SDK for ThreeJS Node APIs for my personal fun fun usage >v

[quacrobat]

I will definitely try to make a GUI + JSON + SDK for ThreeJS Node APIs for my personal fun fun usage >v

did you do it??

[wonglok]

I will definitely try to make a GUI + JSON + SDK for ThreeJS Node APIs for my personal fun fun usage >v

did you do it??

i did it yayyyyyyyy
https://v2.effectnode.com/

[pailhead]

Nice! Is this using NodeMaterial under the hood?

[wonglok]

Yes :D

https://github.com/wonglok/effectnode2-gui

[quacrobat]

i did it yayyyyyyyy
https://v2.effectnode.com/

super cool!
but how do you use the result in code?

[makc]

@quacrobat apparently you could press "copy" button and then do this but with the copied json.

[quacrobat]

@quacrobat apparently you could press "copy" button and then do this but with the copied json.

Thanks, I saw that, but still couldn't figure it out, mainly because I'm not familiar with the Vue.js syntax...
Also, I think this only copies a JSON text that describes the nodes GUI, and is not parsed code that Three.js could read directly. Am I wrong?

I wish that "copy" button just copied code that looks like:

var params = { ... }
var nodeMaterial = new TREE.StandardNodeMaterial(params)

:)

[makc]

vue? no, that is makeMat function that you get from this 120 lines of code file and it should make a material useable in 3js directly

[quacrobat]

Aha! exactly what i was looking for. Thanks again @makc