Nodemcu-firmware: Unify documentation for Lua modules in lua_modules/ folder

[galjonsfigur]

Documentation for NodeMCU and it's modules is written very well and it's really helpful. However, for lua_modules there are only README.md files and sometimes examples for modules in separate .lua files.

The idea is to unify this documentation in docs subfolder (like /docs/en/lua_modules/) and rewrite .md files to work with ReadTheDocs + restructure them to look like documentation for C modules. This will be more friendly approach and will encourage to use these modules, especially after LFS patch.

I would happily do this and make PR, but first I want to know if this would be desired approach.

[devsaurus]

Sounds like a good idea to take the Lua modules to the next level :+1:

[nwf]

:+1:

@marcelstoer I wonder if it makes sense to make some of the Lua modules options in the cloud builder, either embedded in the initial SPIFFS or LFS images? This might take some refactoring of the modules, but putting them on equal footing to the C modules seems like a good plan?

[HHHartmann]

Definitely a good way to go now that there are c modules removed in favor to lua modules.
Maybe there could be two LFS slots. One for compiled in modules and one for LFS as we know it now. So lua modules would become first class citizen in the firmware.
There should be a common place to enable them in the sourcecode as well.

[marcelstoer]

I would happily do this and make PR, but first I want to know if this would be desired approach.

Great, I like your spirit. I'm all in favor of giving the Lua modules more visibility. Can you propose a new navigation tree branch you would hook into?

[galjonsfigur]

Now the navigation tree looks like this:

• Overview
• Basics
• ...
• Modules

Lua modules could be added as a separate tree and Modules tree could be renamed to C modules like this:

• Overview
• Basics
• ...
• Lua modules
  
  
  
  • ds18b20
  
  
  • ...
  
  
• C modules
  
  
  
  • adc
  
  
  • ...
  
  

or adding a subtree inside Modules tree like this:

• Overview
• Basics
• ...
• Modules
  
  
  
  • adc
  
  
  • ads1115
  
  
  • ...
  
  
  • Lua modules
    
    
    
    
    
    • ds18b20
    
    
    
    • ...
    
    
    
  
  
  • mcp4725
  
  
  • ...
  
  

I think the separate tree option is better but other ideas are welcome.

Also new page to FAQ about advantages and disadvantages of Lua and C modules can be added.

[HHHartmann]

I could imagine one list for all modules and maybe having a badge trying whether it is a c or lua module.

I also would like to structure the modules in sections as
Internal or core for node and internal hw related.
Software for sjson and others
Sensors/environmental
Sensors/motion
Sensors/ ...
But that's probably for a different task

[galjonsfigur]

@HHHartmann The problem with having those in one folder is name conflict between C Modules and Lua versions (for example ds18b20) so I'm not sure if that's the best approach - the label in module name would be needed and that's not the cleanest way to do it (I think). But now I'm just rewriting documentation so that can be decided later, before the PR.

Also there is one more thing to decide. Some modules are depreciated (like bmp085) - should the documentation mention or just ignore them?

And some things for other PR's

• some of examples in lua_modules folder are too complex to be used in module documentation (like ds18b20_web.lua) and should probably be moved to lua_examples folder.
• some guide to writing new Lua modules in more standardized way would be handy

[devsaurus]

One for compiled in modules and one for LFS as we know it now. So lua modules would become first class citizen in the firmware.

If Lua modules can become first class citizens, then I'd see no need to have their docs separated from C modules in the navigation tree structure. We could also have a common "How to" chapter for Lua modules and link to this from the respective docs.

Potential name clashes will then need to be solved by deciding which flavor we want to maintain and omit the other.

Some modules are depreciated (like bmp085) - should the documentation mention or just ignore them?

I'd say these should be skipped finally. We've added these notes back then to foward users to the C modules docs. There's no use for that anymore once Lua and C are documented in the same system.

[marcelstoer]

@HHHartmann

But that's probably for a different task

👍 I strongly suggest we digest the ideas here one at a time.

Lua modules could be added as a separate tree and "Modules" tree could be renamed to C modules like this:

Sounds reasonable to me: not much effort (apart from re-writing current READMEs), logical, little impact on the rest.

[HHHartmann]

My usecase of the documentation is that I want to see if there is a module for device xyz. In the first place i don't care about the technology and don't want to lock in different places. So having everything in one place would be good.
Also name clashes can be resolved by adding _lua or something.
Maybe each implementation has focus on another aspect of the device.

[TerryE]

IMO, there is one fundamental difference -- for most Lua developers at least -- between a C module and a Lua module and that is that for these developers who aren't skilled in firmware development or building, the _only_ path to adding a C module is to do a new firmware build and re-image their ESP modules. Adding a Lua module is more a matter of cut and paste of the Lua source into SPIFFS or their LFS build: this latter is intrinsically more incremental in nature and more tractable/modifiable by the Lua developer. So Lua and C modules are in essence different IMO.

A secondary point is that any major restructuring of the documentation involves a reasonable commitment in terms of developer resource. In my mind there is a material different between two statements which are in effect:

• it needs to be done (or in other words would someone else do what I propose), and
• _I_ am willing to do XYZ.

The number of people with the skills needed and the ability to commit time resource are limited, so we should divert these resources lightly.

[HHHartmann]

@TerryE

IMO, there is one fundamental difference -- for most Lua developers at least -- between a C module and a Lua module and that is that for these developers who aren't skilled in firmware development or building, the only path to adding a C module is to do a new firmware build and re-image their ESP modules. Adding a Lua module is more a matter of cut and paste of the Lua source into SPIFFS or their LFS build: this latter is intrinsically more incremental in nature and more tractable/modifiable by the Lua developer. So Lua and C modules are in essence different IMO.

I Agree on that. But wouldn't it the be a logical consequence to have the lua modules in a different repository allowing most Lua devs to only have to checkout the (small) lua modules repo and not the relatively huge firmware?

• it needs to be done (or in other words would someone else do what I propose), and
• I am willing to do XYZ.

@TerryE Terry I understand your point and admit that some of my comments fall in the first part but others just try to find a good solution as @galjonsfigur is _willing to do_ it.

If I understand you correctly it would be better to just keep the first ones to myself.

[marcelstoer]

But wouldn't it the be a logical consequence to have the lua modules in a different repository allowing most Lua devs to only have to checkout the (small) lua modules repo and not the relatively huge firmware?

This is unrelated to this issue, which is about _documentation_, but no, I don't think so. Nobody interested in just the Lua modules will checkout the entire repo. They would likely just download the source from this site I suppose**. Besides, having all in one repo helps ensure consistency - same reason we keep the documentation (source) here. Furthermore, providing one integrated set of documentation pages would become a tad more complicated if it spanned more than one repo.

So, again, thank you @galjonsfigur for offering to rewrite the Lua module READMEs. Regardless of what comes out of the discussion here this will be greatly appreciated by many. As I currently cannot image anyone of us committers having the bandwidth and interest to do major refactorings on our main documentation we should keep things simple. That would mean exposing the updated READMEs in a structurally logical way, see @galjonsfigur's first proposal.

** we can encourage that by adding a source/download link to the README just as we have a link to the C source in the main documentation

[nwf]

I apologize for derailing the conversation; the documentation work stands alone and @galjonsfigur is a gentleman and a scholar for being willing to take it on.

[NicolSpies]

I am only a Lua application developer that has evolved from the cloud build service to Docker builds and LFS. The only time I go and scan the C modules is to gain a better understanding when I find the Lua module documentation cryptic and limited. I also scan it to see if there might be "hidden" features not documented. I find myself spending more time in the repo to enable me to interface better with the C developers in the process to give feedback when required.

The documentation structure contains a lot of information that has developed over time and it takes time to learn where to go and look for what is needed during application development. Small incremental changes allows the Lua app developers to grow and keep up with the development.
Upstream planning is easy and sets the framework that will grow and evolve as progress evolves downstream. The sacrifice is that change becomes increasingly time consuming and difficult. I would compare the effort with the benefit in the end and try to see if it makes sense.

Having been on this forum a few years I notice ebb and flow of enthusiasm that seems to be based on breakthroughs triggered by the C developers. This stirs Lua developers to want to participate and also contribute in some way. This is very healthy and brings balance. Collective channeling/management of the willingness to participate will always be challenging and needs to be done in way to encourage further participation. My feeling is that we are not doing to badly.

[jmd13391]

@NicolSpies Spot on.

[marcelstoer]

The only time I go and scan the C modules is to gain a better understanding when I find the Lua module documentation cryptic and limited.

Doesn't ever happen, does it 😜Seriously, if you find such corners in our documentation please re-write and contribute them. If you, with all the experience, find explanations cryptic then it's quite likely that they might be real road blocks for others.

[alexandruantochi]

Regarding Lua vs. C module, is it better to write a new module in C than in Lua? Is having the module inside the firmware and being written in C provide a greater advantage than running a .lua file?

[TerryE]

@alexandruantochi this is the wrong place to have howto discussions. Our Gitter page is a better location. Developing and debugging C modules is a lot harder than doing this in Lua, so only write C when you have to.