Nodemcu-firmware: Are "we" welcoming enough for new modules?

[marcelstoer]

I admit, as a NodeMCU committer I live in my own little ESP8266 bubble which represents just a tiny fraction of the ESP-universe.

Every now and then I find cool stuff built on top of NodeMCU. Sometimes the projects contain modules we don't have in our repository and I wonder "why wouldn't they contribute back to us"? Two recent examples were esp-ginx by @israellot and ESP-RFM69 by @someburner which uses the former. Both don't just provide a single module but a full firmware plus module(s) and configuration. Neither of them started as a fork of this repository. And neither of them use a recent version of the firmware they integrate.

It's true that our module inclusion system doesn't scale well beyond several dozen modules but a) we know that and b) that shouldn't prevent someone from developing new modules under our project umbrella.

OR am I looking at this from the wrong (or too narrow) angle? Hoping for a constructive discussion.

[NicolSpies]

I believe it relates to the focus of a contributor. I share a wider focus and and am constantly scouting for new module additions and application possibilities most probably because I am not a "low" level developer and reside on the application or Lua level.

I suspect that the developers of complete solutions are most probably low level developers that can cook a solution without requiring any assistance unlike myself that are reliant on other low level module contributors.
The fact that they do not use the latest firmware release most probably relates to the fact that they can "fix" or "improve" specifically to their project and are less reliant on firmware upgrades.
If we assume the previous statement to hold water, then maybe the reason can be narrowed down to a mindset of when you are dependent on others and receive help you are willing to take time and give back. A principle of sowing forward.

If this is true, how do we proceed? I believe gatekeepers ensure module integrity. Gatekeepers also know best how to integrate and stay clear of potential problematic areas. So maybe the only way is for a "low level gatekeeper-developer" to incorporate the discovered modules after applying a voting mechanism to judge how many people will value the inclusion.

@marcelstoer, I hope my ramblings are constuctive enough :wink:

[marcelstoer]

Thanks Nicol for that other perspective. Before I created this issues I tried to get the perspective of a module developer.

So, hypothetically, if I were to start module X or project A based on NodeMCU and I wanted to open-source it. Wouldn't I want

• my project to be associated with NodeMCU?
• to profit from an established infrastructure (CI, documentation)?
• my project to stay up-to-date in terms of Espressif SDK?
• my project to be known by / marketed to a large developer community?
• to get feedback from NodeMCU firmware developers and followers i.e. potential users?

If I cook my own soup as we say I have total freedom but I wouldn't get any of the above. The only reason why I'd want to do that I can think of is because I need / want total control over my repository.

My assumption is that for as long as we stick to our current way of including modules we're interested in having as many "good" modules (high added value, low maintenance effort) as possible. That's where the original question whether we should / could do more to attract more contributions stems from.

[devsaurus]

The only reason I can think of is because I need / want total control over my repository.

I think that the threshold for including modules (or contributions in general) is relatively high since there are several steps/decisions to be taken.

As a module developer, am I

• willing to share the new module at all?
• ok to step into the spotlight and have my code reviewed and eventually see it refused?
• motivated enough to adapt and rework according to the feedback?

As collaborators, are we

• understanding/accepting the use case?
• able to review and test the code?
• willing to take over responsibility?

At the moment, a contribution has to go the full way to get it finally included - there's no in-between. Or just as detached projects/repos like Marcel mentioned them.
We've once discussed the question of establishing a looser coupling for non-core modules. Ideas like https://github.com/nodemcu/nodemcu-firmware/issues/1055#issuecomment-187510432 could attract more contributions which would stay away otherwise.

[NicolSpies]

Marcel, I agree with your "control over my repository" view. If the control kept and it is shared on NodeMCU, two repositories will have to be kept updated i.e. more effort.

There may be a indirect relationship between the competence of the developer and the desire for the benefits you listed. Hence my previous thinking; "the better I am or think I am, the less recognition I need". A social group with common interest requires a level of "social skills " that maybe less important (less developed :wink: ) in hotshot developers.

I am scared about looser coupling it implies less checks and balances that could lead to lower quality.

[luismfonseca]

I can share my opinion as I module developer with a ongoing attempt to have my module merged.

Here's why I bothered making a merge request (#1433):

• I wanted to work with a low level language for a while
• I wanted to do something in which others could benefit from my knowledge
• I really would like a native websockets implementation for some of my NodeMCU use cases

I thought about not making a merge request to this project, mostly due to fear of wasting my time. I think that happens with any open-source project at some level. But after creating an issue about it and having no responses I guess I was alone in my initiative (maintainers do not understand/care about other use cases?).

Not sure if my insight helps. Anyhow I really like this project which was why I did it anyway. :)

[israellot]

I can comment using the esp-ginx project as an example. I did it as a proof of concept, for my own learning of the esp capabilities and limitations. I felt comfortable borrowing some code from other projects but keeping my freedom to architect it as I prototype, breaking design rules that would be required on nodemcu for example. In the particular case I was not interested in the whole lua stack.
At the point I made it public, my hope was that people more engaged in the community would take it over and port to different other projects. I believed there was no point on keeping all that learning to myself, so I open-sourced it. But guess what? No one wanted to contribute to it, but everyone wanted to require more features, that would cost me time, which by then I was running short already.

Could I port it back to nodemcu? Definitely.
Did I had the time to do it yet? Definitively not.
Did someone else actually contributed to esp-ginx? Barely. I received almost no pull-requests and a lot more issue reports.
Could it be rewritten into a reusable module that could be added not only to nodemcu but also other projects? Sure. But it seems everyone is waiting the project owner to do it instead of taking the task for themselves.

I´m not sure why developers behave this way, but I would be more than happy to see more collaboration and less fragmentation.

[someburner]

Basically ditto what @israellot said, except that I am guilty of not making pull requests to his project when I could have. Initially I didn't feel that I had a comfortable enough understanding of how his code worked to make pull requests to it. But now that I made a copy specifically for the RFM69, it seems futile to maintain or update that project since there has not been a lot of interest.

I think the nice thing about NodeMCU for the earlier versions ~SDK 1.4.1 was that there was a lot of useful native components that could be used independent of Lua. Like Israel I was interested in making the project run as efficiently as possible, which is what attracted me to esp-ginx in the first place. My own comparisons with esp-ginx and other ESP8266 server projects showed esp-ginx to be much faster. But adding the rest of the Lua stack necessitates a performance decrease. I have seen in issues elsewhere on this repo that the purpose of this firmware is for Lua, so naturally anyone not needing/interested in Lua are barred from making contributions.

[hwmland]

@someburner This sounds to me as clasic egg and chicken problem. What should be first: interest or module:)
I think that new module can attract new user (and perhaps developer of just another module)

[djphoenix]

Another look from me:
I am "full-stack" developer with experience in iOS, *NIX, node.js, Web, OSDev (ouch!) and many more directions. I began developing for IoT not long ago, and ESP8266 is very interesting platform to go. So NodeMCU is most interesting base firmware for prototyping and lightweight projects (for more stability and performance we need native C, of course).

So I have a bit of free time, and an interest for this project. And when I develop something that can be used for NodeMCU (like dynamic timers that have merged to dev recently), I PR'd it back.

The question is "what is the question?". NodeMCU have a flexible and simple module structure, cloud firmware build tool and good community (that will expand). So I think people that develop something outside this project

• don't really think that their work will be helpful
• aren't ready to spend time to make PRs and clean up their code (that works, but may have unstable parts)

So I think we need to extend our README with contribution guidelines with something like "If you're made something really cool, but won't spend time to integrate it in base project, share it in your fork on github and mention it in Issues section, we'll take a look at this anyway".

[marcelstoer]

@djphoenix thanks for you feedback, valid points. I'll update the contribution guidelines.