Circuitpython: allow users to add their own C modules

[v923z]

circuitpython could support user code at the C level, if external modules could be added without having to modify the core. This could be achieved by implementing make fragments as is done in micropython.

The broader context of this issue can be found in https://github.com/adafruit/circuitpython/pull/2811

[hierophect]

Figured it might be helpful to link Micropythons implementation committed here? They have documentation on external C modules here as well.

[tannewt]

I'd happily accept changes to make it easier to integrate custom C code into a CircuitPython build. It shouldn't be done through another module like ulab.

It isn't a priority for me though because it is a very niche use case.

Also, any builds with custom C that is not in this repo of CircuitPython make the build CircuitPython-compatible and not vanilla CircuitPython.

[v923z]

Also, any builds with custom C that is not in this repo of CircuitPython make the build CircuitPython-compatible and not vanilla CircuitPython.

I don't want to weigh in too heavily on this issue, because it is not impossible to insert a custom module, even now. But is the distinction really so important for you? I mean, as soon as you make circuitpython open source, it is not in your hands anymore what others do with it. The question is only how hard you make the modifications.

I can only speak for myself, but for me the turning point in this regard was, when user modules became available in micropython. I had developed C modules earlier, but it was always hackish, and it was hard to distribute. It is really more fun to do anything at the C level now. It also lowers the contribution threshold (c.f. ease of distribution).

[hierophect]

@v923z I don't think Scott is rejecting the idea, so much as establishing what the labeling guidelines will be if it is implemented and what's considered priority for the sponsored-by-Adafruit team. I'd also be very excited to see C modules supported and I saw it in three Circuitpython 2020s, so I don't think anyone would object to a PR being submitted to align with Micropython's implementation. It's just not high on the list for sponsored folks.

I see "Circuitpython" as a label as important because it basically provides an outline of what is being officially supported and maintained by the Adafruit network: forum support, updated guides, working libraries, etc. I can understand Scott's defense of the distinction between "Circuitpython" and "Circuitpython-compatible" because as soon as custom C modules get involved, you risk users seeking official support for untested, potentially buggy 3rd party code that they've confused for mainline content.

I know some people who are using Circuitpython for climate research who are very interested in native C, so I'm a big proponent of the idea, but I can see why good documentation and labelling clarity will be important if it does come in as a PR.

[tannewt]

Also, any builds with custom C that is not in this repo of CircuitPython make the build CircuitPython-compatible and not vanilla CircuitPython.

I don't want to weigh in too heavily on this issue, because it is not impossible to insert a custom module, even now. But is the distinction really so important for you? I mean, as soon as you make circuitpython open source, it is not in your hands anymore what others do with it. The question is only how hard you make the modifications.

Yes, the CircuitPython trademark and brand is incredibly important to me. Losing control of what it means to be CircuitPython means that guides, drivers and experiences will not match user's expectations. By being open source, we give others the right to use the code, not the trademark. I outlined more about it here: https://github.com/adafruit/circuitpython#branding

If folks want to be "CircuitPython" they should have modifications checked into master of the adafruit/circuitpython repo so that it is versioned and released with all other CircuitPython releases.

@v923z I don't think Scott is rejecting the idea, so much as establishing what the labeling guidelines will be if it is implemented and what's considered priority for the sponsored-by-Adafruit team. I'd also be very excited to see C modules supported and I saw it in three Circuitpython 2020s, so I don't think anyone would object to a PR being submitted to align with Micropython's implementation. It's just not high on the list for sponsored folks.

Thanks for clarifying for me @hierophect. I should state what I am for.

We (Adafruit sponsored folks) will always review and merge PRs that don't disrupt the core beginner-friendly goal of CircuitPython. This includes merging custom modules for custom non-Adafruit boards. That way they can be proper "CircuitPython".

[v923z]

Yes, the CircuitPython trademark and brand is incredibly important to me. Losing control of what it means to be CircuitPython means that guides, drivers and experiences will not match user's expectations. By being open source, we give others the right to use the code, _not_ the trademark. I outlined more about it here: https://github.com/adafruit/circuitpython#branding

Fair enough. I wasn't trying to fan the flames, and I can definitely understand and respect this standpoint. Thanks for clarifying!

[tannewt]

Fair enough. I wasn't trying to fan the flames, and I can definitely understand and respect this standpoint. Thanks for clarifying!

No problem! I'm just passionate about it because I think the use of "MicroPython" in other forks has hurt micropython/MicroPython. We renamed ourselves to CircuitPython so we wouldn't further confuse what MicroPython is.

[kevinjwalters]

This idea seems to make sense for libraries that only a small percentage of developers will use ones that would otherwise bulk up the CircuitPython interpreter?

I've done a hacky port of https://github.com/adafruit/Talkie into CircuitPython and that will only be usable for generating samples in non-real-time. That's fine for what I want but it would be nice to be able to use it as it's intended, for real-time playback of speech with an ultra-compact data representation. This is probably one of the more fiddly examples as I can't imagine there is or could easily be a clean interface between this potential library and CP but I think some portion of compiled code is the only option here?

[tannewt]

@kevinjwalters I would love to see us have a speech module like Micro:Bit: https://microbit-micropython.readthedocs.io/en/latest/tutorials/speech.html

I'm ok including bespoke modules as long as we have the space. Most of the new boards have plenty of space.