Esp-idf: Adding a component as a git submodule without altering the submodule with component.mk

[WayneKeenan]

I'd like a facility to be able to add a component as a submodule from git but I would like to keep the submodule untouched so avoid adding component.mk to the component folder.

To do this may I suggest (optionally) supporting a file named similar to the component folder that could be added to the projects git repo without effecting the component submodule, e.g.:

[PROJECT_ROOT]/components/[COMPONENT_FOLDER_NAME].component.mk
is for the folder:
[PROJECT_ROOT]/components/[COMPONENT_FOLDER_NAME]

e.g. The following would add the cJson library as a git submodule and include the folder in the IDF build system without effecting the cJSON submodule repo at all.

git submodule add https://github.com/DaveGamble/cJson components/cJson
touch components/cJson.component.mk
git add components/cJson.component.mk
make

Thanks
Wayne

[WayneKeenan]

or just make any folder that appears in the components folder a component, perhaps?

[vonnieda]

@WayneKeenan I do this extensively in my project by making the submodule a subdirectory of the component directory and then add the component.mk in the component directory. For example, if I want to use https://github.com/semitrivial/csv_parser as a component, I do something like this:

cd my_project/components
mkdir csv_parser
cd csv_parser
git submodule add https://github.com/semitrivial/csv_parser
# csv_parser code is now in my_project/components/csv_parser/csv_parser
touch component.mk

And then I just create a component.mk that references that subdirectory, like this:

COMPONENT_SRCDIRS := csv_parser
COMPONENT_ADD_INCLUDEDIRS := csv_parser

The final directory layout looks like this:

./components/csv_parser
./components/csv_parser/component.mk
./components/csv_parser/csv_parser
./components/csv_parser/csv_parser/csv.h
./components/csv_parser/csv_parser/Makefile
./components/csv_parser/csv_parser/fread_csv_line.c
./components/csv_parser/csv_parser/tests
./components/csv_parser/csv_parser/tests/test.csv
./components/csv_parser/csv_parser/tests/test.c
./components/csv_parser/csv_parser/split.c
./components/csv_parser/csv_parser/README.md
./components/csv_parser/csv_parser/csv.c
./components/csv_parser/csv_parser/.gitignore
./components/csv_parser/csv_parser/.git

This lets you manage the submodule as a clean copy of the repo it belongs to, and keeps the component.mk in your own repo.

In addition, I find this structure to be very helpful because it makes it easy to add project specific code for a submodule. For instance, in my port of Paho MQTT, I needed to add ESP32 specific networking functions. Those go in components/paho/esp32 while the unchanged Paho code from Github goes in components/paho/paho.

Hope this helps!

[WayneKeenan]

That’s a good idea, and I prefer it to mine, especially for cases of needing esp32 glue, thank you very much for sharing the tip @vonnieda
This should be in a tips and trick guide, or a note in the IDF build guide.

[projectgus]

Hi @vonnieda @WayneKeenan,

Thanks for explaining this method. This is also very similar to the method which we try to use in IDF itself, for example you can see in the cJSON component or libsodium:
https://github.com/espressif/esp-idf/tree/master/components/json
https://github.com/espressif/esp-idf/tree/master/components/libsodium

(libsodium is a particularly good example because it has the port-specific code in the IDF tree, but the bulk of the upstream code is in the submodule).

Angus