Esp-idf: [TW#13263] C++ concurrency is not implemented

[yaqwsx]

The current port of libstdc++ does not contain concurrency part of the library (thread, mutex, conditional_variable, ...).

There are already libraries implementing this functionality on top of FreeRTOS, which should be easy to integrate into ESP-IDF, e.g. https://www.researchgate.net/publication/282350782_C14_concurrency_on_ARM_Cortex-M_using_FreeRTOS_and_GCC

[hwmaier]

I read that C++11 concurrency support is planned:
https://www.esp32.com/viewtopic.php?t=362#p1612

How mature and robust is the implementation published in the paper you mentioned?
Is the implementation available somewhere to download? I could not find a link.
And the paper mentioned ARM Cortex M as architecture, is the implementation available for other architectures like the Xtensa CPU the ESP is using?

[HubbyGitter]

I'd like to suggest changing the title.

In C++11, at least one concurrency issue subject to code generation is addressed, namely the initialisation of static variables at runtime. C++11 requires that this occurs without race condition and deadlock. I would call that "C++ concurrency".

The issue reported above might be called "libstdc++ concurrency part not implemented".

[igrr]

Yes, static init guards are implemented on top of FreeRTOS in components/cxx/cxx_guards.cpp.

[yaqwsx]

I contacted the authors of the paper; their implementation is available here. However, I haven't looked at it yet.

On the other hand, I am thinking if implementing Pthreads wrapper for FreeRTOS isn't more useful - it will allow for implementation of C++14 concurrency and also make porting existing pieces of code much easier.

[igrr]

Agree on pthreads wrapper — it's also a useful component for porting existing code to the ESP32.
There is a pthreads wrapper for FreeRTOS available in Lua-RTOS-ESP32 project, but we haven't had time to look into it in detail yet.

[yaqwsx]

I've looked at both the implementations. My impressions are:

• the one I provided is just thin wrapper of FreeRTOS, however I didn't managed to compile the thing. Also it is non-standard implementation with questionable support.
• Pthread implementation from Lua-RTOS-ESP32 looks nearly complete. However, in my opinion is quite heavy (I looked at implementation of Pthread mutex, which is in my opinion much more complicated than neccessary).

Implementing C++ concurrency using Pthread is in my opinion much better way than the native implementation. Writing a custom Pthread implementation from scratch seems stupid to me. I think it would be a nice to take the Lua-RTOS-ESP32 Pthread implementation, compile libstdc++ with concurrency support against it and if necessary, change the implementation to be more lightweight.

Is this something ESP-IDF developers would appreciate/is worth working on?

[HubbyGitter]

There is an Open Source portable PTHREADS implementation here which is apparently based on a portable abstraction layer with simple functions. How about that? It is designed for exactly this case.

[igrr]

Is this something ESP-IDF developers would appreciate/is worth working on?

Definitely, we consider that to be a useful addition. We haven't started working on this yet and we have some more pressing issues to be solved for the next month at least. We would welcome any contribution on this though, especially if it comes with some tests.

[HubbyGitter]

@igrr I just noticed that xtensa-esp32-elf-gcc (crosstool-NG crosstool-ng-1.22.0-61-gab8375a, 5.2.0) will report Thread model: single.

Quote:

All normal disclaimers aside, multithreaded C++ application are only supported when libstdc++ and all user code was built with compilers which report (via gcc/g++ -v ) the same thread model and that model is not _single_.

Could this be a problem?

[igrr]

IIRC this switch affects libgcc and libstdc++, not the compiler itself.

Even if this is a problem, we can release new version of the toolchain built with "pthreads" thread model if/when this feature is implemented.

[HubbyGitter]

@igrr Is GCC applying the static init guards which we discussed above when thread model _single_ is configured?

[igrr]

It does. What happens is that with thread model single the actual implementation of static init guards in libgcc becomes trivial: https://github.com/jcmvbkbc/gcc-xtensa/blob/d230f4322ce0df8cdeb624d99310e4af9e33060c/libstdc%2B%2B-v3/libsupc%2B%2B/guard.cc#L233.
We force our implementation of static init guards to be linked instead of the one in libgcc.

[igrr]

As reported in https://esp32.com/viewtopic.php?f=2&t=2653, another consequence of missing thread support is that shared_ptr becomes non thread safe.

[HubbyGitter]

@igrr Thank you for referring to the forum discussion in the context of this issue. I wasn't aware of this. I guess it explains something at my end.

[PerMalmberg]

@igrr Is there a plan for when this will be worked on? (Is there a public roadmap somewhere?)

[igrr]

Sorry, this is on the backlog and hasn't been assigned to any milestone yet. I will update this thread once there is any movement on this.

[PerMalmberg]

Are talking weeks or months? I'm trying to plan my project so that I don't get stuck waiting for this.

In all honesty, it is a bit of a let-down considering how the ESP32 is advertised. Having the hardware doesn't go a long way when the vendor-provided tools (i.e. the C++ compiler/libs) don't support one of the most prominent features of the hardware, i.e. multi-core/multi-threading.

I don't know what else you're currently working on and I think you're doing an excellent job with ESP-IDF, but I urge you to prioritize this.

[igrr]

I understand your disappointment, please bear with us. My colleague will start working on this next Monday, and we will try to get mutexes support added before 3.0 feature freeze, to resolve the issue with shared_ptr. Other pthreads features will then be added over time.

[PerMalmberg]

Awesome news, thank you so much for that. Solving shared_ptr as a first step will go along way for us modern C++-lovers.

[igrr]

@PerMalmberg We have recently updated the toolchain, it is now built with posix thread model. Minimal posix threads implementation is provided in IDF. It is sufficient to get the right locking behavior for std::shared_ptr, and also to use std::thread and std::mutex.

We will keep working on pthread wrapper to get the rest of the C++ concurrency features supported.

[PerMalmberg]

@igrr Very good news. I will make sure to testdrive this thoroughly.

[PerMalmberg]

Closely related to std::mutex is its friend std::condition_variable. Is that functional, or on the way in? Same question goes for std::recursive_mutex. std::lock_guard and, std::unique_lock.

[PerMalmberg]

Sorry to bug you, I'd really appreciate an update regarding the above @igrr From my testing std::condition_variable is not functional, it'd be really good to have that, together with std::unique_lock and std::recursive_mutex available. It'd open up a lot of opportunities to port code for easier debugging on any POSIX system.

std::thread and std::mutex seems to work as far as I can tell, except that you can't set a stacksize for each separate thread.

[igrr]

As mentioned above, we have a minimal pthread implementation now. Condition variable support is planned, but will need another toolchain update. That is unlikely to happen in the next month or so. This issue will be updated when there is any progress.

[igrr]

edb2400742d2fded2a2a19f349cf8842c1f77f36 adds support for std::condition_variable

[PerMalmberg]

Great, now I can make my own implementation obsolete :) Keep up the good work.

[hak8or]

Can this issue now be closed since the newest Release of IDF (Pre-release 3.0-rc1) changelog says the following?

• libstdc++ threading support
• libstdc++ concurrency support (std::condition_variable, std::mutex, etc)

[yaqwsx]

I am fine and happy for the changes, so it is ok for me to close this issue.