Esp-idf: Linker fails on Mac OS X with "Too many open files" error (IDFGH-1764)

[Micrified]

Environment

• Development Kit: ESP32-DevKitC
• Kit version: v4
• Module or chip used: ESP32-WROOM-32
• IDF version: v4.0-dev-1443-g39f090a4f
• Build System: CMake
• Compiler version: xtensa-esp32-elf-gcc (crosstool-NG esp32-2019r1) 8.2.0
• Operating System: macOS 10.11.6
• Power Supply: USB

Problem Description

Invoking idf.py build on a custom project fails to compile, citing that the following libraries could not be found:

• stdc++
• libnewlib.a
• libhal.a
• lgcov
• lc
• lm
• lgcc

Expected Behavior

I expect this project to compile successfully. I am able to successfully build and flash this project on a machine running Ubuntu.

Actual Behavior

I can successfully compile and flash the hello-world program from this machine. But I cannot compile this project or the example ones for BLE (e.g. BLE server example). The linker will consistently produce this output

Steps to repropduce

1. Invoke idf.py build
2. Fails

Code to reproduce this issue

The code is quite large spanning several files. The project is structured as follows:

- main
    | - include
    |   | - ble.h
    |   | - ble_profiles.h
    |   | - wifi.h
    | - src
    |   | - ble.c
    |   | - wifi.c
- CMakeLists.txt
- component.mk
- esp_32_wifi_main.c

Files component.mk is empty and unchanged. The file CMakeLists.txt contains:

idf_component_register(SRCS "esp32_wifi_main.c" "src/wifi.c" "src/ble.c"
                    INCLUDE_DIRS "include")

Debug Logs

Checking Python dependencies...
Python requirements from /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt are satisfied.
Executing action: all (aliases: build)
Running cmake in directory /Users/Owatch/Documents/Somnox/esp32-wifi/build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DESP_PLATFORM=1 --warn-uninitialized -DCCACHE_ENABLE=0 /Users/Owatch/Documents/Somnox/esp32-wifi"...
Warn about uninitialized values.
-- Found Git: /usr/local/bin/git (found version "2.19.0") 
-- Unexpected file in components directory: /Users/Owatch/Documents/Somnox/esp-idf/components/.DS_Store
-- IDF_TARGET not set, using default target: esp32
-- The C compiler identification is GNU 8.2.0
-- The CXX compiler identification is GNU 8.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
-- Check for working C compiler: /Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
-- Check for working C compiler: /Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++
-- Check for working CXX compiler: /Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Project version: 032af9b
-- Building ESP-IDF components for target esp32
-- Found PythonInterp: /Users/Owatch/Documents/Somnox/esp-idf/tools/python_env/idf4.0_py2.7_env/bin/python (found version "2.7.15") 
-- Found Perl: /usr/bin/perl (found version "5.18.2") 
-- Adding linker script /Users/Owatch/Documents/Somnox/esp32-wifi/build/esp-idf/esp32/esp32_out.ld
-- Adding linker script /Users/Owatch/Documents/Somnox/esp-idf/components/esp32/ld/esp32.project.ld.in
-- Adding linker script /Users/Owatch/Documents/Somnox/esp-idf/components/esp32/ld/esp32.peripherals.ld
-- Adding linker script /Users/Owatch/Documents/Somnox/esp-idf/components/esp_rom/esp32/ld/esp32.rom.ld
-- Adding linker script /Users/Owatch/Documents/Somnox/esp-idf/components/esp_rom/esp32/ld/esp32.rom.libgcc.ld
-- Adding linker script /Users/Owatch/Documents/Somnox/esp-idf/components/esp_rom/esp32/ld/esp32.rom.syscalls.ld
-- Adding linker script /Users/Owatch/Documents/Somnox/esp-idf/components/esp_rom/esp32/ld/esp32.rom.newlib-data.ld
-- Adding linker script /Users/Owatch/Documents/Somnox/esp-idf/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld
-- Components: app_trace app_update asio bootloader bootloader_support bt coap console cxx driver efuse esp-tls esp32 esp_adc_cal esp_common esp_eth esp_event esp_gdbstub esp_http_client esp_http_server esp_https_ota esp_https_server esp_local_ctrl esp_ringbuf esp_rom esp_websocket_client esp_wifi espcoredump esptool_py expat fatfs freemodbus freertos heap idf_test jsmn json libsodium log lwip main mbedtls mdns mqtt newlib nghttp nvs_flash openssl partition_table protobuf-c protocomm pthread sdmmc smartconfig_ack soc spi_flash spiffs tcp_transport tcpip_adapter ulp unity vfs wear_levelling wifi_provisioning wpa_supplicant xtensa
-- Component paths: /Users/Owatch/Documents/Somnox/esp-idf/components/app_trace /Users/Owatch/Documents/Somnox/esp-idf/components/app_update /Users/Owatch/Documents/Somnox/esp-idf/components/asio /Users/Owatch/Documents/Somnox/esp-idf/components/bootloader /Users/Owatch/Documents/Somnox/esp-idf/components/bootloader_support /Users/Owatch/Documents/Somnox/esp-idf/components/bt /Users/Owatch/Documents/Somnox/esp-idf/components/coap /Users/Owatch/Documents/Somnox/esp-idf/components/console /Users/Owatch/Documents/Somnox/esp-idf/components/cxx /Users/Owatch/Documents/Somnox/esp-idf/components/driver /Users/Owatch/Documents/Somnox/esp-idf/components/efuse /Users/Owatch/Documents/Somnox/esp-idf/components/esp-tls /Users/Owatch/Documents/Somnox/esp-idf/components/esp32 /Users/Owatch/Documents/Somnox/esp-idf/components/esp_adc_cal /Users/Owatch/Documents/Somnox/esp-idf/components/esp_common /Users/Owatch/Documents/Somnox/esp-idf/components/esp_eth /Users/Owatch/Documents/Somnox/esp-idf/components/esp_event /Users/Owatch/Documents/Somnox/esp-idf/components/esp_gdbstub /Users/Owatch/Documents/Somnox/esp-idf/components/esp_http_client /Users/Owatch/Documents/Somnox/esp-idf/components/esp_http_server /Users/Owatch/Documents/Somnox/esp-idf/components/esp_https_ota /Users/Owatch/Documents/Somnox/esp-idf/components/esp_https_server /Users/Owatch/Documents/Somnox/esp-idf/components/esp_local_ctrl /Users/Owatch/Documents/Somnox/esp-idf/components/esp_ringbuf /Users/Owatch/Documents/Somnox/esp-idf/components/esp_rom /Users/Owatch/Documents/Somnox/esp-idf/components/esp_websocket_client /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi /Users/Owatch/Documents/Somnox/esp-idf/components/espcoredump /Users/Owatch/Documents/Somnox/esp-idf/components/esptool_py /Users/Owatch/Documents/Somnox/esp-idf/components/expat /Users/Owatch/Documents/Somnox/esp-idf/components/fatfs /Users/Owatch/Documents/Somnox/esp-idf/components/freemodbus /Users/Owatch/Documents/Somnox/esp-idf/components/freertos /Users/Owatch/Documents/Somnox/esp-idf/components/heap /Users/Owatch/Documents/Somnox/esp-idf/components/idf_test /Users/Owatch/Documents/Somnox/esp-idf/components/jsmn /Users/Owatch/Documents/Somnox/esp-idf/components/json /Users/Owatch/Documents/Somnox/esp-idf/components/libsodium /Users/Owatch/Documents/Somnox/esp-idf/components/log /Users/Owatch/Documents/Somnox/esp-idf/components/lwip /Users/Owatch/Documents/Somnox/esp32-wifi/main /Users/Owatch/Documents/Somnox/esp-idf/components/mbedtls /Users/Owatch/Documents/Somnox/esp-idf/components/mdns /Users/Owatch/Documents/Somnox/esp-idf/components/mqtt /Users/Owatch/Documents/Somnox/esp-idf/components/newlib /Users/Owatch/Documents/Somnox/esp-idf/components/nghttp /Users/Owatch/Documents/Somnox/esp-idf/components/nvs_flash /Users/Owatch/Documents/Somnox/esp-idf/components/openssl /Users/Owatch/Documents/Somnox/esp-idf/components/partition_table /Users/Owatch/Documents/Somnox/esp-idf/components/protobuf-c /Users/Owatch/Documents/Somnox/esp-idf/components/protocomm /Users/Owatch/Documents/Somnox/esp-idf/components/pthread /Users/Owatch/Documents/Somnox/esp-idf/components/sdmmc /Users/Owatch/Documents/Somnox/esp-idf/components/smartconfig_ack /Users/Owatch/Documents/Somnox/esp-idf/components/soc /Users/Owatch/Documents/Somnox/esp-idf/components/spi_flash /Users/Owatch/Documents/Somnox/esp-idf/components/spiffs /Users/Owatch/Documents/Somnox/esp-idf/components/tcp_transport /Users/Owatch/Documents/Somnox/esp-idf/components/tcpip_adapter /Users/Owatch/Documents/Somnox/esp-idf/components/ulp /Users/Owatch/Documents/Somnox/esp-idf/components/unity /Users/Owatch/Documents/Somnox/esp-idf/components/vfs /Users/Owatch/Documents/Somnox/esp-idf/components/wear_levelling /Users/Owatch/Documents/Somnox/esp-idf/components/wifi_provisioning /Users/Owatch/Documents/Somnox/esp-idf/components/wpa_supplicant /Users/Owatch/Documents/Somnox/esp-idf/components/xtensa
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/Owatch/Documents/Somnox/esp32-wifi/build
Running ninja in directory /Users/Owatch/Documents/Somnox/esp32-wifi/build
Executing "ninja all"...
[301/1073] Performing configure step for 'bootloader'
-- Found Git: /usr/local/bin/git (found version "2.19.0") 
-- Unexpected file in components directory: /Users/Owatch/Documents/Somnox/esp-idf/components/.DS_Store
-- The C compiler identification is GNU 8.2.0
-- The CXX compiler identification is GNU 8.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
-- Check for working C compiler: /Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
-- Check for working C compiler: /Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++
-- Check for working CXX compiler: /Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Project version: v4.0-dev-1443-g39f090a4f
-- Building ESP-IDF components for target esp32
-- Adding linker script /Users/Owatch/Documents/Somnox/esp-idf/components/esp32/ld/esp32.peripherals.ld
-- Adding linker script /Users/Owatch/Documents/Somnox/esp-idf/components/esp_rom/esp32/ld/esp32.rom.ld
-- Adding linker script /Users/Owatch/Documents/Somnox/esp-idf/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld
-- Adding linker script /Users/Owatch/Documents/Somnox/esp-idf/components/esp_rom/esp32/ld/esp32.rom.libgcc.ld
-- Adding linker script /Users/Owatch/Documents/Somnox/esp-idf/components/bootloader/subproject/main/esp32.bootloader.ld
-- Adding linker script /Users/Owatch/Documents/Somnox/esp-idf/components/bootloader/subproject/main/esp32.bootloader.rom.ld
-- Components: bootloader bootloader_support efuse esp32 esp_common esp_rom esptool_py log main micro-ecc partition_table soc spi_flash xtensa
-- Component paths: /Users/Owatch/Documents/Somnox/esp-idf/components/bootloader /Users/Owatch/Documents/Somnox/esp-idf/components/bootloader_support /Users/Owatch/Documents/Somnox/esp-idf/components/efuse /Users/Owatch/Documents/Somnox/esp-idf/components/esp32 /Users/Owatch/Documents/Somnox/esp-idf/components/esp_common /Users/Owatch/Documents/Somnox/esp-idf/components/esp_rom /Users/Owatch/Documents/Somnox/esp-idf/components/esptool_py /Users/Owatch/Documents/Somnox/esp-idf/components/log /Users/Owatch/Documents/Somnox/esp-idf/components/bootloader/subproject/main /Users/Owatch/Documents/Somnox/esp-idf/components/bootloader/subproject/components/micro-ecc /Users/Owatch/Documents/Somnox/esp-idf/components/partition_table /Users/Owatch/Documents/Somnox/esp-idf/components/soc /Users/Owatch/Documents/Somnox/esp-idf/components/spi_flash /Users/Owatch/Documents/Somnox/esp-idf/components/xtensa
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/Owatch/Documents/Somnox/esp32-wifi/build/bootloader
[345/1073] Performing build step for 'bootloader'
[1/62] Generating project_elf_src.c
[2/62] Building C object CMakeFiles/bootloader.elf.dir/project_elf_src.c.obj
[3/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/esp32/gpio_periph.c.obj
[4/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/esp32/rtc_periph.c.obj
[5/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/esp32/rtc_pm.c.obj
[6/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/esp32/cpu_util.c.obj
[7/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/esp32/rtc_init.c.obj
[8/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/esp32/rtc_time.c.obj
[9/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/esp32/sdmmc_periph.c.obj
[10/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/esp32/rtc_wdt.c.obj
[11/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/esp32/rtc_clk_init.c.obj
[12/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/esp32/soc_memory_layout.c.obj
[13/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/esp32/spi_periph.c.obj
[14/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/src/lldesc.c.obj
[15/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/esp32/sdio_slave_periph.c.obj
[16/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/src/memory_layout_utils.c.obj
[17/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/esp32/rtc_sleep.c.obj
[18/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/src/soc_include_legacy_warn.c.obj
[19/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/src/hal/spi_slave_hal.c.obj
[20/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/src/hal/spi_flash_hal.c.obj
[21/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/esp32/rtc_clk.c.obj
[22/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/src/hal/spi_slave_hal_iram.c.obj
[23/62] Building ASM object esp-idf/xtensa/CMakeFiles/__idf_xtensa.dir/debug_helpers_asm.S.obj
[24/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/src/hal/spi_hal.c.obj
[25/62] Building C object esp-idf/xtensa/CMakeFiles/__idf_xtensa.dir/eri.c.obj
[26/62] Building C object esp-idf/xtensa/CMakeFiles/__idf_xtensa.dir/debug_helpers.c.obj
[27/62] Building C object esp-idf/xtensa/CMakeFiles/__idf_xtensa.dir/trax.c.obj
[28/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/src/hal/spi_hal_iram.c.obj
[29/62] Building C object esp-idf/efuse/CMakeFiles/__idf_efuse.dir/esp32/esp_efuse_table.c.obj
[30/62] Building C object esp-idf/soc/CMakeFiles/__idf_soc.dir/src/hal/spi_flash_hal_iram.c.obj
[31/62] Building C object esp-idf/log/CMakeFiles/__idf_log.dir/log.c.obj
[32/62] Linking C static library esp-idf/log/liblog.a
[33/62] Linking C static library esp-idf/xtensa/libxtensa.a
[34/62] Linking C static library esp-idf/soc/libsoc.a
[35/62] Building C object esp-idf/efuse/CMakeFiles/__idf_efuse.dir/src/esp_efuse_api.c.obj
[36/62] Building C object esp-idf/efuse/CMakeFiles/__idf_efuse.dir/src/esp_efuse_fields.c.obj
[37/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/bootloader_clock.c.obj
[38/62] Building C object esp-idf/spi_flash/CMakeFiles/__idf_spi_flash.dir/spi_flash_rom_patch.c.obj
[39/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/bootloader_flash.c.obj
[40/62] Building C object esp-idf/efuse/CMakeFiles/__idf_efuse.dir/src/esp_efuse_utility.c.obj
[41/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/bootloader_flash_config.c.obj
[42/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/bootloader_common.c.obj
[43/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/flash_encrypt.c.obj
[44/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/flash_partitions.c.obj
[45/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/bootloader_random.c.obj
[46/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/flash_qio_mode.c.obj
[47/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/esp32/bootloader_sha.c.obj
[48/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/esp32/secure_boot_signatures.c.obj
[49/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/esp_image_format.c.obj
[50/62] Building C object esp-idf/main/CMakeFiles/__idf_main.dir/bootloader_start.c.obj
[51/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/bootloader_utility.c.obj
[52/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/esp32/flash_encrypt.c.obj
[53/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/esp32/secure_boot.c.obj
[54/62] Building C object esp-idf/bootloader_support/CMakeFiles/__idf_bootloader_support.dir/src/bootloader_init.c.obj
[55/62] Building C object esp-idf/micro-ecc/CMakeFiles/__idf_micro-ecc.dir/micro-ecc/uECC.c.obj
[56/62] Linking C static library esp-idf/micro-ecc/libmicro-ecc.a
[57/62] Linking C static library esp-idf/bootloader_support/libbootloader_support.a
[58/62] Linking C static library esp-idf/efuse/libefuse.a
[59/62] Linking C static library esp-idf/spi_flash/libspi_flash.a
[60/62] Linking C static library esp-idf/main/libmain.a
[61/62] Linking C executable bootloader.elf
[62/62] Generating binary image from built executable
esptool.py v2.7-dev
Generated /Users/Owatch/Documents/Somnox/esp32-wifi/build/bootloader/bootloader.bin
[1072/1073] Linking CXX executable esp32-wifi.elf
FAILED: esp32-wifi.elf 
: && /Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++  -mlongcalls -Wno-frame-address  -nostdlib CMakeFiles/esp32-wifi.elf.dir/project_elf_src.c.obj  -o esp32-wifi.elf  esp-idf/esp_ringbuf/libesp_ringbuf.a esp-idf/driver/libdriver.a esp-idf/wpa_supplicant/libwpa_supplicant.a esp-idf/efuse/libefuse.a esp-idf/bootloader_support/libbootloader_support.a esp-idf/app_update/libapp_update.a esp-idf/spi_flash/libspi_flash.a esp-idf/nvs_flash/libnvs_flash.a esp-idf/smartconfig_ack/libsmartconfig_ack.a esp-idf/esp_wifi/libesp_wifi.a esp-idf/esp_eth/libesp_eth.a esp-idf/lwip/liblwip.a esp-idf/tcpip_adapter/libtcpip_adapter.a esp-idf/esp_event/libesp_event.a esp-idf/pthread/libpthread.a esp-idf/espcoredump/libespcoredump.a esp-idf/esp32/libesp32.a esp-idf/xtensa/libxtensa.a esp-idf/esp_common/libesp_common.a esp-idf/esp_rom/libesp_rom.a esp-idf/soc/libsoc.a esp-idf/log/liblog.a esp-idf/heap/libheap.a esp-idf/freertos/libfreertos.a esp-idf/vfs/libvfs.a esp-idf/newlib/libnewlib.a esp-idf/cxx/libcxx.a esp-idf/app_trace/libapp_trace.a esp-idf/asio/libasio.a esp-idf/bt/libbt.a esp-idf/coap/libcoap.a esp-idf/console/libconsole.a esp-idf/nghttp/libnghttp.a esp-idf/esp-tls/libesp-tls.a esp-idf/esp_adc_cal/libesp_adc_cal.a esp-idf/esp_gdbstub/libesp_gdbstub.a esp-idf/tcp_transport/libtcp_transport.a esp-idf/esp_http_client/libesp_http_client.a esp-idf/esp_http_server/libesp_http_server.a esp-idf/esp_https_ota/libesp_https_ota.a esp-idf/protobuf-c/libprotobuf-c.a esp-idf/protocomm/libprotocomm.a esp-idf/mdns/libmdns.a esp-idf/esp_local_ctrl/libesp_local_ctrl.a esp-idf/esp_websocket_client/libesp_websocket_client.a esp-idf/expat/libexpat.a esp-idf/wear_levelling/libwear_levelling.a esp-idf/sdmmc/libsdmmc.a esp-idf/fatfs/libfatfs.a esp-idf/freemodbus/libfreemodbus.a esp-idf/jsmn/libjsmn.a esp-idf/json/libjson.a esp-idf/libsodium/liblibsodium.a esp-idf/mqtt/libmqtt.a esp-idf/openssl/libopenssl.a esp-idf/spiffs/libspiffs.a esp-idf/ulp/libulp.a esp-idf/unity/libunity.a esp-idf/wifi_provisioning/libwifi_provisioning.a esp-idf/main/libmain.a -Wl,--cref -Wl,--Map=/Users/Owatch/Documents/Somnox/esp32-wifi/build/esp32-wifi.map esp-idf/esp_ringbuf/libesp_ringbuf.a esp-idf/driver/libdriver.a esp-idf/wpa_supplicant/libwpa_supplicant.a esp-idf/efuse/libefuse.a esp-idf/bootloader_support/libbootloader_support.a esp-idf/app_update/libapp_update.a esp-idf/spi_flash/libspi_flash.a esp-idf/nvs_flash/libnvs_flash.a esp-idf/smartconfig_ack/libsmartconfig_ack.a esp-idf/esp_wifi/libesp_wifi.a esp-idf/esp_eth/libesp_eth.a esp-idf/lwip/liblwip.a esp-idf/tcpip_adapter/libtcpip_adapter.a esp-idf/esp_event/libesp_event.a esp-idf/pthread/libpthread.a esp-idf/espcoredump/libespcoredump.a esp-idf/esp32/libesp32.a esp-idf/xtensa/libxtensa.a esp-idf/esp_common/libesp_common.a esp-idf/esp_rom/libesp_rom.a esp-idf/soc/libsoc.a esp-idf/log/liblog.a esp-idf/heap/libheap.a esp-idf/freertos/libfreertos.a esp-idf/vfs/libvfs.a esp-idf/newlib/libnewlib.a esp-idf/cxx/libcxx.a esp-idf/app_trace/libapp_trace.a esp-idf/asio/libasio.a esp-idf/bt/libbt.a esp-idf/coap/libcoap.a esp-idf/console/libconsole.a esp-idf/nghttp/libnghttp.a esp-idf/esp-tls/libesp-tls.a esp-idf/esp_adc_cal/libesp_adc_cal.a esp-idf/esp_gdbstub/libesp_gdbstub.a esp-idf/tcp_transport/libtcp_transport.a esp-idf/esp_http_client/libesp_http_client.a esp-idf/esp_http_server/libesp_http_server.a esp-idf/esp_https_ota/libesp_https_ota.a esp-idf/esp_http_client/libesp_http_client.a esp-idf/protobuf-c/libprotobuf-c.a esp-idf/protocomm/libprotocomm.a esp-idf/mdns/libmdns.a esp-idf/esp_local_ctrl/libesp_local_ctrl.a esp-idf/esp_websocket_client/libesp_websocket_client.a esp-idf/expat/libexpat.a esp-idf/wear_levelling/libwear_levelling.a esp-idf/sdmmc/libsdmmc.a esp-idf/fatfs/libfatfs.a esp-idf/wear_levelling/libwear_levelling.a esp-idf/sdmmc/libsdmmc.a esp-idf/freemodbus/libfreemodbus.a esp-idf/jsmn/libjsmn.a esp-idf/json/libjson.a esp-idf/libsodium/liblibsodium.a esp-idf/mqtt/libmqtt.a esp-idf/tcp_transport/libtcp_transport.a esp-idf/esp-tls/libesp-tls.a esp-idf/openssl/libopenssl.a esp-idf/spiffs/libspiffs.a esp-idf/ulp/libulp.a esp-idf/unity/libunity.a esp-idf/wifi_provisioning/libwifi_provisioning.a esp-idf/protocomm/libprotocomm.a esp-idf/esp_http_server/libesp_http_server.a esp-idf/nghttp/libnghttp.a esp-idf/bt/libbt.a -L/Users/Owatch/Documents/Somnox/esp-idf/components/bt/controller/lib -lbtdm_app esp-idf/protobuf-c/libprotobuf-c.a esp-idf/mdns/libmdns.a esp-idf/console/libconsole.a esp-idf/json/libjson.a esp-idf/cxx/libcxx.a esp-idf/newlib/libnewlib.a esp-idf/freertos/libfreertos.a esp-idf/heap/libheap.a esp-idf/log/liblog.a esp-idf/soc/libsoc.a esp-idf/esp_rom/libesp_rom.a esp-idf/esp_common/libesp_common.a esp-idf/xtensa/libxtensa.a esp-idf/esp32/libesp32.a esp-idf/esp_ringbuf/libesp_ringbuf.a esp-idf/lwip/liblwip.a esp-idf/mbedtls/mbedtls/library/libmbedtls.a esp-idf/mbedtls/mbedtls/library/libmbedcrypto.a esp-idf/mbedtls/mbedtls/library/libmbedx509.a esp-idf/bootloader_support/libbootloader_support.a esp-idf/spi_flash/libspi_flash.a esp-idf/efuse/libefuse.a esp-idf/app_update/libapp_update.a esp-idf/tcpip_adapter/libtcpip_adapter.a esp-idf/wpa_supplicant/libwpa_supplicant.a esp-idf/nvs_flash/libnvs_flash.a esp-idf/smartconfig_ack/libsmartconfig_ack.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libcoexist.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libcore.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libespnow.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libmesh.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libnet80211.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libphy.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libpp.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/librtc.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libsmartconfig.a esp-idf/esp_event/libesp_event.a esp-idf/driver/libdriver.a esp-idf/vfs/libvfs.a esp-idf/esp_wifi/libesp_wifi.a esp-idf/esp_eth/libesp_eth.a esp-idf/app_trace/libapp_trace.a esp-idf/pthread/libpthread.a esp-idf/espcoredump/libespcoredump.a esp-idf/cxx/libcxx.a esp-idf/newlib/libnewlib.a esp-idf/freertos/libfreertos.a esp-idf/heap/libheap.a esp-idf/log/liblog.a esp-idf/soc/libsoc.a esp-idf/esp_rom/libesp_rom.a esp-idf/esp_common/libesp_common.a esp-idf/xtensa/libxtensa.a esp-idf/esp32/libesp32.a esp-idf/esp_ringbuf/libesp_ringbuf.a esp-idf/lwip/liblwip.a esp-idf/mbedtls/mbedtls/library/libmbedtls.a esp-idf/mbedtls/mbedtls/library/libmbedcrypto.a esp-idf/mbedtls/mbedtls/library/libmbedx509.a esp-idf/bootloader_support/libbootloader_support.a esp-idf/spi_flash/libspi_flash.a esp-idf/efuse/libefuse.a esp-idf/app_update/libapp_update.a esp-idf/tcpip_adapter/libtcpip_adapter.a esp-idf/wpa_supplicant/libwpa_supplicant.a esp-idf/nvs_flash/libnvs_flash.a esp-idf/smartconfig_ack/libsmartconfig_ack.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libcoexist.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libcore.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libespnow.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libmesh.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libnet80211.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libphy.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libpp.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/librtc.a /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32/libsmartconfig.a esp-idf/esp_event/libesp_event.a esp-idf/driver/libdriver.a esp-idf/vfs/libvfs.a esp-idf/esp_wifi/libesp_wifi.a esp-idf/esp_eth/libesp_eth.a esp-idf/app_trace/libapp_trace.a esp-idf/pthread/libpthread.a esp-idf/espcoredump/libespcoredump.a -lstdc++ -u __cxa_guard_dummy -u __cxx_fatal_exception esp-idf/newlib/libnewlib.a -u newlib_include_locks_impl -u newlib_include_heap_impl -u newlib_include_syscalls_impl -u newlib_include_pthread_impl -Wl,--undefined=uxTopUsedPriority -L /Users/Owatch/Documents/Somnox/esp-idf/components/esp_rom/esp32/ld -T esp32.rom.ld -T esp32.rom.libgcc.ld -T esp32.rom.syscalls.ld -T esp32.rom.newlib-data.ld -T esp32.rom.newlib-funcs.ld -Wl,--gc-sections /Users/Owatch/Documents/Somnox/esp-idf/components/xtensa/esp32/libhal.a -L /Users/Owatch/Documents/Somnox/esp32-wifi/build/esp-idf/esp32 -T esp32_out.ld -u app_main -L /Users/Owatch/Documents/Somnox/esp32-wifi/build/esp-idf/esp32/ld -T esp32.project.ld -L /Users/Owatch/Documents/Somnox/esp-idf/components/esp32/ld -T esp32.peripherals.ld -u call_user_start_cpu0 -u ld_include_panic_highint_hdl -u esp_app_desc -u vfs_include_syscalls_impl -L /Users/Owatch/Documents/Somnox/esp-idf/components/esp_wifi/lib_esp32 -lgcov -lc -lm -lgcc -u pthread_include_pthread_impl -u pthread_include_pthread_cond_impl -u pthread_include_pthread_local_storage_impl && :
/Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld: cannot find -lstdc++
/Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld: cannot find esp-idf/newlib/libnewlib.a: Too many open files
/Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld: cannot find /Users/Owatch/Documents/Somnox/esp-idf/components/xtensa/esp32/libhal.a: Too many open files
/Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld: cannot find -lgcov
/Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld: cannot find -lc
/Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld: cannot find -lm
/Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld: cannot find -lgcc
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
ninja failed with exit code 1

Other items if possible

• [x] sdkconfig file (attach the sdkconfig file from your project folder): Linked here
• [ ] elf file in the build folder: None
• [ ] coredump: None

[dobairoland]

Hi @Micrified.

Please make sure that you get the same failure after you have removed the build directory (rm -r build).

Are you trying to create your own component or you just want an application which can be flashed? In the former case, I don't think that esp_32_wifi_main.c should be there (I'm assuming an app_main is inside). If the latter case, no idf_component_register is required.

Note that in the case of idf_component_register you probably want to specify its dependencies by the REQUIRES keyword (e.g. https://github.com/espressif/esp-idf/blob/master/components/app_update/CMakeLists.txt#L4).

Can you link please the exact path of the example which cannot be compiled at your end? That would be easier to use for tracking down your problem.

[Micrified]

@dobairoland

Hello.

So I removed the build directory and tried again - same error. Since I'm trying to build an application that can be flashed, I want esp_32_wifi_main.c to be there. I build it by invoking: idf.py build, which is what produces these linker errors for me.

What am I supposed to list in the requires section if these libraries don't exist? How come they exist if I build the exact same project on Linux? This is confusing to me a bit. The linker knows they should be there but it cannot find them for some reason.

---

Problem with compiling BLE example

Alright I ran tree but only to three levels deep to show you the placement of the example with respect to esp-idf:

.
├── esp-idf
│   ├── CMakeLists.txt
│   ├── CONTRIBUTING.rst
│   ├── Kconfig
│   ├── LICENSE
│   ├── README.md
│   ├── add_path.sh
│   ├── components
│   │   ├── app_trace
│   │   ├── app_update
│   │   ├── asio
│   │   ├── bootloader
│   │   ├── bootloader_support
│   │   ├── bt
│   │   ├── coap
│   │   ├── console
│   │   ├── cxx
│   │   ├── driver
│   │   ├── efuse
│   │   ├── esp-tls
│   │   ├── esp32
│   │   ├── esp_adc_cal
│   │   ├── esp_common
│   │   ├── esp_eth
│   │   ├── esp_event
│   │   ├── esp_gdbstub
│   │   ├── esp_http_client
│   │   ├── esp_http_server
│   │   ├── esp_https_ota
│   │   ├── esp_https_server
│   │   ├── esp_local_ctrl
│   │   ├── esp_ringbuf
│   │   ├── esp_rom
│   │   ├── esp_websocket_client
│   │   ├── esp_wifi
│   │   ├── espcoredump
│   │   ├── esptool_py
│   │   ├── expat
│   │   ├── fatfs
│   │   ├── freemodbus
│   │   ├── freertos
│   │   ├── heap
│   │   ├── idf_test
│   │   ├── jsmn
│   │   ├── json
│   │   ├── libsodium
│   │   ├── log
│   │   ├── lwip
│   │   ├── mbedtls
│   │   ├── mdns
│   │   ├── mqtt
│   │   ├── newlib
│   │   ├── nghttp
│   │   ├── nvs_flash
│   │   ├── openssl
│   │   ├── partition_table
│   │   ├── protobuf-c
│   │   ├── protocomm
│   │   ├── pthread
│   │   ├── sdmmc
│   │   ├── smartconfig_ack
│   │   ├── soc
│   │   ├── spi_flash
│   │   ├── spiffs
│   │   ├── tcp_transport
│   │   ├── tcpip_adapter
│   │   ├── ulp
│   │   ├── unity
│   │   ├── vfs
│   │   ├── wear_levelling
│   │   ├── wifi_provisioning
│   │   ├── wpa_supplicant
│   │   └── xtensa
│   ├── docs
│   │   ├── Doxyfile
│   │   ├── README.md
│   │   ├── TEMPLATE_EXAMPLE_README.md
│   │   ├── _static
│   │   ├── check_doc_warnings.sh
│   │   ├── check_lang_folder_sync.sh
│   │   ├── conf_common.py
│   │   ├── docs_common.mk
│   │   ├── en
│   │   ├── gen-dxd.py
│   │   ├── gen-toolchain-links.py
│   │   ├── gen-version-specific-includes.py
│   │   ├── html_redirects.py
│   │   ├── issue_template.md
│   │   ├── link-roles.py
│   │   ├── local_util.py
│   │   ├── page_redirects.txt
│   │   ├── requirements.txt
│   │   ├── sphinx-known-warnings.txt
│   │   └── zh_CN
│   ├── examples
│   │   ├── README.md
│   │   ├── bluetooth
│   │   ├── build_system
│   │   ├── common_components
│   │   ├── ethernet
│   │   ├── get-started
│   │   ├── mesh
│   │   ├── peripherals
│   │   ├── protocols
│   │   ├── provisioning
│   │   ├── security
│   │   ├── storage
│   │   ├── system
│   │   └── wifi
│   ├── export.bat
│   ├── export.sh
│   ├── include_export.sh
│   ├── install.bat
│   ├── install.sh
│   ├── make
│   │   ├── common.mk
│   │   ├── component_common.mk
│   │   ├── component_wrapper.mk
│   │   ├── ldgen.mk
│   │   ├── project.mk
│   │   ├── project_config.mk
│   │   └── version.mk
│   ├── requirements.txt
│   ├── sdkconfig.rename
│   └── tools
│       ├── ble
│       ├── catch
│       ├── check_kconfigs.py
│       ├── check_python_dependencies.py
│       ├── ci
│       ├── cmake
│       ├── dist
│       ├── docker
│       ├── eclipse-code-style.xml
│       ├── elf_to_ld.sh
│       ├── esp_app_trace
│       ├── esp_prov
│       ├── format-minimal.sh
│       ├── format.sh
│       ├── gen_esp_err_to_name.py
│       ├── idf.py
│       ├── idf_monitor.py
│       ├── idf_size.py
│       ├── idf_tools.py
│       ├── kconfig
│       ├── kconfig_new
│       ├── ldgen
│       ├── mass_mfg
│       ├── python_env
│       ├── set-submodules-to-github.sh
│       ├── test_check_kconfigs.py
│       ├── test_idf_monitor
│       ├── test_idf_size
│       ├── test_idf_tools
│       ├── tiny-test-fw
│       ├── toolchain_versions.mk
│       ├── tools
│       ├── tools.json
│       ├── tools_schema.json
│       ├── unit-test-app
│       └── windows
└── gatt_server
    ├── CMakeLists.txt
    ├── Makefile
    ├── README.md
    ├── main
    │   ├── CMakeLists.txt
    │   ├── Kconfig
    │   ├── component.mk
    │   └── gatts_demo.c
    ├── sdkconfig
    ├── sdkconfig.defaults
    └── tutorial
        ├── Gatt_Server_Example_Walkthrough.md
        └── image

Just for your information, my .bash_profile contains the following:

export IDF_TOOLS_PATH=/Users/Owatch/Documents/Somnox/esp-idf/tools
export IDF_PATH=/Users/Owatch/Documents/Somnox/esp-idf

[renzbagaporo]

Hi @Micrified,

Can you try increasing the open files limit? https://superuser.com/questions/827984/open-files-limit-does-not-work-as-before-in-osx-yosemite/1171026#1171026

[dobairoland]

In that case, you don't need an idf_component_register statement.
Your CMakeList.txt file should look as simple as https://github.com/espressif/esp-idf/blob/master/examples/bluetooth/bluedroid/ble/gatt_server/CMakeLists.txt.

esp-idf/examples/bluetooth/bluedroid/ble/gatt_server compiles for me without any issues. You have the same error about missing standard libraries for that example as well?

Are you using the install.sh and export.sh scripts provided in the root ESP-IDF directory?

[Micrified]

@dobairoland

Hello again,

Yeah, so my CMakeList.txt file basically is unchanged. The project simply spits out these errors when I try to compile. It compiles fine on my other machines. I've copied the example several times to compile it, and obtained the entire git repository again more than once. It always produces these compile errors.

---

Install.sh / Export.sh

CXR-MBP13:esp-idf Owatch$ ./install.sh 
Installing ESP-IDF tools
Installing tools: xtensa-esp32-elf, esp32ulp-elf, openocd-esp32
Skipping [email protected] (already installed)
Skipping [email protected] (already installed)
Skipping [email protected] (already installed)
Installing Python environment and packages
Installing Python packages from /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already satisfied: setuptools in ./tools/python_env/idf4.0_py2.7_env/lib/python2.7/site-packages (from -r /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt (line 4)) (41.1.0)
Requirement already satisfied: click>=5.0 in ./tools/python_env/idf4.0_py2.7_env/lib/python2.7/site-packages (from -r /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt (line 8)) (7.0)
Requirement already satisfied: pyserial>=3.0 in ./tools/python_env/idf4.0_py2.7_env/lib/python2.7/site-packages (from -r /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt (line 9)) (3.4)
Requirement already satisfied: future>=0.15.2 in ./tools/python_env/idf4.0_py2.7_env/lib/python2.7/site-packages (from -r /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt (line 10)) (0.17.1)
Requirement already satisfied: cryptography>=2.1.4 in ./tools/python_env/idf4.0_py2.7_env/lib/python2.7/site-packages (from -r /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt (line 11)) (2.7)
Requirement already satisfied: pyparsing<2.4.0,>=2.0.3 in ./tools/python_env/idf4.0_py2.7_env/lib/python2.7/site-packages (from -r /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt (line 12)) (2.3.1)
Requirement already satisfied: pyelftools>=0.22 in ./tools/python_env/idf4.0_py2.7_env/lib/python2.7/site-packages (from -r /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt (line 13)) (0.25)
Requirement already satisfied: enum34; python_version < "3" in ./tools/python_env/idf4.0_py2.7_env/lib/python2.7/site-packages (from cryptography>=2.1.4->-r /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt (line 11)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in ./tools/python_env/idf4.0_py2.7_env/lib/python2.7/site-packages (from cryptography>=2.1.4->-r /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt (line 11)) (0.24.0)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in ./tools/python_env/idf4.0_py2.7_env/lib/python2.7/site-packages (from cryptography>=2.1.4->-r /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt (line 11)) (1.12.3)
Requirement already satisfied: six>=1.4.1 in ./tools/python_env/idf4.0_py2.7_env/lib/python2.7/site-packages (from cryptography>=2.1.4->-r /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt (line 11)) (1.12.0)
Requirement already satisfied: ipaddress; python_version < "3" in ./tools/python_env/idf4.0_py2.7_env/lib/python2.7/site-packages (from cryptography>=2.1.4->-r /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt (line 11)) (1.0.22)
Requirement already satisfied: pycparser in ./tools/python_env/idf4.0_py2.7_env/lib/python2.7/site-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.1.4->-r /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt (line 11)) (2.19)
WARNING: You are using pip version 19.2.2, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
All done! You can now run:

  . ./export.sh

CXR-MBP13:esp-idf Owatch$ . ./export.
-bash: ./export.: No such file or directory
CXR-MBP13:esp-idf Owatch$ . ./export.sh
Adding ESP-IDF tools to PATH...
Checking if Python packages are up to date...
Python requirements from /Users/Owatch/Documents/Somnox/esp-idf/requirements.txt are satisfied.
Added the following directories to PATH:
  /Users/Owatch/Documents/Somnox/esp-idf/components/esptool_py/esptool
  /Users/Owatch/Documents/Somnox/esp-idf/components/espcoredump
  /Users/Owatch/Documents/Somnox/esp-idf/components/partition_table/
  /Users/Owatch/Documents/Somnox/esp-idf/tools/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin
  /Users/Owatch/Documents/Somnox/esp-idf/tools/tools/esp32ulp-elf/2.28.51.20170517/esp32ulp-elf-binutils/bin
  /Users/Owatch/Documents/Somnox/esp-idf/tools/tools/openocd-esp32/v0.10.0-esp32-20190708/openocd-esp32/bin
  /Users/Owatch/Documents/Somnox/esp-idf/tools/python_env/idf4.0_py2.7_env/bin
  /Users/Owatch/Documents/Somnox/esp-idf/tools
Done! You can now compile ESP-IDF projects.
Go to the project directory and run:

  idf.py build

---

Yes I do use the export.sh scripts. I've also tried building using just make, and then make flash monitor but they basically fail for the same reasons. There must just be some misconfigured libraries somewhere but I'm not sure how to fix that because the build system shouldn't be looking only in my own system libraries to find these.

[dobairoland]

I'm sorry but I have to ask again to confirm that you are talking about examples/bluetooth/bluedroid/ble/gatt_server and you didn't changed anything in it. Could you share the full build log for this example?

The given example can be compiled on Linux and MacOS as well and I cannot reproduce your issue.

Could you try a newer version of ESP-IDF (either the master branch or newer release/v4.0). There are a lot of related changes since the commit you are using. However, I can build even with the mentioned commit without an issue.

Another thing you could try is to remove the ~/.espressif/tools/xtensa-esp32-elf/ directory and reinstall it by install.sh.

[Micrified]

Hey @dobairoland

Yeah, it's that example precisely. In fact, I made a screencast showing me successfully compiling hello_world, but not gatt_server. There are some time-stamps in the description you can jump to because my laptop took some time to compile them (all under 5m in total though)

I've yet to try increasing the open-file limit suggested by @renzbagaporo. I'll do that now.

Edit:
It seems I allow an unlimited number of open files already (but for the hard-limit)

CXR-MBP13:gatt_server Owatch$ launchctl limit maxfiles
    maxfiles    256            unlimited    

[dobairoland]

Hello @Micrified. Thank you checking all the ideas we gave you and making the screencast. I'm sorry we could not give you a direct and simple solution yet. Unfortunately, I still cannot reproduce the issue.

We tried on macOS to build the same example using the same IDF version.

Could you please give to me a verbose version of the build for the gatt_server example? I would like to examine it more closely. It will be long so it would be better to put into a file like this:
rm -r build sdkconfig ; idf.py -v build > log_file 2>&1

Thank you for your patience!

[sieren]

Chiming in to say I'm having the same issue on macOS 10.14.6 (18G95) with exactly the same error messages.
Increasing the open files did not solve anything.

ESP-IDF based on head on release/v4.0 branch

[sieren]

I solved it for myself by removing ~/.espressif and reinstalling the toolchain.
But please, fix https://github.com/espressif/esp-idf/issues/4081 https://github.com/espressif/esp-idf/issues/4111 ... its kind of a joke that the toolchain server is unreachable for this long without modifying your install scripts ignore SSL certificates.

[Micrified]

@dobairoland Sorry I forgot about this thread when I took a break. Here's the log file. It seems to report a successfully build.

Can you explain a bit how your instructions fixed my issue?

Edit: I can build my project successfully now! You can close the thread now

[dobairoland]

I'm happy for you @Micrified! So you did reinstall the toolchain and that helped? Or did you do anything else?

[Micrified]

@dobairoland Thanks! I did what you asked I try, which is this: rm -r build sdkconfig ; idf.py -v build > log_file 2>&1

[sieren]

I'm back at this error, turns out I was using the old toolchain when it resolved itself.
I tried to run the commands, but it seems like it breaks SPIFFS:

CMake Error at /Users/sierenmusic/Documents/Development/esp-idf/components/esptool_py/project_include.cmake:171 (file):
  file RELATIVE_PATH must be passed a full path to the file: FLASH_IN_PROJECT
Call Stack (most recent call first):
  /Users/sierenmusic/Documents/Development/esp-idf/components/spiffs/project_include.cmake:45 (esptool_py_flash_project_args)
  main/CMakeLists.txt:118 (spiffs_create_partition_image)

[dobairoland]

Hi @sieren. Please create a new issue for the SPIFSS one because I believe these are unrelated. Thanks!

[manuelbl]

After running into the same issue (too many open files), I've tried most of the options mentioned in this issue. Unfortunately, this issue does not go to the root of the cause. Instead, the conclusion is: it resolved itself.

I was able to fix it, and I can pinpoint a single measure: increase the limit of open files and reboot.

For increasing the limit, see https://wilsonmar.github.io/maximum-limits/ and follow the instructions for creating /Library/LaunchDaemons/limit.maxproc.plist. Then reboot. (I first tried withsudo launchctl limit maxfiles 65536 200000 and it did not solve the issue.)

If this solution helps you as well, add a thumbs up.

[julioppgca]

After running into the same issue (_too many open files_), I've tried most of the options mentioned in this issue. Unfortunately, this issue does not go to the root of the cause. Instead, the conclusion is: _it resolved itself_.

I was able to fix it, and I can pinpoint a single measure: increase the limit of open files and reboot.

For increasing the limit, see https://wilsonmar.github.io/maximum-limits/ and follow the instructions for creating /Library/LaunchDaemons/limit.maxproc.plist. Then reboot. (I first tried withsudo launchctl limit maxfiles 65536 200000 and it did not solve the issue.)

If this solution helps you as well, add a _thumbs up_.

Yeah! Thumbs up!
That's solved my problem!

Thank you.

[lbrnmdev]

After running into the same issue (_too many open files_), I've tried most of the options mentioned in this issue. Unfortunately, this issue does not go to the root of the cause. Instead, the conclusion is: _it resolved itself_.
I was able to fix it, and I can pinpoint a single measure: increase the limit of open files and reboot.
For increasing the limit, see https://wilsonmar.github.io/maximum-limits/ and follow the instructions for creating /Library/LaunchDaemons/limit.maxproc.plist. Then reboot. (I first tried withsudo launchctl limit maxfiles 65536 200000 and it did not solve the issue.)
If this solution helps you as well, add a _thumbs up_.

Yeah! Thumbs up!
That's solved my problem!

Thank you.

I'm running into this issue as well (_too many open files_). In my case I'm trying to run the hello_world example from the esp-idf get-started guide.
I'm running macOS Mojave 10.14.6. My question is has the need to increase the open file limit always been the case or is this a recent issue? That is would using an older version of the esp-idf toolchain solve the issue?

[manuelbl]

Yes, it's clearly related to ESP-IDF v4.x, either to changes in the build system or the newer toolchain (gcc etc.). Going back to ESP-IDF v3.3 and the older toolchain will solve it.

[AniketRane]

After running into the same issue (_too many open files_), I've tried most of the options mentioned in this issue. Unfortunately, this issue does not go to the root of the cause. Instead, the conclusion is: _it resolved itself_.

I was able to fix it, and I can pinpoint a single measure: increase the limit of open files and reboot.

For increasing the limit, see https://wilsonmar.github.io/maximum-limits/ and follow the instructions for creating /Library/LaunchDaemons/limit.maxproc.plist. Then reboot. (I first tried withsudo launchctl limit maxfiles 65536 200000 and it did not solve the issue.)

If this solution helps you as well, add a _thumbs up_.

I am running macOS Mojave version 10.14.2 (18C54) and was having the same issue but using sudo launchctl limit maxfiles 65536 200000 solved it.

Remember: The max files limit is reset at every reboot so make sure you add the command to your ~/.bash_profile

[Francois-Belanger]

Hi, having the same issue in macOS Catalina.

Using the sudo command works, but you need to close and re-open your terminal window to work. (or open a new terminal)

Hope this helps.

[antoinevg]

Same issue in macOS Mojave 10.14.6

Neither the sudo command nor the launchctl solution worked.

For now I've dropped back to an earlier version (c19106876) that doesn't suffer from this problem.

[Francois-Belanger]

Have you tried using a new console after using the sudo command ?

[dobairoland]

Also there is a new version of the toolchain in the master branch (and will be available soon in v4.0 as well). Those who experience this problem could try to update to that and report back whether it solves the issue or not.

[antoinevg]

Tried on new console, tried on reboot etc. etc.

Also, this was running against the latest commit to master branch: fb3d648

[igrr]

Can you please try ulimit -n 1024 (no sudo required, but has to be run in the same console where you are doing the build)

[antoinevg]

Okay, that worked :-)

Some log files from building the examples/get-started/hello_world example against fb3d648.

Log for failing build: fail.log

rm -rf build sdkconfig && idf.py -v build > build.log 2>&1

Log for successful build: success.log

ulimit -n 1024 && rm -rf build sdkconfig && idf.py -v build > build.log 2>&1

Thank you muchly for taking the time to look into this!

[projectgus]

Reopening this as not resolved yet. A fix has been merged on our internal branch, should push to GitHub soon.

[Quentinb]

Running sudo launchctl limit maxfiles 65536 200000 solved it for me on macOS Catalina 10.15.1