Esp32-snippets: Sketch too big. BLEDevice::init() consumes 713 352 bytes of program storage

[william-ferguson-au]

Including

BLEDevice::init("SomeDevcieName");

causes the program storage size of my sketch to increase by 713 352 bytes (and dynamic memory to jump by 51 272 bytes).

This is with ESP32_BLE_Arduino version 0.4.7

That doesn't leave much room for any other libraries.
Is there any way to reduce the program storage requirements?

[nkolban]

Howdy,
The ESP-IDF environment for building applications only links into the executables those functions which are needed. If you write an application that doesn't use function XYZ() then your executable won't cause the inclusion of the code of XYZ(). As such, the moment you do use an additional function, the size of your executables increase.

I checked a compilation of an ESP-IDF application outside of the Arduino environment and then ran "make size-components" and obtained the following list:

Total sizes:
 DRAM .data size:    8680 bytes
 DRAM .bss  size:   37952 bytes
Used static DRAM:   46632 bytes (  68568 available, 40.5% used)
Used static IRAM:   50468 bytes (  80604 available, 38.5% used)
      Flash code:  592918 bytes
    Flash rodata:  280480 bytes
Total image size:~ 932546 bytes (.bin may be padded larger)
Per-archive contributions to ELF file:
            Archive File DRAM .data & .bss   IRAM Flash code & rodata   Total
                 libbt.a        521  25166    546     260132   140776  427141
             libstdc++.a        148   4284      0     117756    69180  191368
                  libc.a          0      4     12      92768     5463   98247
          libcpp_utils.a          2     21      0      28278    41706   70007
           libbtdm_app.a        258   4634  11883      28294     2388   47457
                libphy.a       1334    864   4584      28418        0   35200
              libesp32.a       1870   1151   5585       7616     5434   21656
           libfreertos.a       2092    728  10320          0     1038   14178
          libnvs_flash.a          0     32      0       7356     5587   12975
                libgcc.a          4     20    104      10017      848   10993
               libheap.a       1287     30   3864       1011     1009    7201
          libspi_flash.a         32    317   4709        737      511    6306
                libsoc.a        785      8   3253          0     1210    5256
             libdriver.a         40      8      0        790     3408    4246
                libvfs.a         40     63      0       2123      409    2635
                librtc.a          0      4   2560          0        0    2564
            libpthread.a         16      8    176        830      968    1998
             libnewlib.a        152    252    774        431      109    1718
               libmain.a         16      0      0        783      557    1356
                libcxx.a          8     16      0        630      537    1191
         libapp_update.a          0      0      0        126      844     970
                liblog.a          8    268    418         77        0     771
                libhal.a          0      0    515          0       32     547
            libcoexist.a         32     10    188         17        0     247
                  libm.a          0      0     92          0        0      92
            libmbedtls.a          0      0      0          0        0       0
 libbootloader_support.a          0      0      0          0        0       0

Two lines are of particular interest to me:

            Archive File DRAM .data & .bss   IRAM Flash code & rodata   Total
                 libbt.a        521  25166    546     260132   140776  427141
           libbtdm_app.a        258   4634  11883      28294     2388   47457

With just a finger in the wind, these seem to be showing me that using Bluetooth is not "cheap" from a resource perspective. 500K of executable/ro-data and 50K of RAM seems to be on the right order.

See also:

• Memory consumption of bluetooth is enormous

I think from an ESP32 perspective, when we start to use ESP32 ... the cost "is what it is" and there is little that this library (which is a wrapper around the base ESP32 BLE libraries) can do.

[ghost]

@william-ferguson-au

1) If you don't need the String representations of i.e. members, characteristics, descriptors, ... you can comment out the long lists of if in the BLEUtils.cpp (or move it to seperate cpp files) . See: https://github.com/nkolban/esp32-snippets/issues/100 (PROGMEM doesn't help/is needed with the ESP32)

This saved me in a fast test 12kb of Flash (depents on your use of BLEUtils functions.)

Der Sketch verwendet 1603178 Bytes (76%) des Programmspeicherplatzes. Das Maximum sind 2097152 Bytes.
Der Sketch verwendet 1591710 Bytes (75%) des Programmspeicherplatzes. Das Maximum sind 2097152 Bytes.

2) you can increase the available flash in the Arduino IDE (default is ~1Mb) up to ~1,5Mb (with OTA, 2x) or ~3,x (without OTA, 1x) in the files \hardware\espressif\esp32\boards.txt and \espressif\esp32\tools\partitions\default.csv (As you can see I have 2MB)

3) The ram is the bigger problem, here an optimization is hard. One thing is to delete the scan results (all be kept in memory as long the BLE object lives) or, if you have control over the GATT to use less Services, Chracteristics, ... and keep the thing simple.

Other things I check is to find and fix memory lecks or use the idf direct.

Peter

[william-ferguson-au]

So I tried changing the partition sizes by modifying /expressif/esp32/tools/partitions/default.csv to

# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xe000,  0x2000,
app0,     app,  ota_0,   0x10000, 0x190000,
app1,     app,  ota_1,   0x1A0000,0x190000,
eeprom,   data, 0x99,    0x330000,0x1000,
spiffs,   data, spiffs,  0x331000,0x0CF000,

Which should have given the 2 app partititions 1,638,400 bytes, which is more than enough for the app including BT. But when compiling in Arduino IDE I still get

Sketch too big
Sketch uses 1485722 bytes (113%) of program storage space. Maximum is 1310720 bytes.

So it doesn't look like the partititions are being respected.

I then tried also altering
esp32.upload.maximum_size=1638400
in /expressif/esp32/boards.txt

but get the same result.

Am I missing something trying to modifying the partitition size?

[ghost]

Which ESP32 hw board in the Arduino IDE have you selected? The generic ?

[ghost]

The "Sketch uses zzzz bytes (1xx%) of program storage space. Maximum is xxxxxx bytes." warning results from a config error in the boards.txt (if I remember correctly)

An error in the default.csv at the time starting up the ESP32

[william-ferguson-au]

@pit001 you're a lifesaver Thank you.
I'm using the AdaFruit ESP32 Feather.

[ghost]

May you like to write down your changes for other Arduino users, thx

[william-ferguson-au]

I changed the size of app0 and app1 in default.csv as outlined above.

And I set

featheresp32.upload.maximum_size=1638400

in boards.txt

[hdavid]

Hi @pit001

I have a sketch that is using both various lib (wifi, ble, mdns etc..) and adjusting the partition size seems to be not enough (misses a few kb).

Is the new limit you proposed for the partition size a hard maximum ? or can it be increased further ? what would be the possible drawbacks of doing so ?

[wegunterjr]

@hdavid try the partition layout mentioned in this link: https://desire.giesecke.tk/index.php/2018/01/30/change-partition-size/
it has a large program partition, but little eeprom and spiffs...
I am also using WiFi, BLE, SPIFFS, etc. Would like to have more SPIFFS, but need the BLE with notify, etc.

[RJ-Fynydd]

@william-ferguson-au I just wanted to say that your partition table helped me out a lot. I have spent hours trying to create my own and getting nowhere, but yours was perfect for my project. 😄
Thank you for sharing!

[william-ferguson-au]

Glad to be of service :-)

[duncanhames]

For anyone else searching for this as I was, there are many options in Arduino IDE, under Tools-->Partition Scheme, and they are nicely labelled.

[lfssp]

Thanks @duncanhames . I've changed my board from ESP32 NodeMCU-32S to ESP32 Dev Module and now I have the option "Partition Scheme" to adjust. Great tip!