Mbed-os: FCCE fails to run on CY8CKIT_062_WIFI_BT_PSA target with ARMC6 compiler

[evgenibo]

Description

Factory Configurator Client Example (FCCE) application fails to allocate 1KByte on heap shortly after it starts to execute.
setup details:

• mbed-os: 5.12.3 (#0f959dbe4749c20416236e4fe1dac5692cbe18ab)
• target: CY8CKIT_062_WIFI_BT_PSA
• toolchain: ARMC6

when the application runs with the mentioned above setup, it fails to allocate 1KB on the heap for mbed-trace since the heap size is too small (1 KB in total)
The same application with the same configuration runs without issues on version 5.12.2.
While debugging, I've noticed that some stack and heap management implementations were modified in version 5.12.3 in platform/mbed_retarget.cpp. Those changes break the expected behavior with the target.

steps to reproduce:

1. clone https://github.com/ARMmbed/factory-configurator-client-example
2. edit mbed-os.lib to https://github.com/ARMmbed/mbed-os/#0f959dbe4749c20416236e4fe1dac5692cbe18ab (version 5.12.3)
3. deploy the libraries
4. build: mbed compile --artifact-name fcc-example --app-config mbed_app_psa_conf.json --profile debug -DFCE_SERIAL_INTERFACE -m CY8CKIT_062_WIFI_BT_PSA -t ARM
5. copy fcc-example.hex under ./BUILD/CY8CKIT_062_WIFI_BT_PSA/ARM-DEBUG/ to the target.
6. Observe with debugger that the application fails to allocate buffer for mbed-trace :
   https://github.com/ARMmbed/mbed-trace/blob/master/source/mbed_trace.c#L158

Issue request type

[ ] Question
[ ] Enhancement
[x] Bug

[evgenibo]

@jenia81, @alzix @SenRamakri @teetak01

[ciarmcom]

Internal Jira reference: https://jira.arm.com/browse/MBOCUSTRIA-1214

[0xc0170]

While debugging, I've noticed that some stack and heap management implementations were modified in version 5.12.3 in platform/mbed_retarget.cpp. Those changes break the expected behavior with the target.

What is the exact commit SHA or PR ? Is it this one: https://github.com/ARMmbed/mbed-os/pull/10078 ? I could not find any other recent change.

cc @ccli8

[0xc0170]

@mprse Do calculation look corect in 10078 ?

@evgenibo What is the heap size with and without 10078 ? Checking the calculations there, might produce different numbers. Where is ZI limit set and Stack base symbols set ? Please provide some addition details.

[alzix]

@0xc0170, heap calculations are done (or expected to be done) in runtime. map file has 0x400 size allocated in scatter file.

take a look https://github.com/ARMmbed/mbed-os/blob/master/platform/mbed_retarget.cpp#L974
i suspect something got broken in this part of the code.

[ccli8]

@0xc0170 #10078 takes ARM_LIB_HEAP into consideration for correcting heap initialization logic if ARM_LIB_HEAP is defined in .sct file. CY8CKIT_062_WIFI_BT_PSA defines ARM_LIB_HEAP in cy8c6xx7_cm4_dual.sct but its size is only 1KiB (if I don't get wrong). The solution would be to fix HEAP section size in startup_psoc6_01_cm4.S.

; Application heap area (HEAP)
ARM_LIB_HEAP  +0
{ 
    * (HEAP) 
}

[0xc0170]

Lesson learned, the fix should have been postponed to 5.13.

@evgenibo Can you send a fix please, to be Incorporated in the next release?

[jenia81]

@0xc0170 I believe you should request it from Cypress guys?
Can we have some temporary fix to unblock PDMC release with 5.12.3?

[0xc0170]

@0xc0170 I believe you should request it from Cypress guys?

+1, not yet in thread, cc @ARMmbed/team-cypress

Can we have some temporary fix to unblock PDMC release with 5.12.3?

Please talk to the management if our next release is not sufficient (5.13.0 ), code freeze is at the end of this week.

[evgenibo]

cy8c6xx7_cm4_dual.sct and startup_psoc6_01_cm4.S didn't change in transition from 5.12.2 to 5.12.3 (or in the last couple of months) and the heap size was configured correctly in the previous versions

[0xc0170]

cy8c6xx7_cm4_dual.sct and startup_psoc6_01_cm4.S didn't change in transition from 5.12.2 to 5.12.3 (or in the last couple of months) and the heap size was configured correctly in the previous versions

Correct, the fix introduced using the symbol from a linker script, assuming the symbol is correctly defined. This issue is valid, reading the ARMCC configuration - we can see we ignored previously 2 regions memory layout. It was fixed and this introduced a bug in some cypress targets. This should be addressed in a new pull request.
I don't find an issue in the current memory layout (following ARMCC guidelines referenced below about one/two region memory setup).

To clarify, my understanding:

1. As it previously was:

heap base = RAM1 ZI limit; 
heap limit = Stack ZI base;

1. Now it is:

If ARM_LIB_STACK/ARM_LIB_HEAP are not defined (one region memory), same as 1. . However, if these 2 are defined, two memory region is used, and heap base/limit are using linker symbols values (ARM_LIB_HEAP base and limit - Note, stack is in another region).

@ccli8 Correct?

As result, cypress target in question here has these symbols defined, as http://www.keil.com/support/man/docs/ARMLINK/ARMLINK_pge1362065977713.htm specifies it, therefore these symbols are used - two region memory. Related fix is also https://github.com/ARMmbed/mbed-os/pull/6985.

Looking at the startup file (reference above):

                IF :DEF:__HEAP_SIZE
Heap_Size       EQU     __HEAP_SIZE
                ELSE
Heap_Size       EQU     0x00000400
                ENDIF

This symbol is later used in __user_initial_stackheap that is defined also in the startup (quick search returns actually few other targets as well that might have this problem). I'll list them here, once this issue is fixed, the rest should follow:

mbed-os\targets\TARGET_ARM_SSG\TARGET_IOTSS\TARGET_IOTSS_BEID\device\TOOLCHAIN_ARM_STD\startup_MPS2.S:
  341  
  342                  IMPORT  __use_two_region_memory
  343:                 EXPORT  __user_initial_stackheap
  344  
  345: __user_initial_stackheap PROC
  346                  LDR     R0, =  Heap_Mem
  347                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_ARM_SSG\TARGET_MPS2\TARGET_MPS2_M0\device\TOOLCHAIN_ARM_STD\startup_MPS2.S:
  251  
  252                  IMPORT  __use_two_region_memory
  253:                 EXPORT  __user_initial_stackheap
  254  
  255: __user_initial_stackheap PROC
  256                  LDR     R0, =  Heap_Mem
  257                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_ARM_SSG\TARGET_MPS2\TARGET_MPS2_M0P\device\TOOLCHAIN_ARM_STD\startup_MPS2.S:
  251  
  252                  IMPORT  __use_two_region_memory
  253:                 EXPORT  __user_initial_stackheap
  254  
  255: __user_initial_stackheap PROC
  256                  LDR     R0, =  Heap_Mem
  257                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_ARM_SSG\TARGET_MPS2\TARGET_MPS2_M3\device\TOOLCHAIN_ARM_STD\startup_MPS2.S:
  271  
  272                  IMPORT  __use_two_region_memory
  273:                 EXPORT  __user_initial_stackheap
  274  
  275: __user_initial_stackheap PROC
  276                  LDR     R0, =  Heap_Mem
  277                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_ARM_SSG\TARGET_MPS2\TARGET_MPS2_M4\device\TOOLCHAIN_ARM_STD\startup_MPS2.S:
  271  
  272                  IMPORT  __use_two_region_memory
  273:                 EXPORT  __user_initial_stackheap
  274  
  275: __user_initial_stackheap PROC
  276                  LDR     R0, =  Heap_Mem
  277                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_ARM_SSG\TARGET_MPS2\TARGET_MPS2_M7\device\TOOLCHAIN_ARM_STD\startup_CMSDK_CM7.S:
  271  
  272                  IMPORT  __use_two_region_memory
  273:                 EXPORT  __user_initial_stackheap
  274  
  275: __user_initial_stackheap PROC
  276                  LDR     R0, =  Heap_Mem
  277                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_Cypress\TARGET_PSOC6\TARGET_CY8CKIT_062_4343W\device\TOOLCHAIN_ARM\startup_psoc6_02_cm4.S:
  736                  ELSE
  737  
  738:                 EXPORT  __user_initial_stackheap
  739  
  740: __user_initial_stackheap PROC
  741                  LDR     R0, =Heap_Mem
  742                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_Cypress\TARGET_PSOC6\TARGET_CY8CKIT_062_BLE\device\TOOLCHAIN_ARM\startup_psoc6_01_cm4.S:
  673                  ELSE
  674  
  675:                 EXPORT  __user_initial_stackheap
  676  
  677: __user_initial_stackheap PROC
  678                  LDR     R0, =Heap_Mem
  679                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_Cypress\TARGET_PSOC6\TARGET_CY8CKIT_062_WIFI_BT\device\TARGET_MCU_PSOC6_M0\TOOLCHAIN_ARM\startup_psoc6_01_cm0plus.S:
  296                  ELSE
  297  
  298:                 EXPORT  __user_initial_stackheap
  299  
  300: __user_initial_stackheap PROC
  301                  LDR     R0, =Heap_Mem
  302                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_Cypress\TARGET_PSOC6\TARGET_CY8CKIT_062_WIFI_BT\device\TARGET_MCU_PSOC6_M4\TOOLCHAIN_ARM\startup_psoc6_01_cm4.S:
  673                  ELSE
  674  
  675:                 EXPORT  __user_initial_stackheap
  676  
  677: __user_initial_stackheap PROC
  678                  LDR     R0, =Heap_Mem
  679                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_Cypress\TARGET_PSOC6\TARGET_CY8CMOD_062_4343W\device\TOOLCHAIN_ARM\startup_psoc6_02_cm4.S:
  736                  ELSE
  737  
  738:                 EXPORT  __user_initial_stackheap
  739  
  740: __user_initial_stackheap PROC
  741                  LDR     R0, =Heap_Mem
  742                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_Cypress\TARGET_PSOC6\TARGET_CYW943012P6EVB_01\device\TARGET_MCU_PSOC6_M0\TOOLCHAIN_ARM\startup_psoc6_01_cm0plus.S:
  296                  ELSE
  297  
  298:                 EXPORT  __user_initial_stackheap
  299  
  300: __user_initial_stackheap PROC
  301                  LDR     R0, =Heap_Mem
  302                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_Cypress\TARGET_PSOC6\TARGET_CYW943012P6EVB_01\device\TARGET_MCU_PSOC6_M4\TOOLCHAIN_ARM\startup_psoc6_01_cm4.S:
  673                  ELSE
  674  
  675:                 EXPORT  __user_initial_stackheap
  676  
  677: __user_initial_stackheap PROC
  678                  LDR     R0, =Heap_Mem
  679                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_GigaDevice\TARGET_GD32F4XX\device\TOOLCHAIN_ARM_STD\startup_gd32f450.S:
  442  
  443                  IMPORT  __use_two_region_memory
  444:                 EXPORT  __user_initial_stackheap
  445  
  446: __user_initial_stackheap PROC
  447                  LDR     R0, =  Heap_Mem
  448                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_NXP\TARGET_MCUXpresso_MCUS\TARGET_LPC55S69\TARGET_M33_S\device\TOOLCHAIN_ARMC6\startup_LPC55S69_cm33_core0.S:
  159                  BLX     R0
  160                  MRS     R0, control    ; Get control value
  161:                 ORR     R0, R0, #2     ; Select switch to PSP, which will be set by __user_initial_stackheap
  162                  MSR     control, R0
  163                  ;CPSIE   I               ; Unmask interrupts

mbed-os\targets\TARGET_ONSEMI\TARGET_NCS36510\device\TOOLCHAIN_ARM\startup_NCS36510.S:
  195                  ENDP
  196  
  197:                 EXPORT  __user_initial_stackheap
  198                  IMPORT  |Image$$ARM_LIB_HEAP$$Base|
  199                  IMPORT  |Image$$ARM_LIB_HEAP$$ZI$$Limit|
  200  
  201: __user_initial_stackheap PROC
  202                  LDR     R0, = |Image$$ARM_LIB_HEAP$$Base|
  203                  LDR     R2, = |Image$$ARM_LIB_HEAP$$ZI$$Limit|

mbed-os\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFM32GG\device\TARGET_1024K\TOOLCHAIN_ARM_STD\startup_efm32gg.S:
  271  
  272                  IMPORT  __use_two_region_memory
  273:                 EXPORT  __user_initial_stackheap
  274  
  275: __user_initial_stackheap PROC
  276                  LDR     R0, =  Heap_Mem
  277                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFM32GG11\device\TOOLCHAIN_ARM_STD\startup_efm32gg11.S:
  359  
  360                  IMPORT  __use_two_region_memory
  361:                 EXPORT  __user_initial_stackheap
  362  
  363: __user_initial_stackheap PROC
  364                  LDR     R0, =  Heap_Mem
  365                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFM32LG\device\TARGET_256K\TOOLCHAIN_ARM_STD\startup_efm32lg.S:
  272  
  273                  IMPORT  __use_two_region_memory
  274:                 EXPORT  __user_initial_stackheap
  275  
  276: __user_initial_stackheap PROC
  277                  LDR     R0, =  Heap_Mem
  278                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFM32PG\device\TARGET_256K\TOOLCHAIN_ARM_STD\startup_efm32pg1b.S:
  237  
  238                  IMPORT  __use_two_region_memory
  239:                 EXPORT  __user_initial_stackheap
  240  
  241: __user_initial_stackheap PROC
  242                  LDR     R0, =  Heap_Mem
  243                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFM32PG12\device\TOOLCHAIN_ARM_STD\startup_efm32pg12b.S:
  292  
  293                  IMPORT  __use_two_region_memory
  294:                 EXPORT  __user_initial_stackheap
  295  
  296: __user_initial_stackheap PROC
  297                  LDR     R0, =  Heap_Mem
  298                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFM32WG\device\TARGET_256K\TOOLCHAIN_ARM_STD\startup_efm32wg.S:
  274  
  275                  IMPORT  __use_two_region_memory
  276:                 EXPORT  __user_initial_stackheap
  277  
  278: __user_initial_stackheap PROC
  279                  LDR     R0, =  Heap_Mem
  280                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFR32MG1\device\TOOLCHAIN_ARM_STD\startup_efr32mg1p.S:
  257  
  258                  IMPORT  __use_two_region_memory
  259:                 EXPORT  __user_initial_stackheap
  260  
  261: __user_initial_stackheap PROC
  262                  LDR     R0, =  Heap_Mem
  263                  LDR     R1, =(Stack_Mem + Stack_Size)

mbed-os\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFR32MG12\device\TOOLCHAIN_ARM_STD\startup_efr32mg12p.S:
  306  
  307                  IMPORT  __use_two_region_memory
  308:                 EXPORT  __user_initial_stackheap
  309  
  310: __user_initial_stackheap PROC
  311                  LDR     R0, =  Heap_Mem
  312                  LDR     R1, =(Stack_Mem + Stack_Size)

[0xc0170]

To summarize:

Startup file should be fixed (possibly as well linker script files to define proper values for heap region, not certain about this, to check) , not to set this hard defined value for heap in ARMCC startup files.

@ARMmbed/team-cypress Please review our findings. This should be addressed by end of this week.

cc @maclobdell

[0xc0170]

Whoever follows this thread, reverting the "use 2 regions memory" would be a workaround:

Removing these lines from mbed_retarget.cpp file as below or just remove ARM_LIB_HEAP from scatter file.

// this check to be removed as a workaround

    // Fix heap if ARM_LIB_HEAP is defined
    if (Image$$ARM_LIB_HEAP$$ZI$$Length) {
        heap_base = (uint32_t) Image$$ARM_LIB_HEAP$$ZI$$Base;
        heap_limit = (uint32_t) Image$$ARM_LIB_HEAP$$ZI$$Limit;
    }

I would go with removing ARM_LIB_HEAP from a scatter file.

@evgenibo you can apply this to continue with your app now (should not block you anymore).

[ccli8]

@0xc0170 Yes. #10078 is to address heap initialization error in two region model. If ARM_LIB_HEAP is not defined in scatter file, it goes as before. If ARM_LIB_HEAP defined, it must be correct for heap initialization.

[evgenibo]

@0xc0170,
just to make it clear, do you want me to make PR with ARM_LIB_HEAP removal?

[0xc0170]

Lets wait for @ARMmbed/team-cypress to review this. The last resort would be to just remove the symbols, not the best fix as they are obviously defined there to be used.
I would rather remove __user_initial_stackheap - this is legacy function, and we use __user_setup_stackheap. That might fix this issue.

My earlier suggestions was just sharing to unblock any ongoing work.

[hennadiykytsun]

As I understand the correct solution will be add the following to the asm file:
IMPORT __use_two_region_memory
IMPORT __user_setup_stackheap

and remove the __user_initial_stackheap with it implementation from the asm file (the following peace of the code):
EXPORT __user_initial_stackheap

__user_initial_stackheap PROC
LDR R0, =Heap_Mem
LDR R1, =(Stack_Mem + Stack_Size)
LDR R2, =(Heap_Mem + Heap_Size)
LDR R3, =Stack_Mem
BX LR
ENDP

[hennadiykytsun]

I see that documentation tells that __user_setup_stackheap needs to be reimplemented in assembler. So we can't import it. So, the proper solution is following:

IMPORT __use_two_region_memory
EXPORT __user_setup_stackheap

__user_setup_stackheap PROC
LDR SP, =Stack_Mem
LDR R0, =Heap_Mem
LDR R2, =(Heap_Mem + Heap_Size)
LDR R3, =(Stack_Mem + Stack_Size)
BX LR
ENDP

[0xc0170]

Mbed OS provides __user_setup_stackheap in mbed_retarget.cpp file. Shouldn't we just remove __user_initial_stackheap in the startup ? This function should not be there as we use __user_setup_stackheap

[0xc0170]

@ARMmbed/team-cypress Any update for this issue?

[mykz-cy]

Kostia,
Is this something you are looking into?

From: Martin Kojtal [e-mail]
Reply-To: ARMmbed/mbed-os [e-mail]
Date: Monday, May 27, 2019 at 2:59 PM
To: ARMmbed/mbed-os [e-mail]
Cc: Mykhailo Zelyanovskyy [e-mail], Team mention [e-mail]
Subject: Re: [ARMmbed/mbed-os] FCCE fails to run on CY8CKIT_062_WIFI_BT_PSA target with ARMC6 compiler (#10613)

@ARMmbed/team-cypresshttps://github.com/orgs/ARMmbed/teams/team-cypress Any update for this issue?

—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/ARMmbed/mbed-os/issues/10613?email_source=notifications&email_token=AKIQYYX2MK2PTBHW5KS6CCLPXPEIPA5CNFSM4HN425PKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWJTUTA#issuecomment-496187980, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKIQYYVDHWNUK227U2Y27NTPXPEIPANCNFSM4HN425PA.

This message and any attachments may contain confidential information from Cypress or its subsidiaries. If it has been received in error, please advise the sender and immediately delete this message.

[jenia81]

Which Mbed Os version will have the fix? Currently, we disabled Cypress testing in our Jenkins, but it's a regression introduced in Mbed Os 5.12.2

[0xc0170]

Which Mbed Os version will have the fix? Currently, we disabled Cypress testing in our Jenkins, but it's a regression introduced in Mbed Os 5.12.2

5.13.0 is the next release

[hennadiykytsun]

Hello @evgenibo

I can't reproduce your steps to reproduce.
The file mbed_app_psa_conf.json is absent.
Could you please attach it?

Thanks.

[evgenibo]

hi @hennadiykytsun
just rename mbed_app_psa.json to mbed_app_psa_conf.json

[jenia81]

5.13.0-RC1 is out and doesn't include fix yet. Which RC will have the fix?
Currently, we have regression in PDMC - we can;t support Cypress board

[0xc0170]

As referenced above, this also was discovered in our nightly tests.

5.13.0-RC1 is out and doesn't include fix yet. Which RC will have the fix?

I hope rc2 will have this fix. @hennadiykytsun mentioned testing the fix the previous week in 10731 issue I created for nightly failures.

@hennadiykytsun It would be great to have this fix in PR today to have it in 5.13.0-rc2

[hennadiykytsun]

@evgenibo could you please check if test case is works for you with this PR?. I am able to compile it, but I can't debug metioned project.

Thank you.

[0xc0170]

@jenia81 Can you also evaluate the PR referenced?

[evgenibo]

@hennadiykytsun
The test case works with this PR. It fixes the issue

[hennadiykytsun]

@evgenibo , thank you so much for the verification.

[0xc0170]

Fixed via 10747