Mbed-os: Mbed TLS example hangs running on UBLOX_EVK_ODIN_W2 when built with ARM and IAR compilers

[ghost]

Description

When building the mbed-os-example-tls/tls-client example using the default profile the program hangs when attempting to create a connection.

This happens predominantly with the ARM compiler, and about 20% of the time with IAR compiler.

The problem doesn't occur when building using the debug profile.

Target
UBLOX_EVK_ODIN_W2

Toolchain:
ARM | IAR

Toolchain Versions:

• IAR: Workbench 7.5
• ARM: 5.06

mbed-cli version:
1.2.0

mbed-os sha:
(master) Which as of now is:

https://github.com/ARMmbed/mbed-os/#2f2da779cbf61132dc7eeea32d455053177ff938

Steps to reproduce

Compiling with the release profile enabled:

$  mbed compile -m UBLOX_EVK_ODIN_W2 -t ARM --profile mbed-os/tools/profiles/release.json

Gives the following output before hanging:

Using Ethernet LWIP
Client IP Address is 10.254.100.5
Connecting with developer.mbed.org

Compiling with the debug profile enabled:

$  mbed compile -m UBLOX_EVK_ODIN_W2 -t ARM --profile mbed-os/tools/profiles/debug.json

Gives the correct output:

Using Ethernet LWIP
Client IP Address is 10.254.100.5
Connecting with developer.mbed.org
Starting the TLS handshake...
TLS connection to developer.mbed.org established
Server certificate:
    cert. version     : 3
    serial number     : 65:7B:6D:8D:15:A5:B6:86:87:6B:5E:BC
    issuer name       : C=BE, O=GlobalSign nv-sa, CN=GlobalSign Organization Validation CA - SHA256 - G2
    subject name      : C=GB, ST=Cambridgeshire, L=Cambridge, O=ARM Ltd, CN=*.mbed.com
    issued  on        : 2017-04-03 13:54:02
    expires on        : 2018-05-06 10:31:02
    signed using      : RSA with SHA-256
    RSA key size      : 2048 bits
    basic constraints : CA=false
    subject alt name  : *.mbed.com, mbed.org, *.mbed.org, mbed.com
    key usage         : Digital Signature, Key Encipherment
    ext key usage     : TLS Web Server Authentication, TLS Web Client Authentication
Certificate verification passed

HTTPS: Received 440 chars from server
HTTPS: Received 200 OK status ... [OK]
HTTPS: Received 'Hello world!' status ... [OK]
HTTPS: Received message:

HTTP/1.1 200 OK
Server: nginx/1.11.10
Date: Wed, 06 Sep 2017 15:59:09 GMT
Content-Type: text/plain
Content-Length: 14
Connection: keep-alive
Last-Modified: Fri, 27 Jul 2012 13:30:34 GMT
Accept-Ranges: bytes
Cache-Control: max-age=36000
Expires: Thu, 07 Sep 2017 01:59:09 GMT
X-Upstream-L3: 172.17.0.4:80
X-Upstream-L2: developer-sjc-indigo-2-nginx
Strict-Transport-Security: max-age=31536000; includeSubdomains

Hello world!

[ghost]

cc @andresag01

[0xc0170]

cc @andreaslarssonublox @andreaspeterssonublox

[ghost]

Changing the -Ospace flag to -Otime in release.json profile makes the example run fine _most_ of the time.

[theotherjimmy]

@scartmell-arm I'm wary of anything that works _most_ of the time. That implies that it's really not working, but that something is masking the breakage. Further, I advocate against changing the profiles to "make code work" for the very same reason. If you application has very special needs, make your own profile.

[ghost]

I wasn't advocating it as a fix, just narrowing down what the cause might be.

[theotherjimmy]

Yeah, understood. I was just making sure that bystanders don't take it as a fix.

[kegilbert]

Poked around a bit, looks like it's hanging in this function after calling into netconn_apimsg from
netconn_connect.

printf statements prior to that call cause the example to run normally while blind waits do not.

[kegilbert]

Reran this example on the same hardware for out of boxing, ARMc5 built with mbed-cli still fails in the same manner, however IAR functioned for me (built on the same machine as before when I reproduced it).

Using the mbed-os-5.6.0-rc2 branch for mbed-os and head of master for the tls-client example:
Commits 6e087488e10143222b3b08790eb45600c784a309 and 5432349374c171aaa79e79904aeb13a2624c4eeb respectively which included the move to os.mbed.com

@scartmell-arm have you reran this recently? Wanted to double check with someone else.

---

_ | mbed-cli| _ | _ | Export | _
--- | --- | --- | --- | --- | ---
ARMc5 | GCC_ARM | IAR | make_armc5 | make_gcc_arm | make_iar
FAIL | PASS | PASS | PASS | PASS | PASS

[0xc0170]

cc @andreaslarssonublox @andreaspeterssonublox

Bump, have you been able to reproduce the issue?

[andreaslarssonublox]

@0xc0170 No, but we're preparing a new release of the driver and will look at it later this week.
It seems to use ethernet and thus not using the wi-fi driver. Do you know if this happens on the NUCLEO_F429ZI board as well @scartmell-arm?

[andresag01]

@andreaslarssonublox: We have been testing the tls-client example on NUCLEO_F429ZI and we have not experienced this problem so far. However, the problem seems to be intermittent, so perhaps it is rare enough that it we just have not observed it yet.

[andresag01]

Update: It seems that this is no longer failing on our CI. However, please note that the CI only runs the test once, so it might be that the problem happens much less often. If would be good if someone can take a look at this...

[andreaslarssonublox]

I tested this for the SHA 2f2da779cbf61132dc7eeea32d455053177ff938 mentioned above and got the same faulty behavior. I also tested it for NUCLEO_F429ZI and it works.. One thing that differs is that in the mbed-os version above crypto HW acceleration is enabled for ODIN. It can't be enabled for NUCLEO_F429 since it has no crypto HW acceleration thus explaining why it works on NUCLEO but not ODIN. When the MBEDTLS_CONFIG_HW_SUPPORT define is removed from target.json for ODIN it seems to work for the version above.

I also tested it before and after crypto HW acceleration was disabled in #5080:
Merge PR #5018 43aa6a263c9319e36e996c7876669d78af94c4d9 - faulty
Merge PR #5080 181d7bc1bb3867814238cc72c73a07f5398632eb - works
This test also shows that it is the crypto HW acceleration that makes it malfunction.

I have only verified it with the ARM compiler.

This is most likely related to this issue:
https://github.com/ARMmbed/mbed-os/issues/5079

[ciarmcom]

ARM Internal Ref: IOTSSL-2331

[simonbutcher]

Is this problem still occurring for anyone? This does not occur for us with Mbed OS 5.9/Mbed TLS 2.10.0 on our CI.

Otherwise I propose we close this as 'can't reproduce'.