Mbed-os: Not supported test floods the log file with errors

[deepikabhavnani]

Description

Tests for specific features are skipped if not supported by target. Few such tests flood the log files for not supported cases.

Log:

BUILD/tests/VBLUNO52/GCC_ARM/features/storage/TESTS/kvstore/securestore_whitebox/features/storage/TESTS/kvstore/securestore_whitebox/main.d -o BUILD/tests/VBLUNO52/GCC_ARM/features/storage/TESTS/kvstore/securestore_whitebox/features/storage/TESTS/kvstore/securestore_whitebox/main.o ./features/storage/TESTS/kvstore/securestore_whitebox/main.cpp
[Error] main.cpp@38,2: #error [NOT_SUPPORTED] Kvstore API tests run only on K64F devices
[Error] main.cpp@42,2: #error [NOT_SUPPORTED] KVStore & SecureStore need to be enabled for this test
[Error] main.cpp@140,5: 'SecureStore' was not declared in this scope
[Error] main.cpp@140,18: 'sec_kv' was not declared in this scope
[Error] main.cpp@140,31: expected type-specifier before 'SecureStore'
[Error] main.cpp@393,12: type '<type error>' argument given to 'delete', expected pointer
[DEBUG] Return: 1
[DEBUG] Output: ./features/storage/TESTS/kvstore/securestore_whitebox/main.cpp:38:2: error: #error [NOT_SUPPORTED] Kvstore API tests run only on K64F devices
[DEBUG] Output:  #error [NOT_SUPPORTED] Kvstore API tests run only on K64F devices
[DEBUG] Output:   ^~~~~
[DEBUG] Output: ./features/storage/TESTS/kvstore/securestore_whitebox/main.cpp:42:2: error: #error [NOT_SUPPORTED] KVStore & SecureStore need to be enabled for this test
[DEBUG] Output:  #error [NOT_SUPPORTED] KVStore & SecureStore need to be enabled for this test
[DEBUG] Output:   ^~~~~
[DEBUG] Output: ./features/storage/TESTS/kvstore/securestore_whitebox/main.cpp: In function 'void white_box_test()':
[DEBUG] Output: ./features/storage/TESTS/kvstore/securestore_whitebox/main.cpp:140:5: error: 'SecureStore' was not declared in this scope
[DEBUG] Output:      SecureStore *sec_kv = new SecureStore(ul_kv, rbp_kv);
[DEBUG] Output:      ^~~~~~~~~~~
[DEBUG] Output: ./features/storage/TESTS/kvstore/securestore_whitebox/main.cpp:140:18: error: 'sec_kv' was not declared in this scope
[DEBUG] Output:      SecureStore *sec_kv = new SecureStore(ul_kv, rbp_kv);
[DEBUG] Output:                   ^~~~~~
[DEBUG] Output: ./features/storage/TESTS/kvstore/securestore_whitebox/main.cpp:140:31: error: expected type-specifier before 'SecureStore'
[DEBUG] Output:      SecureStore *sec_kv = new SecureStore(ul_kv, rbp_kv);
[DEBUG] Output:                                ^~~~~~~~~~~
[DEBUG] Output: ./features/storage/TESTS/kvstore/securestore_whitebox/main.cpp:393:12: error: type '<type error>' argument given to 'delete', expected pointer
[DEBUG] Output:      delete sec_kv;
[DEBUG] Output:             ^~~~~~

[Error] main.cpp@18,2: #error [NOT_SUPPORTED] QSPI not supported for this target
[Error] main.cpp@32,2: #error [NOT_SUPPORTED] QSPI test not supported for this target
[Error] qspi_test_utils.h@33,20: 'qspi_bus_width_t' has not been declared
[Error] qspi_test_utils.h@33,49: 'qspi_bus_width_t' has not been declared
[Error] qspi_test_utils.h@33,78: 'qspi_bus_width_t' has not been declared
[Error] qspi_test_utils.h@34,20: 'qspi_bus_width_t' has not been declared
[Error] qspi_test_utils.h@34,48: 'qspi_address_size_t' has not been declared
[Error] qspi_test_utils.h@34,79: 'qspi_alt_size_t' has not been declared
[Error] qspi_test_utils.h@41,5: 'qspi_command_t' does not name a type
[Error] qspi_test_utils.h@44,5: 'qspi_command_t' does not name a type
[Error] qspi_test_utils.h@48,5: 'qspi_t' does not name a type
[Error] qspi_test_utils.h@130,1: 'qspi_status_t' does not name a type
[Error] qspi_test_utils.h@131,1: 'qspi_status_t' does not name a type
[Error] qspi_test_utils.h@137,1: 'qspi_status_t' does not name a type
[Error] qspi_test_utils.h@138,1: 'qspi_status_t' does not name a type
[Error] qspi_test_utils.h@142,1: 'qspi_status_t' does not name a type
[Error] qspi_test_utils.h@143,1: 'qspi_status_t' does not name a type
[Error] qspi_test_utils.h@145,1: 'qspi_status_t' does not name a type
[Error] qspi_test_utils.h@146,1: 'qspi_status_t' does not name a type
[Error] qspi_test_utils.h@148,1: 'qspi_status_t' does not name a type
[Error] qspi_test_utils.h@150,23: 'qspi_bus_width_t' was not declared in this scope
[Error] qspi_test_utils.h@150,52: 'qspi_bus_width_t' was not declared in this scope
[Error] qspi_test_utils.h@150,81: 'qspi_bus_width_t' was not declared in this scope
[Error] qspi_test_utils.h@150,108: expression list treated as compound expression in initializer [-fpermissive]
[Error] qspi_test_utils.h@151,19: 'qspi_bus_width_t' was not declared in this scope
[Error] qspi_test_utils.h@151,48: 'qspi_bus_width_t' was not declared in this scope
[Error] qspi_test_utils.h@151,77: 'qspi_bus_width_t' was not declared in this scope
[Error] qspi_test_utils.h@151,104: expression list treated as compound expression in initializer [-fpermissive]
[Error] qspi_test_utils.h@152,19: 'qspi_bus_width_t' was not declared in this scope
[Error] qspi_test_utils.h@152,48: 'qspi_bus_width_t' was not declared in this scope
[Error] qspi_test_utils.h@152,77: 'qspi_bus_width_t' was not declared in this scope
[Error] qspi_test_utils.h@152,104: expression list treated as compound expression in initializer [-fpermissive]
[Error] main.cpp@48,25: 'QSPI_PAGE_SIZE' was not declared in this scope
[Error] main.cpp@48,25: 'QSPI_PAGE_SIZE' was not declared in this scope
[Error] main.cpp@83,46: 'QSPI_SECTOR_COUNT' was not declared in this scope
[Error] main.cpp@83,67: 'QSPI_SECTOR_SIZE' was not declared in this scope
[Error] main.cpp@9

-----------
**~4500 lines of failing log**
------------



md5-2d1c3dc3113f507a4214f38c9def92f8



[Error] main.cpp@23,2: #error [NOT_SUPPORTED] SIM pin code is needed. Skipping this build.
[DEBUG] Return: 1
[DEBUG] Output: ./features/cellular/TESTS/api/cellular_device/main.cpp:23:2: error: #error [NOT_SUPPORTED] SIM pin code is needed. Skipping this build.
[DEBUG] Output:  #error [NOT_SUPPORTED] SIM pin code is needed. Skipping this build.
[DEBUG] Output:   ^~~~~
Building project wifi (VBLUNO52, GCC_ARM)

Issue request type

[ ] Question
[ ] Enhancement
[X] Bug

[ciarmcom]

Internal Jira reference: https://jira.arm.com/browse/MBOCUSTRIA-854

[Tharazi97]

I have simple idea of changing test files like in this example: 53c44678db2e15141a1b81a754054b9cef500811

Would this meet your expectation of solving this problem?

@fkjagodzinski @maciejbocianski @deepikabhavnani

[fkjagodzinski]

@Tharazi97, I like that. :+1:

@jamesbeyond, @SeppoTakalo, what do you guys think; shall we update all greentea tests as proposed above?

[SeppoTakalo]

I think it should be OK to update testcases all look like

#if !CONDITION
#error [NOT_SUPPORTED] This test requires CONDITION
#else
#include "CONDITION_or_something.h"
...
testcases
...
#endif // CONDITION

One #error is enough.

Build errors and missing headers hide the fact that this test cannot be build if specific features are missing.

[jamesbeyond]

I think Seppo's solution would be appropriate

[Tharazi97]

I think in some cases multiple errors are more clear, like in example below. Should I leave these directives as they are, or should I truncate them, so they give only one error?

#if !defined(MBED_CONF_APP_ECHO_SERVER)       || \
    !defined(MBED_CONF_APP_ECHO_SERVER_TRACE) || \
    !defined(MBED_CONF_APP_WIFI_SCAN)
#error [NOT_SUPPORTED] Requires parameters from mbed_app.json
#else

#define ETHERNET 1
#define WIFI 2

#if MBED_CONF_TARGET_NETWORK_DEFAULT_INTERFACE_TYPE != ETHERNET && \
    MBED_CONF_TARGET_NETWORK_DEFAULT_INTERFACE_TYPE != WIFI
#error [NOT_SUPPORTED] Either wifi or ethernet testing need to be enabled
#else

#if (MBED_CONF_TARGET_NETWORK_DEFAULT_INTERFACE_TYPE == WIFI) && \
    !defined(TARGET_UBLOX_EVK_ODIN_W2)      && \
    !defined(TARGET_REALTEK_RTL8195AM)      && \
    !defined(TARGET_MTB_ADV_WISE_1530)      && \
    !defined(TARGET_MTB_USI_WM_BN_BM_22)    && \
    !defined(TARGET_MTB_MXCHIP_EMW3166)     && \
    !defined(TARGET_MTB_UBLOX_ODIN_W2)      && \
    !defined(TARGET_UNO_91H)
#error [NOT_SUPPORTED] Wifi tests are not valid for the target
#else