Mbed-cli: mbed test cannot target custom targets

[AlessandroA]

Custom targets are a feature of the config system which is (to my knowledge) not supported by mbed test.

Why it is needed

The use of custom targets is a requirement for uVisor testing. We use custom targets to test variations/configurations of the original target with different uVisor flavors.

For more information, please read this document.

@screamerbg @bogdanm
@PrzemekWirkus @mazimkhan @adbridge

---

Steps to reproduce

Initial setup: A top-level folder with a TESTS folder with a test in it, and a link to mbed OS:

~/reproducible
  |__ mbed-os.lib
  |__ TESTS
      |__ test
          |__ test.cpp

Deploy the OS:

$ cd ~/reproducible
$ mbed deploy

Add a custom target definition:

$ echo '{
    "custom_targets": {
        "K64F_CUSTOM": {
            "inherits": ["K64F"],
            "extra_labels_add": ["K64F"],
            "features_add": ["UVISOR"]
        }
    }
}' > mbed_app.json

And try to build and run the test:

$ mbed test -m K64F_CUSTOM -t GCC_ARM

You will get the following error:

KeyError: 'K64F_CUSTOM'
[ERROR] 'K64F_CUSTOM'

[bogdanm]

Can you please try to manually update your mbedmicro/mbed (inside mbed-os/core) to the latest version and try again?

[AlessandroA]

@bogdanm The output is different but the outcome is the same:

usage: test.py [-h] [-m MCU] [-t TOOLCHAIN] [--color] [-c] [-o OPTIONS]
               [-D MACROS] [-j JOBS] [--source SOURCE_DIR] [--build BUILD_DIR]
               [-l] [-p PATHS] [-f FORMAT] [--continue-on-build-fail]
               [-n NAMES] [--test-spec TEST_SPEC]
               [--build-report-junit BUILD_REPORT_JUNIT] [-v]
test.py: error: argument -m/--mcu: K64F_CUSTOM is not a supported MCU. Supported MCUs are:
ARCH_BLE,              ARCH_BLE_BOOT,         ARCH_BLE_OTA,
...

Just to clarify, my mbed_app.json is in the top-level folder, that also contains TESTS (not the mbed tests, a new folder).

[bogdanm]

OK, thank you. I'll look into this.

[bogdanm]

https://github.com/mbedmicro/mbed/pull/2196 is supposed to fix this. Please try it if you can.

[AlessandroA]

I can confirm that mbedmicro/mbed#2196 fixes the issue both for apps and tests. Thanks for the fix!

[screamerbg]

Closing here as this is being tracked in mbedmicro/mbed