Mbed-os: Executing test commands from below the top level project folder fails

[maclobdell]

Description

I'm running tests from a mbed-os repo. My top level folder is /proj/maclobdell/mbed-os/.

If I create a directory (test_output in my case), cd into it, then launch the test command from in there, the tools seem to jump out of the mbed-os repo (top level of this particular project) and get into other projects and get confused.

Note, that this folder contains a clone of mbed-os Probably added with 'git clone' but maybe added with 'mbed import' (I can't remember which).

The mbed-os directory contains a .mbed file, which I heard tells the tools where the top level of the project is.
-rwxrwxrwx 1 mac mac 49 Nov 14 22:16 .mbed

Running the test command within a sub folder results in errors from the tools trying to link multiple projects together. The projects are in a directory above the mbed-os. (mbed-os-exampe-blinky, smcc-tests, etc, in my case).

mac@thing1:/proj/maclobdell/mbed-os/test_output$ mbed test -n tests-integration-basic
[mbed] Auto-installing missing Python modules...
[mbed] WARNING: Cannot run icetea tests. Current folder does not contain TEST_APPS folder.
---
Building library mbed-build (MTS_DRAGONFLY_L471QG, ARM)
Scan: maclobdell
Library name 'cellular' is not unique (defined in '/proj/maclobdell/mbed-os-example-blinky/mbed-os/features/cellular/mbed_lib.json' and '/proj/maclobdell/smcc-tests/mbed-os/features/cellular/mbed_lib.json')
Failed to build library
Memory map breakdown for built projects (values in Bytes):
| name | target | toolchain | static_ram | total_flash |
|------|--------|-----------|------------|-------------|
Traceback (most recent call last):
  File "/proj/maclobdell/mbed-os/tools/test.py", line 288, in <module>
    status = print_report_exporter.report(build_report)
  File "/proj/maclobdell/mbed-os/tools/test_exporters.py", line 92, in report
    return self.exporter_print(test_summary_ext, print_log_for_failures=print_log_for_failures)
  File "/proj/maclobdell/mbed-os/tools/test_exporters.py", line 345, in exporter_print
    raise Exception("'test_run' did not have a 'result' value")
Exception: 'test_run' did not have a 'result' value
[ERROR] 'test_run' did not have a 'result' value
[mbed] ERROR: "/usr/bin/python" returned error.
       Code: 1
       Path: "/proj/maclobdell/mbed-os"
       Command: "/usr/bin/python -u /proj/maclobdell/mbed-os/tools/test.py -t ARM -m MTS_DRAGONFLY_L471QG --source .. --build ../BUILD/tests/MTS_DRAGONFLY_L471QG/ARM --test-spec ../BUILD/tests/MTS_DRAGONFLY_L471QG/ARM/test_spec.json --build-data ../BUILD/tests/MTS_DRAGONFLY_L471QG/ARM/build_data.json -n tests-integration-basic --greentea"
       Tip: You could retry the last command with "-v" flag for verbose output
---
mac@thing1:/proj/maclobdell/mbed-os/test_output$

If I execute the test command from the mbed-os directory, this problem goes away.

mbed-os version tested: 5.10.2
tools versions: mbed-cli (1.8.2), mbed-greentea (1.4.0), mbed-host-tests (1.4.1), mbed-ls (1.6.1)
Tested on: Linux (Debian)
target tested: various
toolchain tested: various

[ ] Question
[ ] Enhancement
[X] Bug

[maclobdell]

cc @theotherjimmy

[ciarmcom]

Internal Jira reference: https://jira.arm.com/browse/MBOCUSTRIA-169

[theotherjimmy]

--source ..

Ah, there's the problem... Probably

[maclobdell]

@theotherjimmy - Any update on this? What does --source mean in this context? Can you explain? Do you have a proposal for a fix for this problem?

[theotherjimmy]

It's quite unusual to see --source ... The test system might be sensitive to where it's being run from. Could you try running mbed test from the parent directory?

[maclobdell]

Could you try running mbed test from the parent directory?

Hi Jimmy, yes, and that works. That is exactly the issue. It works fine when running from the parent directly. Running from a sub directly used to work in the past, but now it does not.

[cmonr]

Hmmm... icetea.
@ARMmbed/mbed-os-test-team @OPpuolitaival (Although I think he's out for a while)

[maclobdell]

Any update from anyone on this? It is annoying me almost daily. If I try to run tests from one directory below the root of the project, it jumps two directories up and tries to build all of my other unrelated adjacent projects.