mbed test --compile -m K64F -t ARMC6 - On master branch is not working
Image: BUILD/tests/K64F/ARMC6/mbed-os/features/storage/filesystem/littlefs/TESTS/filesystem_recovery/resilience/resilience.bin
Traceback (most recent call last):
File "C:\Users\deebha02\work_mbed\Nuvoton_fix_DOMAIN_NS\mbed-os\tools\test_api.py", line 2209, in build_test_worker
bin_file, _ = build_project(*args, **kwargs)
File "C:\Users\deebha02\work_mbed\Nuvoton_fix_DOMAIN_NS\mbed-os\tools\build_api.py", line 537, in build_project
objects = toolchain.compile_sources(resources, sorted(resources.get_file_paths(FileType.INC_DIR)))
File "C:\Users\deebha02\work_mbed\Nuvoton_fix_DOMAIN_NS\mbed-os\tools\toolchains\__init__.py", line 392, in compile_sources
object = self.relative_object_path(self.build_dir, source)
File "C:\Users\deebha02\work_mbed\Nuvoton_fix_DOMAIN_NS\mbed-os\tools\toolchains\__init__.py", line 308, in relative_object_path
mkdir(obj_dir)
File "C:\Users\deebha02\work_mbed\Nuvoton_fix_DOMAIN_NS\mbed-os\tools\utils.py", line 179, in mkdir
makedirs(path)
File "c:\yotta\python\lib\os.py", line 157, in makedirs
mkdir(name, mode)
WindowsError: [Error 206] The filename or extension is too long: 'BUILD\\tests\\K64F\\ARMC6\\mbed-os\\features\\storage\\filesystem\\littlefs\\TESTS\\filesystem_recovery\\resilience_funct\\mbed-os\\features\\storage\\filesystem\\littlefs\\TESTS\\filesystem_recovery\\resilience_funct'
Expected:
Build skips:
* K64F::ARMC6::MBED-OS-FEATURES-STORAGE-FILESYSTEM-LITTLEFS-TESTS-FILESYSTEM_RECOVERY-RESILIENCE_FUNCTIONAL
Observation:
I created multiple renamed copies of resilience_functional at same path as resilience_fun , resilience_funct and resilience_functi to make sure its filename length issue or not and below is the result.
Build skips:
* K64F::ARMC6::MBED-OS-FEATURES-STORAGE-FILESYSTEM-LITTLEFS-TESTS-FILESYSTEM_RECOVERY-RESILIENCE_FUN
Build failures:
* K64F::ARMC6::MBED-OS-FEATURES-STORAGE-FILESYSTEM-LITTLEFS-TESTS-FILESYSTEM_RECOVERY-RESILIENCE_FUNCT
[Error] @,: ARMC6 does not support ARM architecture v7 targets
Building project resilience_funct (K64F, ARMC6)
Scan: ARMC6
Scan: resilience_funct
Scan: COMMON
* K64F::ARMC6::MBED-OS-FEATURES-STORAGE-FILESYSTEM-LITTLEFS-TESTS-FILESYSTEM_RECOVERY-RESILIENCE_FUNCTI
[Error] @,: ARMC6 does not support ARM architecture v7 targets
Building project resilience_functi (K64F, ARMC6)
Scan: ARMC6
Scan: resilience_functi
Scan: COMMON
* K64F::ARMC6::MBED-OS-FEATURES-STORAGE-FILESYSTEM-LITTLEFS-TESTS-FILESYSTEM_RECOVERY-RESILIENCE_FUNCTIONAL
[Error] @,: ARMC6 does not support ARM architecture v7 targets
Building project resilience_functional (K64F, ARMC6)
Scan: ARMC6
Scan: resilience_functional
Scan: COMMON
Detail error as listed above.
[ ] Question
[ ] Enhancement
[X] Bug
We need refactoring of folder names and path, we need to use optimized name for each folder and test.
Apart from that we also need proper solution from tools for long paths.
CC @ARMmbed/mbed-os-maintainers
Interesting. Thinking about disabling the test and asking @ARMmbed/mbed-os-storage to rename.
@ARMmbed/mbed-os-test Looking forward to being able to test PRs in Windows so that these kind of issues are caught before merge :)
CC @gaborkertesz
@ARMmbed/mbed-os-tools What are the proposals for this one ?
The test path is https://github.com/ARMmbed/mbed-os/tree/master/features/storage/filesystem/littlefs/TESTS/filesystem_recovery/resilience_functional - does not look that long or wrongly placed - easily can be reproduced by adding a new feature with TESTS 馃槴
From the error: WindowsError: [Error 206] The filename or extension is too long: 'BUILD\\tests\\K64F\\ARMC6\\mbed-os\\features\\storage\\filesystem\\littlefs\\TESTS\\filesystem_recovery\\resilience_funct\\mbed-os\\features\\storage\\filesystem\\littlefs\\TESTS\\filesystem_recovery\\resilience_funct'
Why path is doubled? BUILD\\tests\\K64F\\ARMC6\\mbed-os\\features\\storage\\filesystem\\littlefs\\TESTS\\filesystem_recovery\\resilience_funct this should be fine, shouldn't it?
I have a minor comment: The FILESYSTEM_RECOVERY-RESILIENCE_FUNCTIONAL test has been skipped a few weeks ago. Now this test is added to the compiling list from the LITTLE_FS tests. Is this intentional?
For Musca with some weeks earlier mbed master the build skipped the following tests of FILESYSTEM-LITTLEFS:
Now:
I have a minor comment: The FILESYSTEM_RECOVERY-RESILIENCE_FUNCTIONAL test has been skipped a few weeks ago. Now this test is added to the compiling list from the LITTLE_FS tests. Is this intentional?
@ARMmbed/mbed-os-storage
I have run the mbed test --compile -m K64F -t ARMC6 on my windows machine in a bash terminal and the compilation succeeded including skipping the K64F::ARMC6::FEATURES-STORAGE-FILESYSTEM-LITTLEFS-TESTS-FILESYSTEM_RECOVERY-RESILIENCE_FUNCTIONA test.
@gaborkertesz, I have no idea how to build the ARM_MUSCA_A1 with mbed os 5.10. its looks like ARM_MUSCA_A1 is not a recognized target. Can you send me an how to instruction for the ARM_MUSCA_A1?
@yossi2le - Its failing for on Windows machine, if path length is larger. If you increase folder levels to mbed-os you will see failure on master branch.
If path to mbed-os is equivalent to C:\Users\xxxxxx02\work_mbed\xxxxxxx_xxx_xxxxxxxxx\mbed-os failure will be encountered.
@yossi2le ARM_MUSCA_A1 is not yet upstreamed to mbed, so you can't compile to that platform on the master branch, but ARM_CM3DS_MPS2 from our platforms produces the same issue.
@yossi2le ARM_MUSCA_A1 is not yet upstreamed to mbed, so you can't compile to that platform on the master branch, but ARM_CM3DS_MPS2 from our platforms produces the same issue.
@gaborkertesz I have run the command "mbed test --compile -m ARM_CM3DS_MPS2 -t ARMC6 -v" on mbed-os master branch and its skip the test you mentioned. below is the skipped test list.
Can it be that you have the MBED_TEST_BLOCKDEVICE macro enabled in your environment?
Build skips:
@yossi2le If I run the build in c:\test\ path I got the same result as you. If I run the build my original working copy: c:\kg\repositoriesmbed-os_development\ the build fails as it was described.
So basically I've just verified now what @deepikabhavnani explained.
Please test your build in a path was suggested by @deepikabhavnani !
@yossi2le - Its failing for on Windows machine, if path length is larger. If you increase folder levels to >mbed-os you will see failure on master branch.
If path to mbed-os is equivalent to C:\Users\xxxxxx02\work_mbed\xxxxxxx_xxx_xxxxxxxxxmbed-os >failure will be encountered.
Thanks @deepikabhavnani , I know about this issue I run it with short path on windows to check @gaborkertesz issue.
@gaborkertesz I see what you mean. in my case, it fails to build the test with a long path. I guess this is a problem if the long path issue which tampered with the build tool. bummer:(.
@yossi2le - Long term fix for path length should come from tools, but for now is it possible to refactor storage paths and folder names to make sure we don't catch this issue too early. It is not possible for users to always work in "C:\" only
@deepikabhavnani Hi! So this is on me and @cmonr to fix then.
Internal Jira reference: https://jira.arm.com/browse/IOTCORE-488
@deepikabhavnani Is this still an issue? Attempted to compile and reproduce the issue using ARMC6, but under our current tools, we don't support using ARMC6 against v7 cores.
Nvm, found the issue.
Hitting the same problem - this is not letting us run Mbed Enabled tests out of the box
Can you please introduce a fix?
Hi,
We are also seeing the same issue of long path from windows, with 5.11. Please let us know the eta for the fix.
Thanks,
Sathy.
We're actively looking at a fix for the problem.
@MarceloSalazar @sathysreenath Would y'all mind opening a new PR for your particular issue? I just attempted to reproduce this issue, but because of the ARMC6 activity currently in progress for 5.12, I'd like to rule out that your particular problem is related to that.
Sorry @cmonr I don't get it. This is the issue to report the problem.
A PR is to fix it, but don't think I'm best for this.
I'm happy to work with you on reproducing and testing a fix.
@MarceloSalazar @sathysreenath Gah, sorry about that. I've been spending too much time with GitHub.
I meant to ask if y'all could open a new issue. I'd like to get more information on how y'all are triggering/seeing the issue.
I looked into this a bit. The main reason this fails is the "260 path length limit" in Windows. So between keeping the paths within Mbed OS shorter and keeping working directory close to the drive's root directory, there isn't really a solution for this problem that will work with arbitrary working paths.
That being said, I looked into what @0xc0170 mentioned about the paths being doubled. It looks like they are indeed being copied a few times, so this is making some pretty long paths. The reason its being doubled is somewhat lengthy to explain:
TESTS/group/case/<all these files are test case sources>TESTS/COMMON/<all these files are common test sources>TESTS/group/COMMON/<all these files are also common test sources>For tests that have relevant COMMON sources, they are added to the test case build step (not the common os library build step). So the build system keeps these object files separate by recreating the directory structure inside the TESTS/group/case directory. It may be possible to remove the need to recreate the directory structure if the COMMON source files were not built during the test case build step, but during the common library build step. I haven't really investigated what this would take because I wanted to double check that this issue was still causing problems for people.
So, is this still causing problems? 馃槃
I think that original issue is fixed and we can close this
I think that original issue is fixed and we can close this
@deepikabhavnani Can you confirm?
Can still reproduce the error:
Path length - C:\Users\xxxxxxxx\xxxx_xxxxx\xxxxxxx_xxx_xxxxxxxxxc\mbed-os
Command - mbed test --compile -m K64F -t ARMC6 -n mbed-os-features-storage-filesystem-littlefs-tests-filesystem_recovery-resilience_functional
Errror -
WindowsError: [Error 206] The filename or extension is too long: 'BUILD\\tests\\K64F\\ARMC6\\mbed-os\\features\\storage\\filesystem\\littlefs\\TESTS\\filesystem_recovery\\resilience_functional\\mbed-os\\features\\storage\\filesystem\\littlefs\\TESTS\\filesystem_recovery\\resilience_functional'
Building project resilience_functional (K64F, ARMC6)
Scan: ARMC6
Scan: resilience_functional
Scan: COMMON
Like I mentioned above, at a certain point when building on Windows, we have to keep the paths reasonably small in the OS. There is a small optimization that I mentioned above that could mitigate this a bit, but its just another mitigation, not a solution. I'd recommend moving the project closer to the root of the file system.
@deepikabhavnani originally said the following above:
We need refactoring of folder names and path, we need to use optimized name for each folder and test.
I think this makes quite a bit of sense!
Apart from that we also need proper solution from tools for long paths.
Windows just doesn't like long paths. The Android SDK faces the same issue I believe.
I've given some details in this issue about why we can't really "fix" this. It's a limitation on Windows. If there is any more discussion we should have we can always reopen this.
I created the issue, but can't reopen it :-(
Anyways, yes agree we have limitation from windows side and hence cannot be fixed by tools but shouldn't we be solving the issue by refactoring folder names and path?
I created the issue, but can't reopen it :-(
Well dang, that seems silly. Shall I reopen it then?
shouldn't we be solving the issue by refactoring folder names and path?
That's fine with me! This issue would need to be assigned to each team's tests though. Perhaps a new issue with each of the paths that need to be refactored would be more helpful?
Most helpful comment
@ARMmbed/mbed-os-test Looking forward to being able to test PRs in Windows so that these kind of issues are caught before merge :)