Runtime: ARM 32-bit progress

Created on 30 Mar 2016  ·  123Comments  ·  Source: dotnet/runtime

I'm opening an issue to track the progress of ARM 32-bit with respects to the regression tests.

Currently I'm getting the following results on Ubuntu 14.04 running on a NVidia Jetson TK1. This is unfortunately not the results you get from a clean checkout, the only difference is that it contains a fix for issue dotnet/runtime#5422 (PR dotnet/coreclr#3879), but overall it should provide a good base to monitor our progress:

=======================
     Test Results
=======================
# Tests Discovered : 6018
# Passed           : 4313
# Failed           : 1361
# Skipped          : 344
=======================

I've attached the XML results:
coreclrtests.zip

arch-arm32

Most helpful comment

Finally...
Here is the latest result from Raspberry Pi 2 which uses Hard FP ABI. :)
445405d84034564a6433074e0f920f74105ba499

=======================
     Test Results 
=======================
# Tests Discovered : 11399
# Passed           : 11035
# Failed           : 0
# Skipped          : 364
=======================
constructing /home/sjlee/unit_test/dumplings.txt
270 minutes and 5 seconds taken to run CoreCLR tests.

With testsUnsupportedOnARM32.txt set as follow

JIT/Directed/tailcall/tailcall/tailcall.sh
JIT/jit64/opt/cse/HugeArray1/HugeArray1.sh
JIT/Methodical/tailcall_v4/hijacking/hijacking.sh
JIT/Methodical/tailcall_v4/smallFrame/smallFrame.sh
JIT/Methodical/xxobj/sizeof/_il_dbgsizeof32/_il_dbgsizeof32.sh
JIT/Methodical/xxobj/sizeof/_il_dbgsizeof64/_il_dbgsizeof64.sh
JIT/Methodical/xxobj/sizeof/_il_dbgsizeof/_il_dbgsizeof.sh
JIT/Methodical/xxobj/sizeof/_il_relsizeof32/_il_relsizeof32.sh
JIT/Methodical/xxobj/sizeof/_il_relsizeof64/_il_relsizeof64.sh
JIT/Methodical/xxobj/sizeof/_il_relsizeof/_il_relsizeof.sh
JIT/Regression/JitBlue/devdiv_902271/DevDiv_902271/DevDiv_902271.sh

All 123 comments

This is unfortunately not the results you get from a clean checkout,

Even though you could get the 1,361 Failed result, this result is meaningful to me. Actually, we have been meeting a lot of "NYI" functions from CoreCLR.:( It means that we have to do a lot of things for stablization of CoreCLR.

Currently I'm getting the following results on Ubuntu 14.04 running on a NVidia Jetson TK1

BTW, Do you also focus on the Ubuntu/ARM32bit without the Ubuntu/ARM64bit?
http://elinux.org/Jetson_TK1

  • CPU: NVIDIA "4-Plus-1" 2.32GHz ARM quad-core Cortex-A15 CPU with Cortex-A15 battery-saving shadow-core
  • GPU: NVIDIA Kepler "GK20a" GPU with 192 SM3.2 CUDA cores (upto 326 GFLOPS)
  • Memory: ~ 8GiB DDR3

\CC: @myungjoo , @lemmaa
They also interested in the regression test of CoreCLR.

@leemgs I'm not sure to understand what you mean by Ubuntu/ARM32bit without the Ubuntu/ARM32bit part?

@manu-silicon Sorry, Typo, I have modified.

@leemgs My board is the 32-bit variant, so it is just Ubuntu/ARM32bit.

https://en.wikipedia.org/wiki/Tegra#Tegra_K1

With the fix for PR dotnet/coreclr#3981 the result is:

=======================
     Test Results
=======================
# Tests Discovered : 6019
# Passed           : 4793
# Failed           : 882
# Skipped          : 344
=======================

coreclrtests.zip

@manu-silicon I am trying to do the same (run coreclr tests on ARM environment) by following this guide https://github.com/dotnet/coreclr/blob/master/Documentation/building/unix-test-instructions.md. According to this, building tests for ARM results in an error (as building the native code using cmake in Windows does not support arm architecture yet). I can make some changes to the buildtest script to build only the C# based tests for ARM architecture.

I wanted to ask how you are building the test binaries for ARM environment (using buildtest.cmd). Have you made some local changes (to build only the C# code and ignore the CMake build) or is there some other way to do it?

@prajwal-aithal I've followed the instructions and build the tests on Windows and copy the tests folder on Linux ARM. I built on ARM the native code for CoreFX (just the native code as the rest won't build). I built CoreFX on Linux x64 and copied the assemblies over. I built mscorlib on Linux x64 and copied it over.

Once I had all the paths setup properly, I launched the command to run the tests.

@manu-silicon Oh, ok. I will try this then. Thank you :)

Today's result with FEATURE_STUBS_AS_IL disabled:

=======================
     Test Results
=======================
# Tests Discovered : 6019
# Passed           : 4836
# Failed           : 839
# Skipped          : 344
=======================

coreclrtests.zip

Target: Raspberry Pi2
CoreClr commit id : ff26d6801b3ce0dec5918a5ad0d3ab90f9656e28 (around april 9)

=======================
     Test Results
=======================
# Tests Discovered : 7422
# Passed           : 6074
# Failed           : 1017
# Skipped          : 331
=======================

I'm curious why my test discovered count is different with others.
CoreClr_UnitTest_Results_160412_01.zip

@jyoungyun Next time, please attach the result XML file as well.

@jyoungyun The difference in the number of tests is related on how I build first the test on Windows and then copied them over to the Linux ARM device. As @myungjoo is saying, having the XML file would enable us to compare and see what the differences are.

@manu-silicon I attached xml file. Thank you.

Target: Raspberry Pi2
Coreclr commit id : 31fada1 + dotnet/coreclr#4460 + dotnet/coreclr#4503

==========================
      Test Results
==========================
# Tests Discovered : 7421
# Passed           : 6547
# Failed           : 543
# Skipped          : 331
==========================

CoreClr_UnitTest_Results_160427.zip

Wow, so those two fixes cleared up about half of the failing tests? Nice!

@masonwheeler Yes! The dotnet/coreclr#4460 patch reduced half of failing tests. :)

Target: Raspberry Pi2
Coreclr commit id: 18268be + dotnet/coreclr#4503 + dotnet/coreclr#4581

==========================                                                      
      Test Results                                                              
==========================                                                      
# Tests Discovered : 6018                                                       
# Passed           : 5295                                                       
# Failed           : 395                                                        
# Skipped          : 328                                                        
==========================  

And I found the reason why my tests discovered count is different with others.
My test directory structure was wrong. Now it's the same with others. :)

CoreClr_UnitTest_Results_160428.zip

@jyoungyun , It's amazing. : ) BTW, Is https://github.com/dotnet/coreclr/commit/18268beae931cbd4d110959ea53d785b193eceb1 related to the number of the reduced failure? It seems that it is not related to the failure numbers of the unit test for Linux/ARM.

@jyoungyun If possible, it would be nice to know how many tests each bug fix addresses.

@jyoungyun Out of curiosity, how long does it take you to run all the tests. For me on the Jetson TK1, it takes roughly half a day.

@manu-silicon We will be pushing out patches that accelerate the execution of corerun at ARM soon. @leemgs is preparing PR for that (release build bugfix)

For reference, @manu-silicon has shared the hardware specification that he used for getting the result of unit test. Here is the detail contents of his development board.

@leemgs Oh, The number(18268be) means the base commit. Someone want to reproduce the result, the base commit info. will be helpful. Reducing failure test cases are more to do with dotnet/coreclr#4460 patch, I think.
@manu-silicon It takes a lot of time to run unix test on Raspberry pi2. In my case, about 12 hours is the usaual time required. But when I use a release mode binary, it takes only 4 hours! I hope that @leemgs patch will be ready soon.

In the lastet commit(92d709193a8ee3d8da3519811912ec0f7a552993), I got much better result. I was investigating several failures and many of them are fixed in the latest master branch.
Can anybody run the whole test again?

@hqueue I can do it on Friday.

Tried to run the test suite on Friday, but it got stuck on 2 tests that I had to manually kill:

FAILED   - JIT/Regression/CLR-x86-JIT/V2.0-Beta2/b425314/b425314/b425314.sh
               BEGIN EXECUTION
               /home/ubuntu/local/Microsoft/tests/Tests/coreoverlay/corerun b425314.exe
               ./b425314.sh: line 62: 32428 Killed                  $_DebuggerFullPath "$CORE_ROOT/corerun" b425314.exe $CLRTestExecutionArguments $Host_Args
               Expected: 100
               Actual: 137
               END EXECUTION - FAILED

FAILED   - JIT/Regression/CLR-x86-JIT/V2.0-Beta2/b426654/b426654/b426654.sh
               BEGIN EXECUTION
               /home/ubuntu/local/Microsoft/tests/Tests/coreoverlay/corerun b426654.exe
               ./b426654.sh: line 62: 10285 Killed                  $_DebuggerFullPath "$CORE_ROOT/corerun" b426654.exe $CLRTestExecutionArguments $Host_Args
               Expected: 100
               Actual: 137
               END EXECUTION - FAILED

The run finally completed. Results on Jetson TK1 commit 3ddd9c43821bf617621f87d8d1519229e7d3b789

=======================
     Test Results
=======================
# Tests Discovered : 6019
# Passed           : 5478
# Failed           : 234
# Skipped          : 307
=======================

coreclrtests_2016_05_06.zip

The latest unit test results!

Target: Raspberry Pi2
Coreclr commit id: 6b92cff6

==========================
      Test Results
==========================
# Tests Discovered : 6530
# Passed           : 5981
# Failed           : 215
# Skipped          : 334
==========================

CoreClr_UnitTest_Results_160510.zip

@manu-silicon When I run unix tests, some tc takes such a long time. And the blocked tc is changed. Have you seen the same case ?

Note that we have about 120 fails due to regression being hopefully fixed easily with dotnet/coreclr#4888. We may see under 100 fails soon.

@jyoungyun I'm not sure what you mean by "tc is changed". But as I said it just before posting my results, I had 2 tests that was hanging preventing the rest of the test cases to run. I had to kill them manually to get to the completion.

@manu-silicon Oh, I mean that I got stuck on different tcs when I tried to run unixtest. Sometimes b425314 was hanging and while other times the tc was not hanging instead the other tc got stuck. So I had to kill them manually every time. I wander why hanging tc is different everytime and want to know you have seen the same situation. If the stuck tc is always the same, we can modify tests/testsFailingOutsideWindows.txt to get to the completion without killing manually even though it's a temporary way.

It's amazing. The failed tc is finally double digits! :+1:

Target: Raspberry Pi2
Coreclr commit id: 9eba7d3

==========================
      Test Results
==========================
# Tests Discovered : 6530
# Passed           : 6137
# Failed           : 98
# Skipped          : 295
==========================

Wow, it's amazing to see what I started last year get this far. Sorry I couldn't bring it any further but it definitely seems like ARM could be a viable target for .NET Core soon.

It appears that we'd even see it being around 40 today. (or less than 40!).

Today's result!

Target : Raspberry Pi2
Coreclr Commt id : 1c2d8a6
tc commit id : 171f7133287ebbee24d6a7f193b13a9f959e9297

==========================
      Test Results
==========================
# Tests Discovered : 6966
# Passed           : 6626
# Failed           : 49
# Skipped          : 291
==========================

It's great to see such progress here!

Very nice! Great progress!
Do you have next target once all of these Pri0 tests pass? Is it Pri1 tests (about 3000 additional tests)?

To All,

Here is another note-taking. I have summarized the unit-test result of coreCLR with Release-build mode on Samsung ARM ChromeBook. Note that the below result was executed by Release-build mode.

Before doing the below 3 steps

=======================
     Test Results
=======================
# CoreCLR Binary Folder: /unit-test/bin.coreclr.20160517/Product/Linux.arm.Release
# Tests Discovered : 6966
# Passed           : 6282
# Failed           : 420 ★
# Skipped          : 264
=======================
* Priority 0 (default)
* Target: Samsung ARM Chromebook
* Run Time: 89m32.452s (13h in debug-build)
* CoreCLR Commit No: May-15-2016

coreclrtests.release.failur-420.20160515.zip

And then, We could get the 67 failures after executing the below 3 steps. In this case, we set the priority 666 (instead of priority 0) to increase test cases.
1) Resync all components in release binaries (without the debug-build mode)
2) Adding the all managed files (Linux, Unix, AnyOS)
3) UNW_ARM_UNWIND_METHOD Back: default
( source: https://wiki.linaro.org/KenWerner/Sandbox/libunwind#overhead_of_the_ARM_specific_unwind-tables)

After doing the above 3 steps

=======================
     Test Results
=======================
# CoreCLR Binary Folder: /unit-test/bin.coreclr.20160517/Product/Linux.arm.Release
# Tests Discovered : 9870
# Passed           : 9458
# Failed           : 67  ★
# Skipped          : 345
=======================
* Priority 666
* Target: Samsung ARM Chromebook
* Run Time: 128m55.349s (15h in debug-build)
* CoreCLR Commit No: May-15-2016

coreclrtests.chromebook.release.failure-67.20160518.zip

Two days ago, results.
I run unit tests including test priority 0 and 1 but the failure count is pretty much the same before. :)

Target : Raspberry Pi2
Coreclr Commt id : e78338e (Date: Tue May 17 17:19:37 2016 -0700)
tc commit id : 4941764c (Date: Mon May 16 22:46:36 2016 -0700)

==========================
      Test Results (Debug)
==========================
# Tests Discovered : 9870
# Passed           : 9423
# Failed           : 102
# Skipped          : 345
==========================

@leemgs

2) Adding the all managed files (Linux, Unix, AnyOS)

Where did you put those managed stuff?
How did you run tests using those files?

Where did you put those managed stuff?

@mkborg , I had got the managed stuffs by building coreFX source on Ubuntu 14.04 X64.

How did you run tests using those files?

  • Before: --coreFxBinDir="../corefx.bin/AnyOS.AnyCPU.Release"
  • After: --coreFxBinDir="../corefx.bin/Linux.AnyCPU.Release;../corefx.bin/Unix.AnyCPU.Release; \
    ../corefx.bin/AnyOS.AnyCPU.Release"

I will try it.

) UNW_ARM_UNWIND_METHOD Back: default

I will try it.

@mkborg , If you can, I recommend that you also try to check the different ARM unwind methods (e.g., UNW_ARM_UNWIND_METHOD=1|2|...|255)

dotnet/coreclr#5087 is tested to resolve 5 failed ARM/Linux test cases (PR0)

Just wanted to say thanks to everyone contributing their time to this issue <3

Hey, I just setup CoreCLR on my ARM Chromebook with TK1. Is the patch from the documentation still needed?

@Toxantron , You can easily setup the corclr/corefx environment on your own arm chromebook by referencing the documentation of coreclr. From my experience, I recommend that you utilize available usb stick for ubuntu/arm 14.04 based dotnet core.

Here is the recently unit test results on Raspberry Pi2.
The CoreCLR commit is 38a89155cb2c53a9603d2234731525abdd974014 and the TC commit is 38a89155cb2c53a9603d2234731525abdd974014 with priority 666.
I added the PAL test results too.

==========================
      Test Results
==========================
# CoreCLR Bin Dir  : /home/dotnet/tmp/coreclr/bin/Product/Linux.arm.Debug
# Tests Discovered : 9853
# Passed           : 9448
# Failed           : 53
# Skipped          : 352
==========================
1493 minutes and 27 seconds taken to run CoreCLR tests.

==========================
PAL Test Results:
  Passed: 807
  Failed: 1
==========================

We are observing about 20 less fails in different ARM devices. We will need to look at what are making such differences, too.

My recent tests have many failures:

=======================
     Test Results
=======================
# CoreCLR Bin Dir  : /coreclr-testing/20160706/odroid-2.arm-softfp/odroid-2.coreclr_46d3809.corefx_860d28c.arm-softfp/CORECLR_NATIVE/coreclr/bin/Product/Linux.arm-softfp.Debug
# Tests Discovered : 9870
# Passed           : 9124
# Failed           : 394
# Skipped          : 352
=======================

Is it my local issue or actual regression?

@jyoungyun What is your current test results?

@mkborg My result is almost same with you. Raspberry pi failed over 400 tests. dotnet/coreclr#6051 It is the related issue.

My result is almost same with you. Raspberry pi failed over 400 tests. dotnet/coreclr#6051 It is the related issue.

Adding @chunseoklee, Could you synchronize a similar issue between @jyoungyun's comment and @mkborkg's test result (e.g. Failed 394)?

/CC: @parjong

Regressions mentioned above are cased by 597e160 commit(PR dotnet/coreclr#6088 ), which change the behavior of exception handler.
This PR dotnet/coreclr#6117 resolves unit test regressions by 597e160 commit.

It's my results in Raspberry Pi2.
commit no. c9287008
I hope that many issues pass when @chunseoklee 's patch is merged.

=======================
     Test Results
=======================
# CoreCLR Bin Dir  : /home/dotnet/tmp/coreclr/bin/Product/Linux.arm.Debug
# Tests Discovered : 9870
# Passed           : 8935
# Failed           : 583
# Skipped          : 352
=======================

On my site, CoreCLR (c29547fc155) with PR dotnet/coreclr#6117 shows the following result (I used Release CoreCLR / Release TC):

=======================
     Test Results
=======================
# CoreCLR Bin Dir  : 
# Tests Discovered : 9870
# Passed           : 9497
# Failed           : 36
# Skipped          : 337
=======================

@parjong Thank you for sharing the unit-test result. BTW, Could you keep the content of # CoreCLR Bin Dir to know the evaluation environment of the experimental device?

@leemgs Unfortunately, CoreCLR Bin Dir line is not available for runtest.sh with --coerOverlayDir (which I use for testing). Instead, I usually kept the information in seprate place.

For the above result, I used CoreCLR with -O1 Release mode c29547fc155 with PR dotnet/coreclr#6117, and TC with Release mode from 387d9fc0a.

Among failing tests, it seems that at least nine tests can be considered as false positive failures for ARM 32-bit by now.

According to issue dotnet/coreclr#6248, six tests fail on ARM 32-bit, becuase unix tests is built for 64-bit architecture(x64) where native int is 64-bit.
I ran six tests built for 32-bit(i.e. x86) and all six tests are passed on arm-softfp/Linux.
However it seems that we cannot buid full set of unix tests for x86 now as tests/buildtest.cmd says it only supports x64. (There is an issue discussing building unix tests in Linux (issue dotnet/coreclr#4437), but not about 32/64-bit)

In addtiion, three tests fail due to lack of FEATURE_FASTTAILCALL for ARM 32-bit as addressed in issue dotnet/coreclr#6217.

Also there are also four tests failure due to lack of R2R support for ARM 32-bit as discussed in issue dotnet/coreclr#6231.

Welcome to Release Build world!!! I mean that we will meet more technical challenges by enabling Release + -O3 (by default). The number of unit-test failures is 47 (such as debug build mode) after applying the PR https://github.com/dotnet/coreclr/pull/6379.

=======================
     Test Results (Release + O3)
=======================
# CoreCLR Bin Dir  : /unit-test/bin.coreclr.release.20160627.1.O3.attribute.1.refacto1/Product/Linux.arm.Release
# Tests Discovered : 9873
# Passed           : 9465
# Failed           : 47 (commit: 20160627-52570683, OOM: 2,  manual kill commands: 0 ) <=== N O W ! ! !
# Skipped          : 361
=======================
117 minutes and 42 seconds taken to run CoreCLR tests.

real    119m15.377s
user    145m21.515s
sys 17m23.065s

FYI. I have evaluated the unit-test of CoreCLR with latest commit number on Linux/ARM board.

  • Commit Number: c358f77a1ddcf31475450c40af64ad115259e622 (Jul-25-2016) + PR dotnet/coreclr#6379 (The PR will be merged shortly)
  • Build Mode: Release build + O3 (by default)
  • On Samsung ARM ChromeBook
  • OS: Ubuntu/ARM 14.04 32bit (hard-fp)
  • Unit-test result after applying the PR: (OOM: 2, manual kill commands: 1 )
=======================
     Test Results
=======================
# CoreCLR Bin Dir  : /unit-test/bin.coreclr.release.O3.054d133a.20160725.1/Product/Linux.arm.Release
# Tests Discovered : 9873
# Passed           : 9484
# Failed           : 28
# Skipped          : 361
=======================
164 minutes and 14 seconds taken to run CoreCLR tests.

real    165m24.788s
user    153m41.410s
sys 56m11.885s
root@target:/unit-test/coreclr.git/tests# 

@leemgs Have you tried to build CoreCLR for ARM soft FP with emulator rootfs? I receive the following error building with your patched clang3.6:

../../coreclr/hosts/unixcoreruncommon/libunixcoreruncommon.a(coreruncommon.cpp.o): In function `std::string::_M_check(unsigned int, char const*) const':
/CROSS-ROOTFS/emulator/usr/lib/gcc/armv7l-tizen-linux-gnueabi/4.9.2/include/c++/bits/basic_string.h:324: undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'
/CROSS-ROOTFS/emulator/usr/lib/gcc/armv7l-tizen-linux-gnueabi/4.9.2/include/c++/bits/basic_string.h:324: undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'
/CROSS-ROOTFS/emulator/usr/lib/gcc/armv7l-tizen-linux-gnueabi/4.9.2/include/c++/bits/basic_string.h:324: undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'

However build is succesful if I use unmodified upstream clang-3.8 or clang-3.5.

@leemgs Have you tried to build CoreCLR for ARM soft FP with emulator rootfs?
I receive the following error building with your patched clang3.6:

No Problem. I can also build CoreCLR for Linux/ARM softFP with emulator rootfs as following:

 . . . . . . . UPPER OMISSION . . . . . . .
  NuGet Config files used:
      /work/dotnet/coreclr.git/src/NuGet.Config
      /home/invain/.nuget/NuGet/NuGet.Config

  Feeds used:
      https://dotnet.myget.org/F/dotnet-core/api/v3/index.json
Generating nuget packages for Linux
  Skipping nuspec generation for this platform.
  Skipping nuspec generation for this platform.
  Skipping nuspec generation for this platform.
  Skipping nuspec generation for this platform.
  Skipping nuspec generation for this platform.
  Skipping nuspec generation for this platform.
  Skipping nuspec generation for this platform.
  Skipping nuspec generation for this platform.
  Skipping nuspec generation for this platform.
Repo successfully built.
Product binaries are available at /work/dotnet/coreclr.git/bin/Product/Linux.arm-softfp.Release
1826.83user 88.72system 5:18.30elapsed 601%CPU (0avgtext+0avgdata 466172maxresident)k
977800inputs+9311672outputs (652major+39210140minor)pagefaults 0swaps
invain@db400:/work/dotnet/coreclr.git$ 
invain@db400:/work/dotnet/coreclr.git$ 
invain@db400:/work/dotnet/coreclr.git$ 
invain@db400:/work/dotnet/coreclr.git$ dpkg -l | grep clang
ii  clang-3.6                                                   1:3.6-2ubuntu1~trusty1ubuntu3                       amd64        C, C++ and Objective-C compiler (LLVM based)
ii  libclang-common-3.6-dev                                     1:3.6-2ubuntu1~trusty1ubuntu3                       amd64        clang library - Common development package
ii  libclang1-3.6:amd64                                         1:3.6-2ubuntu1~trusty1ubuntu3                       amd64        C interface to the clang library
invain@db400:/work/dotnet/coreclr.git$ 

\CC: @sjsinju , @prajwal-aithal

/CROSS-ROOTFS/emulator/usr/lib/gcc/...gnueabi/4.9.2/include/c++/bits/basic_string.h:324: undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'

@mkborg , If you still get the same issue, I recommend that you refer to Issue dotnet/coreclr#5600 that I shared previously.

I have summarized the file size comparison of major files among the different optimization levels in release build mode (e.g. -O1, -O2, -O3 {by default} , -Oz, and -Ofast) as follows. Also, you can see the file size of Mono 4.4.1 (latest version) at the "6. Linux/ARM: Release Build + Mono 4.4.1: Stripped , (Nightly 4.4.1.0/4747417 Wed Jun 22 11:38:12 UTC 2016)"

1. Linux/ARM: Release Build + O1: stripped , commit: 561b64d2c210b4d999de7f1ac55756704eaba784 (Jul-26-2016)

-rwxr-xr-x 1 leemgs leemgs 22,608 Jul 29 07:30 corerun
-rwxr-xr-x 1 leemgs leemgs 3,698,544 Jul 29 07:29 libcoreclr.so
-rwxr-xr-x 1 leemgs leemgs 1,254,968 Jul 29 07:29 libclrjit.so
-rwxr--r-- 1 leemgs leemgs 29,696 Jul 29 00:55 ./mscorlib.dll
-rwxr--r-- 1 leemgs leemgs 2,211,840 Jul 29 00:54 ./System.Private.CoreLib.dll
Total: 7,217,656 bytes

2. Linux/ARM: Release Build + O2: stripped , commit: 561b64d2c210b4d999de7f1ac55756704eaba784 (Jul-26-2016)

-rwxr-xr-x 1 leemgs leemgs 18,496 Jul 27 07:49 corerun
-rwxr-xr-x 1 leemgs leemgs 4,275,520 Jul 27 07:49 ./libcoreclr.so
-rwxr-xr-x 1 leemgs leemgs 1,447,324 Jul 27 07:49 ./libclrjit.so
-rwxr--r-- 1 leemgs leemgs 29,696 Jul 27 00:55 ./mscorlib.dll
-rwxr--r-- 1 leemgs leemgs 2,211,840 Jul 27 00:54 ./System.Private.CoreLib.dll
Total: 7,982,876 bytes

3. Linux/ARM: Release Build + O3(by default): stripped , commit: 561b64d2c210b4d999de7f1ac55756704eaba784 (Jul-26-2016)

-rwxr-xr-x 1 leemgs leemgs 18,472 Jul 27 16:14 corerun
-rwxr-xr-x 1 leemgs leemgs 1,512,828 Jul 27 18:33 libclrjit.so
-rwxr-xr-x 1 leemgs leemgs 4,525,328 Jul 27 18:33 libcoreclr.so
-rwxr--r-- 1 leemgs leemgs 29,696 Jul 27 00:55 ./mscorlib.dll
-rwxr--r-- 1 leemgs leemgs 2,211,840 Jul 27 00:54 ./System.Private.CoreLib.dll
Total: 8,298,164 bytes

4. Linux/ARM: Release Build + Ofast: stripped , commit: 561b64d2c210b4d999de7f1ac55756704eaba784 (Jul-26-2016)

-rwxr-xr-x 1 leemgs leemgs 18,496 Jul 26 05:00 ./corerun
-rwxr-xr-x 1 leemgs leemgs 1,512,844 Jul 26 04:59 ./libclrjit.so
-rwxr-xr-x 1 leemgs leemgs 4,525,360 Jul 26 04:59 ./libcoreclr.so
-rwxr--r-- 1 leemgs leemgs 29,696 Jul 26 00:49 ./mscorlib.dll
-rwxr--r-- 1 leemgs leemgs 2,211,840 Jul 26 00:48 ./System.Private.CoreLib.dll
Total: 8,298,236 bytes

5. Linux/ARM: Release Build + Oz: stripped , commit: 561b64d2c210b4d999de7f1ac55756704eaba784 (Jul-26-2016)

-rwxr-xr-x 1 leemgs leemgs 14,400 Jul 29 05:02 ./corerun
-rwxr-xr-x 1 leemgs leemgs 1,181,160 Jul 29 04:40 ./libclrjit.so
-rwxr-xr-x 1 leemgs leemgs 3,407,200 Jul 29 04:40 ./libcoreclr.so
-rwxr--r-- 1 leemgs leemgs 29,696 Jul 29 00:49 ./mscorlib.dll
-rwxr--r-- 1 leemgs leemgs 2,211,840 Jul 29 00:48 ./System.Private.CoreLib.dll
Total: 6,844,296 bytes

6. Linux/ARM: Release Build + Mono 4.4.1: Stripped , (Nightly 4.4.1.0/4747417 Wed Jun 22 11:38:12 UTC 2016)

lrwxrwxrwx 1 leemgs leemgs 9 Jun 22 12:00 /usr/bin/mono -> mono-sgen
-rwxr-xr-x 1 leemgs leemgs 2,252,904 Jun 22 12:00 /usr/bin/mono-sgen (Mono Runtime [SGen])
-rw-r--r-- 1 leemgs leemgs 2,403,964 Jun 22 12:00 /usr/lib/libmonosgen-2.0.so.1.0.0 (Mono JIT library [SGen GC])
-rw-r--r-- 1 leemgs leemgs 3,753,984 Jun 22 11:22 /usr/lib/mono/4.5/mscorlib.dll
Total: 8,410,861 bytes

And, Below experimental results are the number of the unit-test failures on Raspberry Pi2 (Ubuntu/ARM 14.04). PR https://github.com/dotnet/coreclr/pull/6548 will shortly be merged as an instruction for -Oz (for size) and Ofast (for speed).

[CoreCLR Unit-Test in Release Build: -O3 VS. -Oz VS. -Ofast]

  • Release Build + O3: Default optimization level in release build mode
=======================
     Test Results (Release + O3 )
=======================
# CoreCLR Bin Dir  : /unit-test/bin.coreclr.release.O3.054d133a.20160725.1/Product/Linux.arm.Release
# Tests Discovered : 9873
# Passed           : 9465
# Failed           : 47  
# Skipped          : 361
=======================
118 minutes and 33 seconds taken to run CoreCLR tests.

real  119m39.397s
user  147m13.410s
sys 17m9.900s
root@target:/unit-test/coreclr.git/tests# 
  • Release Build + Oz: For size
=======================
     Test Results  (Release + Oz )
=======================
# CoreCLR Bin Dir  : /unit-test/bin.coreclr.release.Oz.054d133a.20160725.1/Product/Linux.arm.Release
# Tests Discovered : 9873
# Passed           : 9483
# Failed           : 29
# Skipped          : 361
=======================
128 minutes and 9 seconds taken to run CoreCLR tests.

real    129m42.436s
user    163m18.785s
sys 15m50.260s
root@target:/unit-test/coreclr.git/tests# 
  • Release Build + Ofast: For speed
=======================
     Test Results  (Release + Ofast )
=======================
# CoreCLR Bin Dir  : /unit-test/bin.coreclr.release.Ofast.054d133a.20160725.1/Product/Linux.arm.Release
# Tests Discovered : 9873
# Passed           : 9477
# Failed           : 35
# Skipped          : 361
=======================
144 minutes and 52 seconds taken to run CoreCLR tests.

real    146m26.534s
user    150m25.765s
sys 40m38.195s
root@target:/unit-test/coreclr.git/tests# 
--------------------------------------------------

+1 for arm devices

@leemgs do you know what is the current status of the CoreCLR unit tests pass rate?

@janvorli I overhear it fluctuating between 1 and 15 of over 10000 cases these days over the cubicles :)
And, it seems that except for the last few, they are due to incorrect test cases or incorrect usage of test cases.

@jyoungyun Can you post one recent & correct result?

Here is the latest result from Raspberry Pi 3 which uses Hard FP ABI

=======================
     Test Results
=======================
Elapsed:    353 min, 54 sec
Discovered: 11386
Passed:     11010
Failed:     14
Skipped:    362

Failed test:
  CoreMangLib.cti.system.datetime.DateTimeParse1.DateTimeParse1
  Exceptions.ForeignThread.ForeignThreadExceptions.ForeignThreadExceptions
  JIT.jit64.opt.cse.HugeArray.HugeArray
  JIT.jit64.opt.cse.hugeexpr1.hugeexpr1
  JIT.jit64.opt.cse.HugeField1.HugeField1
  JIT.jit64.opt.cse.HugeField2.HugeField2
  JIT.jit64.opt.cse.hugeSimpleExpr1.hugeSimpleExpr1
  JIT.Methodical.xxobj.sizeof._il_dbgsizeof32._il_dbgsizeof32
  JIT.Methodical.xxobj.sizeof._il_dbgsizeof64._il_dbgsizeof64
  JIT.Methodical.xxobj.sizeof._il_dbgsizeof._il_dbgsizeof
  JIT.Methodical.xxobj.sizeof._il_relsizeof32._il_relsizeof32
  JIT.Methodical.xxobj.sizeof._il_relsizeof64._il_relsizeof64
  JIT.Methodical.xxobj.sizeof._il_relsizeof._il_relsizeof
  JIT.superpmi.superpmicollect.superpmicollect

Here is the latest result from another ARM device which uses Soft FP ABI:

=======================
     Test Results
=======================
Elapsed:    179 min, 52 sec
Discovered: 11386
Passed:     11012
Failed:     12
Skipped:    362

Failed test:
  CoreMangLib.cti.system.datetime.DateTimeParse1.DateTimeParse1
  GC.Regressions.v2.0-rtm.494226.494226.494226
  JIT.jit64.opt.cse.hugeexpr1.hugeexpr1
  JIT.jit64.opt.cse.HugeField1.HugeField1
  JIT.jit64.opt.cse.HugeField2.HugeField2
  JIT.jit64.opt.cse.hugeSimpleExpr1.hugeSimpleExpr1
  JIT.Methodical.xxobj.sizeof._il_dbgsizeof32._il_dbgsizeof32
  JIT.Methodical.xxobj.sizeof._il_dbgsizeof64._il_dbgsizeof64
  JIT.Methodical.xxobj.sizeof._il_dbgsizeof._il_dbgsizeof
  JIT.Methodical.xxobj.sizeof._il_relsizeof32._il_relsizeof32
  JIT.Methodical.xxobj.sizeof._il_relsizeof64._il_relsizeof64
  JIT.Methodical.xxobj.sizeof._il_relsizeof._il_relsizeof

Here is my observation for the latter result.

  • dotnet/coreclr#6680 is related with all the failures under JIT.Methodical.xxobj.sizeof._il.
  • The lack of physical memory (as the device have only 1G memory) seems to cause all the other failures except DateTimeParse1.
  • dotnet/coreclr#7931 seems to address DateTimeParse1 failure, but I used a bit outdated testsuites for the above results (testsuites from 171962f5 commit).

@parjong thank you for the detailed information!

Finally...
Here is the latest result from Raspberry Pi 2 which uses Hard FP ABI. :)
445405d84034564a6433074e0f920f74105ba499

=======================
     Test Results 
=======================
# Tests Discovered : 11399
# Passed           : 11035
# Failed           : 0
# Skipped          : 364
=======================
constructing /home/sjlee/unit_test/dumplings.txt
270 minutes and 5 seconds taken to run CoreCLR tests.

With testsUnsupportedOnARM32.txt set as follow

JIT/Directed/tailcall/tailcall/tailcall.sh
JIT/jit64/opt/cse/HugeArray1/HugeArray1.sh
JIT/Methodical/tailcall_v4/hijacking/hijacking.sh
JIT/Methodical/tailcall_v4/smallFrame/smallFrame.sh
JIT/Methodical/xxobj/sizeof/_il_dbgsizeof32/_il_dbgsizeof32.sh
JIT/Methodical/xxobj/sizeof/_il_dbgsizeof64/_il_dbgsizeof64.sh
JIT/Methodical/xxobj/sizeof/_il_dbgsizeof/_il_dbgsizeof.sh
JIT/Methodical/xxobj/sizeof/_il_relsizeof32/_il_relsizeof32.sh
JIT/Methodical/xxobj/sizeof/_il_relsizeof64/_il_relsizeof64.sh
JIT/Methodical/xxobj/sizeof/_il_relsizeof/_il_relsizeof.sh
JIT/Regression/JitBlue/devdiv_902271/DevDiv_902271/DevDiv_902271.sh

Nice! So now what can we do about those unsupported/skipped ones?

Great Job! Well done guys...! Soon as this is ready I can get back to my R PI3 based, heating system controller :)

Excited to see official ARM release as well!

I'd love to see an official ARM release also - is there any idea on the next steps that are necessary to make this happen ?

[Congratulations to all involved in making this happen - great work guys]

Just to let you know, if you want to run it on the Pi3, we have a compiled version available here (packaged with our software)

Thanks ayende! Do you know if this will also execute on the Pi2?

@paultechguy my guess it it should work on Pi 2 just fine.

We haven't tested that, but I'm fairly certain that it should.

That said, it might be arm7 vs. arm6.

*Hibernating Rhinos Ltd *

Oren Eini* l CEO l *Mobile: + 972-52-548-6969

Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811

On Mon, Dec 5, 2016 at 11:22 PM, paultechguy notifications@github.com
wrote:

Thanks ayende! Do you know if this will also execute on the Pi2?


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dotnet/coreclr/issues/3977#issuecomment-264980988,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAHIs8nVGoD0YKE4UkNgD074tTayK9DGks5rFIB5gaJpZM4H7Qi3
.

The Pi2 is ARM7

Yea, I just searched and found that ARM7 note. It would still be awesome to get just a normal install of .NET core for ARM when available. It looks like folks are rock'n it so hopefully soon.

@ayende Looking at your binary package, I see some install script stuff and the compiled binaries in the dotnet package, but I'm still early days on this. Any chance you or someone else here can give me a step by step on how to get my own code running on this binary runtime? I'm not as personally concerned with building from source as I am with kicking the wheels on the running system and there's just TONS of stuff to sift through here, a lot of which is over my head without diving quite deeply into the build portion. Normally I'd love to contribute more, but I find myself with a shortage of time for side projects lately and am hoping to maintain focus on my current efforts rather than diving head-first down this rabbit hole.

Apologies, I think I must have scrolled too far, I didn't see corerun/coreconsole in the package at first. I may be able to play around without help; I'll report back if I'm still confused when I get back to my Pi

@TheXenocide I am able to run console apps, but not aspnet web apps yet with the build from @ayende.

Also you will need to use Ubuntu 16.04 image and the setup script will ensure you have the required packages installed.

From your compiled app directory (I just copied mine over via scp) and run ../path_to/Raven/dotnet/corerun MyAppBinary.dll

@Gonkers @TheXenocide @ayende FYI. We have built whole .NET Core runtime from core-setup, coreclr and corefx for Ubuntu 14.04 ARM and Ubuntu 16.04 ARM. It works on Pi2 and Pi3 and it works like Ubuntu 14.04 x64, i.e. you don't have to use corerun or coreconsole, just use dotnet.
You can download tarball and find more information at https://github.com/dotnet/core-setup/issues/725. (You can find tarballs at the end of issue.) BTW we support both arm (hardfp) and armel (softfp).

Results (out-dated. Please find the latest at https://github.com/dotnet/core-setup/issues/725)

ubuntu.14.04-arm
dotnet-ubuntu-arm.1.2.0-beta-001206-00.tar.gz
ubuntu.16.04-arm
dotnet-ubuntu-arm.1.2.0-beta-001206-00.tar.gz
debian.8-armel
dotnet-debian.8-armel.1.2.0-beta-001271-00.tar.gz
tizen.4.0.0-armel
dotnet-tizen.4.0.0-armel.1.2.0-beta-001273-00.tar.gz

@TheXenocide I just found this guide made by @Petermarcu that you might helpful.
https://github.com/dotnet/core/blob/master/samples/RaspberryPiInstructions.md

I try Ubuntu 16.04 ARM build on Orange Pi PC board (ARM Cortex-A7 Quad-Core, H3 Allwinner CPU) and it also works great.

@Gonkers, Page https://github.com/dotnet/core/blob/RaspberryPi/samples/RaspberryPiInstructions.md returns 404 Not Found
I am also interested in launching on the .net core Raspberry Pi (It is also on ARM): #https://github.com/dotnet/core/issues/140

I can not run at Raspbian... I try “debian.8-armel” and “ubuntu.16.04-arm”

@wang9563 Check the calling convention. You probably are not using armel, but armhf format in your Ubuntu.

Hm, I can get a simple "hello world" .NET Core application to run on a Raspberry Pi with Ubuntu MATE 16.04 using the linked instructions, but unfortunately I can't get my ASP.NET Core application to work. This is the latest I've got:

me@my-pi:~/myapp$ ./dotnet publish/<<Company>>.<<Product>>Web.ServiceAdapters.WebSockets.dll


Unhandled Exception: System.IO.FileLoadException: Could not load file or assembly 'System.ComponentModel.Primitives, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
   at Microsoft.Extensions.FileProviders.PhysicalFileProvider.CreateFileWatcher(String root)
   at Microsoft.AspNetCore.Hosting.Internal.HostingEnvironmentExtensions.Initialize(IHostingEnvironment hostingEnvironment, String applicationName, String contentRootPath, WebHostOptions options)
   at Microsoft.AspNetCore.Hosting.WebHostBuilder.BuildCommonServices()
   at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build()
   at <<Company>>.<<Product>>Web.ServiceAdapters.WebSockets.Program.Main(String[] args) in C:\Users\davidfallah\Documents\Visual Studio 2015\Projects\<<Product>>web-core\src\<<Company>>.<<Product>>Web.ServiceAdapters.WebSockets\Program.cs:line 10
Aborted (core dumped)

@TAGC that would be a separate issue to this.

@TAGC @khionu I encountered the same issue.

I created a separate issue for it https://github.com/dotnet/coreclr/issues/9168

I created the simplest "hello world" .NET core app in Windows 10 following the Pi readme and published with the very last 2.0.0 beta version as win10-x64 (that works fine in Windows 10 desktop), win10-arm and win8-arm. ARM versions don't work in Windows 10 (as expected) but don't work in Raspberry Pi 3 with Windows 10 IoT either.

Reading the readme instructions I guessed the win8-arm version would work. Am I correct?

I guess I need to install the .NET core runtime (2.0.0) somehow in the Pi, but can't find any document on how to do it, if even possible.

Thanks!

Yea, I've searched to the end of the internet and not found a good, reliable, set of instructions on how to build and install the latest core runtime. :-( Wouldn't it just be awesome to have an MS build for ARM and just install the package?

I'm pretty sure we will have it when ARM support is published. But so far for the beta version I would be happy with a quick&dirty workaround :)

@jaestevan @paultechguy I recently saw this mentioned on the dotnet community stand-up that may help you. Disclaimer: I have not tried it yet.

Then is Ubuntu in the Pi the only way to run the project? I was guessing it can be ran in Windows 10 IoT too.

I'm curious what you hit on Win10 IoT? You shouldn't need to install any core runtime. The instruction put the runtime with your application.

In case it would help anyone, I'm posting my instruction notes on how to install the dotnet tool on RPi3 to run .net core apps.

  1. Clone coreclr repo
  2. Compile coreclr

    1. Install everything from https://github.com/dotnet/coreclr/blob/master/Documentation/building/windows-instructions.md

    2. build.cmd arm release skiptests skipbuildpackages

  3. Clone corefx repo
  4. Compile corefx

    1. build-native.cmd -release -buildArch=arm

    2. build-managed.cmd -release -buildArch=arm -skiptests

  5. Clone core-setup repo
  6. Compile core-setup

    1. build.cmd -Configuration Release -TargetArch arm

  7. Copy output files to Windows 10 IoT
  8. Get-ChildItem "U:\netcore\corefx\" -Directory | ForEach-Object { Copy-Item $("U:\netcore\corefx{0}{0}.dll" -f $_.name) "U:\netcore\corefx" }
  9. Get-ChildItem "U:\netcore\corefx\" -Directory | ForEach-Object {Write-Host $_.name } Remove-Item -recurse

With this I got working an .NET Core console app and an ASP.NET Core web app that is running 24/7 for about 2 months.

EDIT: My notes were taken when there were no official instructions on how to install on RPi3. Consider first trying these ones https://github.com/dotnet/core/blob/master/samples/RaspberryPiInstructions.md

Sorry @jaestevan I should have been more clear. I meant to point out the screen shots of how/were you can update the framework version. To see if that would help you get it running. That was my biggest hang up for a while, figuring out building for the correct version of the framework.

@Petermarcu by following this instructions I got a CLR error: _Failed to initialize CoreCLR, HRESULT: 0X80131534_

@Gonkers oh, I see what you mean. Thanks.

@darxis I will try to build it myself when I have time, thanks.

@all: Found this link: http://developers.de/blogs/damir_dobric/archive/2015/09/22/how-to-run-net-core-application-on-pi2-win10-iot-core.aspx Still have not tried it myself but I will. Anyway I guess it should be a cleaner way when ARM support is official.

@jaestevan , I'll give it another try, hadn't done it on Win10 IoT in a few weeks. You shouldn't have to build the full stack but its always an option.

@Petermarcu Thanks Peter, let me know how it works. I can send you the details I followed if that helps.

I tried it again yesterday and everything worked really well.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.0</TargetFramework>
    <RuntimeIdentifiers>win10-arm</RuntimeIdentifiers>
  </PropertyGroup>
</Project>

dotnet restore
dotnet publish -r win10-arm

Then I copy bin\Debug\netcoreapp2.0\win10-arm\publish\* to the Pi and execute .\Helloworld.exe

I have one guess as to what is happening. Can you run dotnet --version to make sure you are on a 2.0.0 SDK and make sure you have the latest one? What I do for the pre-release SDK's is I extract the zip to some location, open up a command windows, make sure the path to where I exctracted the zip is the first thing on my path, and then run dotnet commands. Right now, 1.0.0 is winning over 2.0.0 because of an issue in the SDK that is going to be fixed in the next couple weeks.

I've just been experimenting with this and have it working with SDK version 2.0.0-preview1-005743 with runtime version 2.0.0-preview1-001915-00.

I've been able to edit and build using that latest Visual Studio 2017! All looking really good.

I've got a repository at https://github.com/sceneskope/PiDotnet which has some projects where this works. I'm actually using docker as containers for my images as the projects I'm working on have to run on multiple PIs.

I've also created a .NET Core app that runs on an RPI 3 using SDK 2.0.0-preview1-005685 and runtime framework 2.0.0-beta-001834-00.

Runs fine natively (although I get exec format errors if trying to run it within a Docker container based on microsoft/dotnet-nightly:2.0.0-beta-runtime).

The Microsoft images won't be for the ARM architecture, which is why I've created my own set of docker images! It would be great to have proper Microsoft images though.

@nrandell Oh nice, I missed that. I'll have to try it out at some point.

@nrandell Is that a fact written down somewhere? I mean no ARM docker images from MS.

@MihaMarkic have a look at https://github.com/dotnet/dotnet-docker/issues/223. I guess this is as official as it gets.

@nrandell So, they'll provide one for .net core 2.0 eventually but skip 1.x.

Upgraded and rebuilt everything with 2.0.0-preview1-001978-00 and now it works in the Pi 💃

Excuse me if this has been discussed before, but will this run on Raspberry Pi Zero too?

Unfortunately not, the Pi Zero is only ARMv6, Mono will work there though.

On Mon, 17 Apr 2017, 11:58 Andrei Rînea, notifications@github.com wrote:

Excuse me if this has been discussed before, but will this run on
Raspberry Pi Zero too?


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dotnet/coreclr/issues/3977#issuecomment-294452108,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAbA_0ZBbplbgWzERjKkrw-MV7buPiDxks5rw0XvgaJpZM4H7Qi3
.

I think pi 1 and zero have ARMv6k that has some enhancements compared to ARMv6.

@wanton7 Hoo boy, even more versionitis for us to keep straight?

@masonwheeler not sure what the differences are but example next golang (1.9 i think) is going to still support ARMv6k but they are dropping support for ARMv6. So must be something significant.

@Petermarcu I propose to close this issue; we're far past the "ARM 32-bit progress" title -- we have many thousands of tests running on ARM Linux in our CI. Agreed?

I agree. ;)

Was this page helpful?
0 / 5 - 0 ratings

Related issues

aggieben picture aggieben  ·  3Comments

iCodeWebApps picture iCodeWebApps  ·  3Comments

jzabroski picture jzabroski  ·  3Comments

Timovzl picture Timovzl  ·  3Comments

bencz picture bencz  ·  3Comments