Runtime: PowerPC and AIX support

[bencz]

After some research, I checked that the .net core has no supported for PowerPC or AIX ... Are there plans to implement support for the Power architecture and the AIX operating system?

[janvorli]

The largest amount of work is always in adding support for a new architecture to the JIT. We have JIT for arm, arm64, x86 and x64 architectures. There are currently no plans on adding new architectures support, but nothing prevents the community from starting such an effort.

[ghost]

@janvorli, based on your experience, when aarch and aarch64 support were added, is there any rough checklist to support more architectures like ppc64le, mips64 etc. and generally RISC? There is also https://github.com/NETMF/netmf-interpreter by Microsoft, which has similar spectrum of support, perhaps convergence with that project will yield broader reach for CoreCLR. Not sure how feasible is the convergence with netmf, though.

[janvorli]

@kasper3 coreclr contains an interpreter too (disabled by default), but it hasn't been turned on or tested for a very long time. I just remember someone made coreclr compile without errors when the FEATURE_INTERPRETER is enabled and there were some attempts to use it by someone from the community.

As for new architectures, a coarse checklist (in an arbitrary order) would be :

• ASM helpers
• Thunks (e.g. the shuffle thunk for delegates) generated as machine code
• Processor context related stuff in stack walking, thread suspension and exception handling.
• Calling convention related code, like the arg iterator, interop, ... The more complex the calling convention, the more work is needed. Structs passed in registers is an example of such a complexity.
• JIT - this is the largest amount of work.
• Unwind info for the new architectures. For the existing ones, we use Windows unwind info format for the JITted code, for architectures that windows don't support, it would need to be designed from scratch.
• GC info format may need to be updated for the new architectures.
• If these architectures are big endian, there might be changes needed due to that.
• Possibly other stuff I've forgotten to mention

As for AIX, it is hard to tell how much surprises it would bring. I have no idea how much it differs from Linux.

[mattwarren]

@kasper3 I wrote a post about the interpreter that you might find useful, see The .NET IL Interpreter.

But as @janvorli mentioned, work was done to make it easier to enable (so you can ignore that part of the post) see dotnet/coreclr#11188 and dotnet/coreclr#11252

[ghost]

@janvorli, @mattwar, thanks, didn't knew about interpreter feature. For the basic interpreter to work in an antoconfig manner, we could probably integrate coreclr interpreter with the qemu assembly transpilation component https://github.com/qemu/qemu/tree/47d3b60858d90ac8a0cc3a72af7f95c96781125a/target, in order to run some code on non-supported architectures (slow version). Gradually, the full support can be added starting from ASM helpers from Jan's list. I am not sure how feasible is Xassembly-to-Yassembly ops transpilation using qemu? Maybe that component alone can be compiled as an independent libasmconv (named after libiconv) from qemu sources for such experimentations (not for production usage).

[ghuntley]

I would love see a power on Linux bringup. AIX isn't worth the engineering cycles tho - see my thread on twitter. https://twitter.com/GeoffreyHuntley/status/979453657472053248?s=19

[NattyNarwhal]

for the audience at home, if you want .NET on AIX or i, I maintain the Mono port for that - somewhat of a WIP, but it compiles all the way through and passes more tests than I expected

[afxgroup]

a Powerpc port would be really great 👍

[turabek]

We also want to run our Core Banking Application on Power PC Processors PPC64LE. May I know from .net core team is this on your roadmap? When this will be possible?

[ghuntley]

It's not on their roadmap @turabek

There are currently no plans on adding new architectures support, but nothing prevents the community from starting such an effort.

[bencz]

Is it not possible to use the JIT used in the .net of xbox?
Or rather use it as a starter?

[xiangzhai]

Hi @mattwarren

@kasper3 I wrote a post about the interpreter that you might find useful, see The .NET IL Interpreter.

Thanks for your great blog The .NET IL Interpreter.

But as @janvorli mentioned, work was done to make it easier to enable (so you can ignore that part of the post) see dotnet/coreclr#11188 and dotnet/coreclr#11252

But when I migrated to the master https://github.com/dotnet/coreclr/pull/24513 Interpreter seems not work:

Histogram of method executions:
   # of execs   |   # meths (%)    |   cum % | % cum execs
   -------------------------------------------------------
            0   |       0 ( -nan%) |   -nan% |   -nan%
            1   |       0 ( -nan%) |   -nan% |   -nan%
            2   |       0 ( -nan%) |   -nan% |   -nan%
            3   |       0 ( -nan%) |   -nan% |   -nan%
            4   |       0 ( -nan%) |   -nan% |   -nan%
            5   |       0 ( -nan%) |   -nan% |   -nan%
            6   |       0 ( -nan%) |   -nan% |   -nan%
            7   |       0 ( -nan%) |   -nan% |   -nan%
            8   |       0 ( -nan%) |   -nan% |   -nan%
            9   |       0 ( -nan%) |   -nan% |   -nan%
           10   |       0 ( -nan%) |   -nan% |   -nan%

For methods sorted in ascending # of executions order, cumulative % of executions:

Total number of calls from interpreted code: 0.
    Also, 0 are intrinsics; 0 of these are not currently handled intrinsically.
    Of these, 0 to potential property getters (0 of these dead simple), 0 to setters.
    Of the dead simple getter calls, 0 have been short-circuited.

Token resolutions by category:
Category     |  opportunities  |   calls   |      %
---------------------------------------------------
   Undefined |               0 |         0 |   0.00%
 Constrained |               0 |         0 |   0.00%
      NewObj |               0 |         0 |   0.00%
      NewArr |               0 |         0 |   0.00%
     LdToken |               0 |         0 |   0.00%
       LdFtn |               0 |         0 |   0.00%
   LdVirtFtn |               0 |         0 |   0.00%
    SFldAddr |               0 |         0 |   0.00%
      LdElem |               0 |         0 |   0.00%
        Call |               0 |         0 |   0.00%
       LdObj |               0 |         0 |   0.00%
       StObj |               0 |         0 |   0.00%
       CpObj |               0 |         0 |   0.00%
     InitObj |               0 |         0 |   0.00%
      IsInst |               0 |         0 |   0.00%
   CastClass |               0 |         0 |   0.00%
    MkRefAny |               0 |         0 |   0.00%
   RefAnyVal |               0 |         0 |   0.00%
      Sizeof |               0 |         0 |   0.00%
      StElem |               0 |         0 |   0.00%
         Box |               0 |         0 |   0.00%
       Unbox |               0 |         0 |   0.00%
    UnboxAny |               0 |         0 |   0.00%
       LdFld |               0 |         0 |   0.00%
      LdFldA |               0 |         0 |   0.00%
       StFld |               0 |         0 |   0.00%
   FindClass |               0 |         0 |   0.00%
   Exception |               0 |         0 |   0.00%

No printf in the Interpreter::ExecuteMethod or Interpreter::Terminate:

diff --git a/src/vm/interpreter.cpp b/src/vm/interpreter.cpp
index b4b18cb..53ae5c2 100644
--- a/src/vm/interpreter.cpp
+++ b/src/vm/interpreter.cpp
@@ -1878,6 +1878,7 @@ void Interpreter::DoMonitorExitWork()

 void Interpreter::ExecuteMethod(ARG_SLOT* retVal, __out bool* pDoJmpCall, __out unsigned* pJmpCallToken)
 {
+    printf("DEBUG: %s:%d\n", __FILE__, __LINE__);
 #if INTERP_DYNAMIC_CONTRACTS
     CONTRACTL {
         THROWS;
@@ -6511,6 +6512,7 @@ CORINFO_CLASS_HANDLE Interpreter::GetTypedRefClsHnd(CEEInfo* info)

 void Interpreter::Initialize()
 {
+    printf("DEBUG: %s:%d\n", __FILE__, __LINE__);
     assert(!s_initialized);

     s_InterpretMeths.ensureInit(CLRConfig::INTERNAL_Interpret);
@@ -6553,6 +6555,7 @@ void Interpreter::InitializeCompilerStatics(CEEInfo* info)

 void Interpreter::Terminate()
 {
+    printf("DEBUG: %s:%d\n", __FILE__, __LINE__);
     if (s_initialized)
     {
         s_methodCacheLock.Destroy();

As comment mentioned:

// If we failed to jit, then fall back to the primary Jit.

AltJIT just fall back to primary JIT, but not Interpreter?
How to force fall back to interpreter? By hacking set ret to CORJIT_SKIPPED:

diff --git a/src/vm/jitinterface.cpp b/src/vm/jitinterface.cpp
index 502b5ad..0be76f6 100644
--- a/src/vm/jitinterface.cpp
+++ b/src/vm/jitinterface.cpp
@@ -12145,6 +12145,7 @@ CorJitResult invokeCompileMethodHelper(EEJitManager *jitMgr,

     bool isInterpreterStub   = false;
     bool interpreterFallback = (s_InterpreterFallback.val(CLRConfig::INTERNAL_InterpreterFallback) != 0);
+    printf("DEBUG: interpreterFallback: %d\n", interpreterFallback);

     if (interpreterFallback == false)
     {
@@ -12169,6 +12170,8 @@ CorJitResult invokeCompileMethodHelper(EEJitManager *jitMgr,
                                           nativeSizeOfCode);
     }

+    // Force fall back to interpreter
+    //ret = CORJIT_SKIPPED;
     if (interpreterFallback == true)
     {
         // If we're doing an "import_only" compilation, it's for verification, so don't interpret.

And I found that AArch64 Remove unnecessary interpreter fallback https://github.com/dotnet/coreclr/pull/20470 So is it able to enable Interpreter at master now?

Thanks,
Leslie Zhai

[mattwarren]

I've not used the Interpreter since I wrote that post, so my memory is a bit hazy! But which (if any) environment variables have you set? See https://mattwarren.org/2017/03/30/The-.NET-IL-Interpreter/#Enabling%20the%20Interpreter and scroll down to the table of 'Host Configuration Knobs' for a bit more info. Or see https://github.com/dotnet/coreclr/blob/master/Documentation/project-docs/clr-configuration-knobs.md#interpreter-configuration-knobs.

From what I remember, you need to at least set the Interpret value to the method(s) you want to have interpreted

[xiangzhai]

I've not used the Interpreter since I wrote that post, so my memory is a bit hazy! But which (if any) environment variables have you set? See https://mattwarren.org/2017/03/30/The-.NET-IL-Interpreter/#Enabling%20the%20Interpreter and scroll down to the table of 'Host Configuration Knobs' for a bit more info. Or see https://github.com/dotnet/coreclr/blob/master/Documentation/project-docs/clr-configuration-knobs.md#interpreter-configuration-knobs.

From what I remember, you need to at least set the Interpret value to the method(s) you want to have interpreted

Set:

$ export | grep COMPlus
declare -x COMPlus_DumpInterpreterStubs="1"
declare -x COMPlus_Interpret="Main"
declare -x COMPlus_InterpreterFallback="1"
declare -x COMPlus_InterpreterPrintPostMortem="1"
declare -x COMPlus_TraceInterpreterEntries="1"
declare -x COMPlus_TraceInterpreterIL="1"
declare -x COMPlus_TraceInterpreterVerbose="1"

HHH, perhaps because this https://github.com/dotnet/coreclr/pull/11252

[to11mtm]

Another Upvote, but I want to speak very specifically;

POWER Architecture would be huge for .NET for a number of reasons, including the existence of open, auditable platforms and the open-sourcing of the ISA and reference designs.

But really, the security angle is where I see a big win.

[YohanSciubukgian]

Do you also plan to support PPC 32 bits ?

[aix1]

Upvote; applications that uses .NET is restricted to our x86 only and for their high demands on Performance and RAS we need the .NET core parts on ppc64le.

[rickswah]

One more Upvote! there is a lot of PPC64LE and PPC64 server around the world using just Java or PHP for the main systems.. the world need .net for PowerPC!!! @NattyNarwhal do a really good job on Mono Project... but, there is some diffs. from .net core

[Tustra]

upvote

[mihaimyh]

Any plans for AIX support? Lots of us are using AIX servers.

[neosaldina]

Is there any plan to start implementing the JIT for ppc, ppc64 or ppc64el ?

[leo-lb]

I also would like to run .NET Core on ppc64le. My workstation system runs Fedora with Linux.

[danmosemsft]

There remains no plan for Microsoft support of AIX, but community work is welcome - FreeBSD has made contributions in this way.

[neosaldina]

Okay, Microsoft may not have plans to support AIX, but what about PowerPC ( ppc, ppc64 and ppc64le ) ?

[neosaldina]

I know that the Mono project has a system to host sites made in ASP.Net and ASP.Net MVC, if I'm not mistaken this project is called XSP ... the last time I tested it on ppc64, an error occurred when I tried to open the site...
@NattyNarwhal you are supporting Mono for ppc, can you tell if XSP is working correctly on ppc64 for ASP.Net MVC?

[tmds]

@janvorli @danmosemsft Mono requires interpreting for some platforms, and .NET Core and mono runtimes are coming closer together. Could we be moving in a direction where there is a production-grade .NET interpreter that can be built from source?

Though it would not perform as a good as a JIT, it would reduce the effort significantly to port .NET.

[danmosemsft]

@marek-safar

[NattyNarwhal]

Mono interpreter requires per-platform trampolines, generated by the JIT. There was work from me and @lewurm to use the C based (but must be pregenerated) trampolines that WebAssembly uses (no JIT there) and fixing up the interpreter more, including making them endian and 64-bit pointer safe, but it wasn't complete.

I'm also still not clear on the whole .NET 5/Mono integration story and how exactly Mono comes into play with dotnet and actually building it. The portability story on the Core side of thing seems really ambiguous in general to me and reads as a lower priority, while Mono has traditionally picked up the slack there.

[marek-safar]

Mono interpreter is already part of this repo and there is ongoing effort to make it stable and supported (see #32754 for example).

@NattyNarwhal someone will need to make an effort to build relevant runtime packages of dotnet/runtime for such platforms. It's something the community can do and .NET team can help with in-terms of landing the changes or advising on the required changes.

[tmds]

Thanks @NattyNarwhal @marek-safar . It's good to know mono interpreter is in the repo, and you'll help community to get it working on unsupported platforms. Are there important limitations, besides speed, you are aware of?

[marek-safar]

It depends on how weird is the new platform but in general, there should not be anything that cannot be changed.