Go: proposal: cmd/go: add GOMIPS64=r2 for mips64r2 code generation

[mengzhuo]

According to the wiki the minimal requirement of MIPS64X is MIPS III.

MIPS III is introduced in 1991 the cpu is R4000 (which clock is only 200MHz) and MIPS64r1 adds instructions like CMOV, CLZ, ROR for better performance but we can't use them due to the minimal requirement.

I propose add a GOMIPS64REV flag for compiler. We having GOMIPS64 flag is for hardware float (hardfloat/softfloat), i.e. GOMIPS64=hardfloat now. For the backward-compatible, we don't change it and add a new flag GOMIPS64REV.

Further more, MIPS had announced MIPS64r6 which deleted some instructions like "likely branch" and "unaligned read/load" which is another reason we should to add GOMIPS64REV flag if we want to support MIPS64r6 at compile time in the future.

Therefore the variable of "GOMIPS64REV" will be one of

mips64r2
mips64r6

mipsiii is same as current minimal support.
No base instructions were introduced between mips64r2 - mips64r5.

Related topic:
https://github.com/golang/go/issues/31265
https://groups.google.com/forum/#!topic/golang-dev/utpaIeO9lx4

@cherrymui @randall77

[cherrymui]

If we want to do this, I think we should just use the GOMIPS64 environment variable, with a comma-separated list, with default values if not all values are provided. For example,

• GOMIPS64=softfloat,r2 means softfloat+MIPS64r2
• GOMIPS64=softfloat means softfloat + minimal ISA (default ISA)
• GOMIPS64=r2 means hardfloat (default FP mode) + MIPS64r2
• GOMIPS64="" means hardfloat + minimal ISA (default for both)

I think this is already proposed when we introduced the hard/softfloat mode. There is really no need to write "mips(64)" in the setting. Maybe we can drop the "r" as well (just GOMIPS64=2, more like GOARM)?

That said, if there are only a few instructions, I think we may want to try run-time CPU feature detection and see how that goes.

[beenshi]

I support adding all to the GOMIPS64 as a comma separated list, than a new env var. But I do not support runtime check. Just like I proposed in #29373

[cherrymui]

To clarify, my personal vote would be not to do this. My comment above is under the assumption that we really want to do this.

For one, the MIPS port is less well tested already, and adding more flavors would make testing even harder. We need softfloat/hardfloat as there is really no way around, and it is well tested as the implementation is mostly portable. It is different for a few new instructions. I'm not sure the benefit overweighs the cost.

[mengzhuo]

I agree with GOMIPS64=hardfloat,r2.

After dig into MIPS64 instruction set manual

I found we can add these instructions into compiler along with CLZ, CLO

SEB Sign-Extend Byte 
SEH Sign-Extend Halftword 
DEXT Doubleword Extract Bit Field 
DEXTM  Doubleword Extract Bit Field Middle 
DEXTU  Doubleword Extract Bit Field Upper 
DINS  Doubleword Insert Bit Field 
DINSM  Doubleword Insert Bit Field Middle 
DINSU  Doubleword Insert Bit Field Upper
DSBH  Doubleword Swap Bytes Within Halfwords
DSHD  Doubleword Swap Halfwords Within Doublewords
DROTR Doubleword Rotate Right
DROTR32 Doubleword Rotate Right Plus 32 
DROTRV Doubleword Rotate Right Variable
ROTR Rotate Word Right
ROTRV Rotate Word Right Variable
MADD.fmt Floating Point Multiply Add
MSUB fmt Floating Point Multiply Subtract
NMADD fmt Floating Point Negative Multiply Add
NMSUB fmt Floating Point Negative Multiply Subtract
RECIP fmt Reciprocal Approximation
RSQRT fmt Reciprocal Square Root Approximation

As matter of tests, we can setup mips64enc.s like arm64 did.

[gopherbot]

Change https://golang.org/cl/203441 mentions this issue: cmd/compile, cmd/go: add mips64r2 flag in GOMIPS64

[gopherbot]

Change https://golang.org/cl/203443 mentions this issue: cmd/compile, cmd/go: add mips64r2 flag in GOMIPS64

[martisch]

I agree with cherrymui@ in that it is unclear (at least to me) whether the added complexity is worth the benefit.

Will there be builders for the new combinations if we do not have any?
Do we have (or even support) codegen tests for these different variants?

It would also be nice to know what the performance benefits are vs not doing this or doing runtime detection of the features (if that is possible).

If pre r2 is not widely used an alternative might be to just make r2 the new minimal requirement.

[mengzhuo]

I agree with cherrymui@ in that it is unclear (at least to me) whether the added complexity is worth the benefit.

Will there be builders for the new combinations if we do not have any?
Do we have (or even support) codegen tests for these different variants?

We don't have codegen test yet. I will work on it soon.

It would also be nice to know what the performance benefits are vs not doing this or doing runtime detection of the features (if that is possible).

That is impossible. I've asked about Linux kernel maintainer to add ISA to uapi/hwcap but they insisted that's a compiler issue.
However they do offer DSP, MSA, CRC32 feature detection.

If pre r2 is not widely used an alternative might be to just make r2 the new minimal requirement.

I would love to set r2 as minimal requirement same as mips32x do.

[gopherbot]

Change https://golang.org/cl/204297 mentions this issue: cmd/asm: add encode testes for MIPS64x

[rsc]

It seems like the alternative is to make r2 the minimum requirement for mips64.
Does anyone have any data about what mips64 chips are typically used today and whether they have r2 or not?

[mengzhuo]

I had send email to MIPS company and ask for more information.
AFAIK, Loongson 3A1000 the first MIPS64r2 in Loongson series which is released in 2009.

Also Debian require of MIPS64r2

[rsc]

Who are the Go mips64 port maintainers? If they can come to a consensus that we should require r2, then let's do that.

[gopherbot]

Change https://golang.org/cl/209557 mentions this issue: doc/go1.14: upgrade minimal requirement of MIPS64x in Go 1.15

[mengzhuo]

@cherrymui Is it ok to upgrade the minimal requirement to r2?

[cherrymui]

I would like to see it can provide enough benefits before bumping up the minimal requirement.

[ianlancetaylor]

@mengzhuo Can you send a message to golang-nuts asking whether anybody still uses older MIPS chips? I feel like we don't have much real data here.

Also, who are the mips and mips64 port maintainers?

[cherrymui]

Also, who are the mips and mips64 port maintainers?

I guess I might be. How can I know for sure?

[ianlancetaylor]

I would typically look to see who committed the port in the first place.

CC @vstefanovic

[cherrymui]

Hmmm, then that is still me (for mips64)...

[ianlancetaylor]

Excellent, then you get to decide.

[minux]

Loongson 2F is still using MIPS III.
2A is the first Loongson to support MIPS64r2, but it is released only 10
years ago. (FTR, i386 port still supports Pentium MMX, which was released
23 years ago!)

I regret very much that I didn't push back hard enough on bumping minimum
architecture for BE ppc64, and that removed a lot of perfectly reasonable
hardware to run Go.
I hope this won't happen for mips64.

The list of additional instructions to that we could use for MIPS64r2
doesn't look common/generic enough to risk removing support for old
hardware.
I think we need clear demonstration that using more advanced instruction
could lead to non-trivial performance gain for general Go programs before
even considering bumping the minimal requirement.

I'm OK with adding GOMIPS64=r2, and adding new instructions to assembler,
but not ok with bumping minimal architecture requirement.

[martisch]

I think we need clear demonstration that using more advanced instruction could lead to non-trivial performance gain for general Go programs before even considering bumping the minimal requirement.

I think the same (making sure the advantage is big enough to warrant the change) can be said for adding a new GOMIPS64 option. A new option will mean more compiler complexity which makes maintaining the compiler and changing the compiler harder. E.g. making any generic rule change needs to understand and test one more configuration. This thereby also has an impact on optimizations efforts for other architectures. This also brings the question if there is capacity to add a new buildbot to make sure neither r2 or non-r2 ports regresses.

[minux]

>

I think the same (making sure the advantage is big enough to warrant the
change) can be said for adding a new GOMIPS64 option. A new option will
mean more compiler complexity which makes maintaining the compiler and
changing the compiler harder. E.g. making any generic rule change needs to
understand and test one more configuration. This thereby also has an impact
on optimizations efforts for other architectures. This also brings the
question if there is capacity to add a new buildbot to make sure neither r2
or non-r2 ports regresses.

I totally agree. My intention (that I didn't make clear in my last reply)
is to first see the performance impact and then start discussing potential
ways forward. Adding GOMIPS64=r2 is the first option if the performance
impact is good enough, and rising minimum requirement should be treated as
a last resort (my thinking is that, the minimum requirement should be set
such that, without those instructions, basic Go features don't even work
without extreme efforts. As a concrete example, for a RISC-V port [where it
has clearly defined ISA subsets], D/F [double/single precision floating
point] should definitely be optional, and it's reasonable to require I
[integer], A [atomic] and M [multiplication] extensions. Anything beyond
IMA, should not be required by Go.

In fact, I think that's precisely the criteria for some of the mainstream
architectures like 386 and amd64. I guess we won't even consider a proposal
to raise minimum required amd64 ISA to AVX, even though AVX processors
became available in 2011 and AVX should bring significant performance boost
for lots of programs (not to mention that VEX 3-operand format will
simplify compiler support). Yet, we are considering raising MIPS minimum
requirement for product that was introduced two years earlier (Loongson 3A
introduced in 2009 is the first Loongson to support r2.)
Given the number of people running Go on modern amd64 hardware and the
potential benefits, it is probably worthwhile to introduce AVX support via
an hypothetical GOX86=avx option, but the proposed GOMIPS64=r2
meets neither of these two bars (very wide-spread hardware support &
significant performance benefits).

We have a policy for using assembly code in packages (
https://golang.org/wiki/AssemblyPolicy), but IMO we don't have a policy for
adding ISA variants to compilers. Perhaps it's time to draft such a policy
and set some specific requirements.

[rsc]

To summarize:

• If the compiler knew the target was r2 or later, it could use a few new instructions, including apparently word rotate, float fma, and sign extend. Maybe also some conditionals.
• MIPS64 introduced r2 in 2009.
  
  
  
  • We assume all chips released since then support r2.
  
  
• Debian requires r2 as of Stretch (two releases ago).
• Fedora also requires r2.
• Gentoo looks like it supports Loongson 2F (pre-r2).

There are two questions to answer in this issue:

1. Should we add optional support for r2 via GOMIPS64?
2. Should we hard code an r2 requirement?

It seems like we need more information:

• How big would the performance wins be if typical Go programs compiled assuming r2?
• How much of existing MIPS64 usage is on pre-r2 systems?
  (The Debian/Fedora requirements suggest they believe it is not much / not worth worrying about.)

Does anyone have any concrete data about these?

[minux]

I think just sampling general Linux distributions will give skewed results,
at least for an embedded focused architecture like MIPS.

Debian and Fedora mostly target desktop-capable systems, but most MIPS
systems are actually used in embedded systems. (e.g. most in routers and
runs more embedded flavored distributions like OpenWRT.)

To put it another way, the reason that Debian/Fedora requires MIPS64r2
might not be that there aren't many pre-MIPS64r2 machines out there, but
most pre-MIPS64r2 machines are not powerful enough to support desktop.
Gentoo is better indication because Gentoo can be (and more likely to be)
used on those smaller machines. (I use Gentoo on Loongson 2F laptop.)

Also, it's also not true that products introduced after MIPS64r2
introduction will support MIPS64r2. Unlike x86, and perhaps ARM, MIPS is
widely implemented by individual companies, not licensed from the company
that defines the MIPS ISA. Therefore, it might take a long time for a new
revision of ISA to propagate to products. Also, for embedded architectures
like MIPS, their average product half life will also be significantly
longer than more mainstream architectures like x86, so only consider new
products ignores all those working products already deployed in the field
that could be benefited by Go.

The most likely significant new instruction in MIPS64r2 for Go is probably
sign extension. Rotation is not used much in general purpose code, and fma
is most likely used in math kernels that probably could benefit from
separate assembly implementations and most embedded MIPS64 processors don't
have FPU anyway, so supporting fma might not be that crucial.

Lastly, given the current situation with the company that governs the MIPS
ISA and the rise of RISC-V, I personally think MIPS usage will gradually
decline and remaining uses will be even more focused on embedded than
general purpose computing. In the long run, it might not be worthwhile to
invest more in MIPS, than in free and open alternatives like RISC-V.

[rsc]

@mengzhuo, @minux is arguing that there is not really a significant benefit to making the assumption of r2. As I said last week, we need to know how big the performance win would be. Do you have any numbers or other specifics?

In the absence of a compelling reason to do this, and also noting that there's not really consensus among the people interested in mips64, it sounds like we are leaning toward not doing this. I won't mark it likely decline yet, in the hope of having performance numbers next week.

[smasher164]

FWIW, I don't think fma should be a reason to require r2. The floating-point implementation register (fcr0) has been available since MIPS III, and allows for runtime feature-detection when using hardfloat.

In MIPS III, most of the bits were reserved.

In MIPS Release I-V, they were filled in.

One just has to check for Has2008 and F64, since accessing this register isn't privileged.

[wzssyqa]

To summarize:

* If the compiler knew the target was r2 or later, it could use a few new instructions, including apparently word rotate, float fma, and sign extend. Maybe also some conditionals.

* MIPS64 introduced r2 in 2009.

In fact MIPS32/64 is introduced in 2002 or 2003.
Before 2011, Loongson has some legal problem with MIPS.
I guess that's why they don't have mips32/64 r2 support.

In fact, almost all mips32/64 machines that we can access is r2+.

  * We assume all chips released since then support r2.

* Debian requires r2 as of Stretch (two releases ago).

When we dicides to set r2 as Debian's baseline, we meet nearl nothing objection.
Of course, from Loongson 2F users.
In fact in nowdays, some upstream packages cannot be built for MIPS III at all,
if we decide to to support MIPS III in Debian, we have to patch quite amount of packages.

* Fedora also requires r2.

* Gentoo looks like it supports Loongson 2F (pre-r2).

By the way, clang also outputs r2 object by default.

There are two questions to answer in this issue:

1. Should we add optional support for r2 via GOMIPS64?

2. Should we hard code an r2 requirement?

It seems like we need more information:

* How big would the performance wins be if typical Go programs compiled assuming r2?

Somebody told me that it is about 10%. I haven't tested it, so I am not sure about it.

* How much of existing MIPS64 usage is on pre-r2 systems?
  (The Debian/Fedora requirements suggest they believe it is not much / not worth worrying about.)

In fact the only hardware that I know about 64bit in nowdays pre-R2 hardware is Loongson 2E/2F.

Does anyone have any concrete data about these?
If it doesn't take too much more work/maintaince, runtime detect of course is preferred.
While if runtime detect course too much work, I think that we can just set the baseline to r2.

[rsc]

Somebody told me that it is about 10%. I haven't tested it, so I am not sure about it.

It's hard to imagine what new instructions would give 10% in a typical Go program (as opposed to a microbenchmark for that one new operation).

We all agree that the Loongson 2F is the reason to keep non-r2 support around. @minux seems to care about keeping that working, and there is no documented performance win for the other systems gained by breaking Loongson 2F. That is, we have heard a concrete objection and no concrete arguments in favor.

This seems like a likely decline.

Leaving open for a week for final comments.

[rsc]

No change in consensus, so declining.