Powershell: Consider introducing $IsUnix as an abstraction over $IsMacOS and $IsLinux

[mklement0]

^{On a side note: The automatic $Is* variables aren't universally loved due to "polluting" the global namespace, but it looks like they're here to stay.}

Platforms with Unix heritage have many commonalities, especially at higher levels of abstraction, so sometimes it is sufficient to distinguish Unix platforms _as a group_ from Windows.

Thus, instead of having to use $IsLinux -or $IsMacOS or $PSVersionTable.Platform -eq 'Unix' (yes, Unix is reported on both macOS and Linux), it would be convenient to be able to say $IsUnix.

_Update to clarify_: The variable $IsUnix would not be for everyone: Its target audience are developers who need to develop portable-across-Unix-like-systems-but-not-Windows functionality.

---

Note that the commonalities are reflected even at a lower level, namely in the CoreFX implementation: as of this writing, among the Unix-related classes, 50 are abstract (*.Unix.cs), 25 are macOS-specific (*.OSX.cs), and 11 Linux-specific (*.Linux.cs)

Environment data

Written as of:

PowerShell Core v6.0.2

[vexx32]

Can we not have some kind of a separate space for variables like this, similar to the $env: variables on Windows? Like have an $Env:OperatingSystem automatic variable?

[SteveL-MSFT]

@vexx32 before we declared PSCore6 GA (aka General Availability, aka done), the @PowerShell/powershell-committee discussed having a prefix/namespace such as $ps:IsWindows or $interop:IsWindows, however, it was deemed too big a breaking change for people already dependent on the existing variables (let alone the code churn we'd endure for our own code in this repo dependent on it). Since it's already shipped we won't be changing them so they are here to stay. We can still introduce an interop namespace, but that would be additive and not replacing these variables.

I would agree there is utility in having $IsUnix

[RichardSiddaway]

Struggling to see the utility of this. We already have $IsWindows so
! $IsWindows
effectively gives the same thing.
seems to be an addition for sake of an addition rather than a practical expansion of functionality

[markekraus]

I'm opposed to this for several reasons. The first of which @RichardSiddaway has already covered regarding !$IsWindows.

The second issue I take is with using Unix on Linux and macOS:

First, the usage is contentious to the point of being inflammatory in some circles. While it is true, there is some shared heritage, it is also very true that Linus is not Unix.

Second, in my years where I worked very heavily with various *nix distributions, BSD clones, and Solaris, The term Unix was used to distinguish more UNIX-like operating systems from Linux distributions. There are certain assumptions about locations, APIs, binaries, and devices that come with Unix-like operating systems that are not applicable to Linux distributions. Using Unix to include Linux could be confusing should we ever need to make the distinction between Linux-like and Unix-like.

My final issue is that, outside of distinguishing these systems from Windows, I don't see any utility in uniting them under a common banner. Not-Windows is what you want. The ecosystems between macOS and Linux are distinct enough to make $IsUnix practically useless, IMO. Yes, some people do quite a bit of work to make their macOS more like Linux distributions, but out of the box they are are more distinct. The same can be said about the $IsLinux and the distinctive ecosystems within distribution and clone silos. However, $IsLinux does serve as a Not-Windows-or-macOS pretty well.

Knowing something is Not-Windows is helpful so you can make assumptions like WMI will be unavailable. But uniting Not-Windows them under the banner of Unix seems superfluous and inaccurate.

[markekraus]

On another note. I believe adding automatic variables should require an RFC. I would suggest drafting an RFC and moving the conversation there.

[mklement0]

@RichardSiddaway:

It's a matter of semantic directness: If I want to express that something should be done on any of the supported Unix-heritage platforms, I want to express that _directly_ (positively, in this case):

if ($IsUnix) ...

PowerShell commendably already adheres to this principle in other areas: take the -contains and -notcontains operator pair, as one example. Yes, -contains in combination with -not is _technically_ enough, but -notcontains is a more succinct and direct expression of the intent.

Additionally, on a largely hypothetical note: While ! $IsWindows is _currently_ the equivalent of the proposed $IsUnix, that would change if support for a new, different platform type were to be introduced in the future.

(Conversely, adding support for a different type of Unix-heritage platform would extend the meaning of $IsUnix, though presumably in a more predictable manner (quasi-POSIX as the common denominator).)

[mklement0]

@markekraus:

Re the _name_ "Unix" and its applicability to Linux:

First, the usage is contentious to the point of being inflammatory in some circles.

That you don't like the _name_ is no reason to dispute the utility of the _feature_ - it's a separate debate.

A few pointers regarding the name, but it can be finalized if and once a consensus has been reached as to whether the feature is worth introducing:

It is the case variant UNIX (all-caps) that is trademarked; I think that the term Unix to loosely refer to all platforms with a UNIX _heritage_ is well-established - see https://en.wikipedia.org/wiki/Unix; to specifically address whether Linux should be considered a "Unix" system (from https://en.wikipedia.org/wiki/Unix-like#Categories):

Dennis Ritchie, one of the original creators of Unix, expressed his opinion that Unix-like systems such as Linux are de facto Unix systems.[12]

These classifications are not an exact science. To me, something like $IsUnixLike sounds like overkill and, given the presence of $IsMacOS and $IsLinux, no one should be tempted to infer that $IsUnix was chosen out of ignorance of the fact that "Unix" isn't a single, specific OS.

[mklement0]

@markekraus:

Re the usefulness of the "Unix" abstraction and whether it warrants a _direct_ expression in something like $IsUnix:

The variable $IsUnix would not be for everyone: Its target audience are developers who need to develop portable-across-Unix-like-systems-but-not-Windows functionality.

Countless questions on Stack Overflow will tell you that there's a need for that.
(And I've personally spent years answering questions about how to make commands / POSIX-like shell scripts run on both macOS and Linux (and others).)

[markekraus]

@mklement0

That you don't like the name is no reason to dispute the utility of the feature - it's a separate debate.

Let's not make this personal, OK? This isn't even about me. I personally don't care if Unix is used to refer to Linux. But, some people really do. I'm bring awareness to that fact. Since PowerShell is a Microsoft product, there is an image issue to consider when dealing with other communities. I'm cautioning against it for that reason alone. I don't have any personal stake in it.

[mklement0]

Let's not make this personal, OK?

I agree. And I didn't. I merely pointed out a logical fallacy: That a feature shouldn't be given the specific name X is no argument against the feature itself. That's all I was trying to say.

Since PowerShell is a Microsoft product, there is an image issue to consider when dealing with other communities.

It's commendable that you care about that and it is a debate worth having (I've already provided my input) - just separate from whether the feature should be introduced or not.

[RichardSiddaway]

Long term are there any plans to port PowerShell onto Unix (as opposed to Linux) platforms. If there is ven the chance of that happening then $IsUnix should be reserved for that case.
If its used as a bucket term for Linux and mac then the confusion when PowerShell on Unix arrives would be spectacular

[mklement0]

@RichardSiddaway: UNIX® is a trademarked _standard_, not a specific OS.

Learn about UNIX®, an Open Group standard ...

http://opengroup.org/unix

A specific OS can be _certified as_ UNIX®[-compliant]:

Only systems that are fully compliant and certified according to the Single UNIX Specification are qualified to use the UNIX® trademark.

http://www.opengroup.org/certifications/unix

SUS (the Single UNIX Specification) is synonymous with POSIX.
These are _abstract standards_ that a certified OSs must comply with.

Here's the list of the currently certified UNIX® systems.

As you can see, macOS _is_ a UNIX® platform, albeit conformant to an _earlier_ version of the standard, UNIX 03 a.k.a. SUSv3 a.ka. POSIX:2001 (current is UNIX V7 a.k.a SUSv4 a.k.a POSIX:2008).

As for Linux:

From https://en.wikipedia.org/wiki/Single_UNIX_Specification#Non-registered_Unix-like_systems

• "Vendors of Unix-like systems such as Linux and FreeBSD do not typically certify their distributions" and "and do not install [the] full [set of] POSIX utilities by default."

• But (emphasis added): "For Linux, the Linux Standard Base was formed in 2001 as an attempt to standardize the internal structures of Linux-based systems for increased compatibility. It is based on the POSIX specifications, the Single UNIX Specification, and other open standards, and also extends them in several areas; but there are some conflicts between the LSB and The POSIX standards."

---

In practice, most Linux distros are _largely_ POSIX-compatible, and targeting a subset of the POSIX features is the key to portability. Knowing which specific POSIX features _cannot_ be relied upon requires experience, but writing portable-across-Unix-like-platforms scripts is doable and desirable.

[SteveL-MSFT]

I forgot about !$Windows which I agree covers the functional aspect. Unless there's another platform we support in the future that is not "Unix" and also not Windows, I rescind my support of $IsUnix.

[mklement0]

@SteveL-MSFT:

Let me try to summarize, and then I'll leave it be:

Again, it comes down to having a direct expression of one's intent:

If I want to target all supported Unix platforms as a group so as to write Unix-portable code (which I've hopefully demonstrated is a real thing), I want to be able to express that directly with something like $IsUnix, and do not want to think about it in terms of what it is _not_ (leaving the aspect that the negative definition can change over time aside).

Yes, it's easy enough to roll one's own - $IsUnix = $PSVersionTable.Platform -eq 'Unix' - but it's cumbersome to do it every time, and, conversely, adding it as an automatic variable is a technically trivial addition that is unlikely to create a name conflict.

Having "Unix" as an abstract entity is also being discussed in the context of #require and module manifests, where I think it would be equally useful.

[markekraus]

I don't believe this change would be trivial. https://github.com/search?q=language%3Apowershell+%24IsUnix&type=Code&utf8=%E2%9C%93
If $IsUinx were made a read-only automatic variable like $IsWindows, there would be a significant number of broken scripts.

I believe for serious consideration, this should be moved to an RFC.

[mklement0]

@markekraus:

Thanks for digging deeper.

It's a fair concern, given the lack of a distinct namespace for automatic variables (sigh).

Yes, making it _read-only_ would definitely break scripts, but not making it read-only is an option (just like many existing automatic variables that _should be_ read-only aren't, for fear of breaking things).

What your results tell me, however, is that there _is_ a need for such a variable; let's take the line from the Platypus repo, for instance:

$global:IsUnix = $MyIsLinux -or $MyIsMacOS

Below is the full list of distinct definitions, and all of them are variations of what $IsUnix would be meant to do:

[bool] $global:isUnix = [System.Environment]::OSVersion.Platform -eq [System.PlatformID]::Unix
IsUnix        = $PSVersionTable.PSEdition -eq "Core" -and -not $IsWindows
$script:IsUnix = $PSVersionTable.PSEdition -and $PSVersionTable.PSEdition -eq "Core" -and !$IsWindows
$IsUnix # assigned via function is_unix() { $PSVersionTable.Platform -eq 'Unix' }
Set-Variable -Name IsUnix -Value $false -Scope Global # guarded by if ($PSVersionTable.PSVersion.Major -lt 6), so not a problem
$global:IsUnix = $MyIsLinux -or $MyIsMacOS

If more people here indicate their support for the idea, I'm happy to write an RFC.

[markekraus]

What your results tell me, however, is that there is a need for such

Need? no. Desire, yes. Most of those mean !$Windows.

[mklement0]

"Need" as in: "This isn't built in, so I need to make my own."
The authors don't _mean_ !$IsWindows, otherwise they would have used just that (whether it is _technically_ equivalent, currently, is beside the point) - they wanted a _positive_ expression of their intent.

[SteveL-MSFT]

@PowerShell/powershell-committee discussed this and like the concept, however we would like community feedback on considering $IsPosix instead of $IsUnix

[dragonwolf83]

I prefer $IsPosix. The intent is more clear and avoids the confusion that I think a $IsUnix would bring.

[mklement0]

At the end of the day, I'm happy with either, as long as there is a way to _positively_ express the commonality of Unix-like platforms.

The caveat re $IsPosix is that it reflects _intent_, not _fact_:

• Limiting your code to [a subset of the] POSIX-compliant features is what will enable it to work across all supported Unix-like platforms; note, however, that there are additional features/utilities you can rely on _de facto_.

• _Factually_, only macOS is a POSIX-certified platform.

[SteveL-MSFT]

@mklement0 the committee wasn't sure if Linux is POSIX-certified (although we suspect Red Hat specifically probably is), but wanted to avoid any controversy of "Unix vs not-Unix" as macOS is Unix, but GNU is not Unix...

[mklement0]

@SteveL-MSFT:

I've quoted the Wikipedia page above, but if we go straight to the source, we'll see that _no_ Linux system is certified (but macOS is), and that is unlikely to change:

https://www.opengroup.org/openbrand/register/

wanted to avoid any controversy of "Unix vs not-Unix" as macOS is Unix, but GNU is not Unix...

Understood; an easy - if verbose - way out of this is $IsUnixLike.

[MaximoTrinidad]

Interesting discussion guys!!

I just recently installed macOS in a VmWare VM and I never fully work on a Mac. Then, my first thoughts was that I was working in Linux with a GUI. Now I've learned is Unix. That's so cool!

So, when is the next PowerShell Core Community Call?
:)

[SteveL-MSFT]

@mklement0 It seems that neither $IsPosix nor $IsUnix fits perfectly. Between those two, I think $IsUnix is more understandable to the majority of users.

@MaximoTrinidad next call is next month May 18th!

[BrucePay]

@mklement0 I think you're getting things a bit mixed up. Open Group certifies operating systems as UNIX not POSIX. POSIX is a family of standards created by the IEEE based on the various Unix variants. Linux is considered "mostly" POSIX-compliant, and with the LSB, will likely be completely so. More significantly, UNIX is a brand for specific operating systems whereas POSIX is a ABI specification. Saying $IsUNIX is claiming that the OS is UNIX which is likely false and in violation of the Open Group trademark. $IsPosix simply says the the platform supports some portion of the POSIX ABI.

Factually, only macOS is a POSIX-certified platform.

That is factually incorrect. Windows, through the POSIX subsystem, is certified as POSIX-compliant, as are a large number of operating systems. (A lot of vendors did this work in the late 90's to comply with FIPS 151-1). What Wndows is not is UNIX.

[mklement0]

Saying $IsUNIX is claiming that the OS is UNIX which is likely false and in violation of the Open Group trademark

Note that the suggestion for the variable name was $IsUnix (note the casing).

• The name of the trademark, as stated earlier, is "UNIX®" - all-uppercase.

• By contrast, "Unix" is not trademarked, and these days is widely understood to be a shorthand for "Unix-like", meaning a loose term to refer to platforms with either an AT&T Unix _heritage_ (in terms of source code) or _behavior_ similar to the original AT&T Unix.

If there is concern about the trademark issue, $IsUnixLike offers a way out, but it feels clumsy - and also frowned upon by the owners of the trademark: "They do not approve of the construction "Unix-like", and consider it a misuse of their trademark." (see previous link)

Linux is considered "mostly" POSIX-compliant, and with the LSB, will likely be completely so.

• Yes, as stated earlier, major Linux _distros_ are _de facto_ mostly POSIX-compliant, but they are not certified.

  • Note that Linux is just a _kernel_, so it would have to be _specific distros_ that receive certification _individually_.

• The LSB is not a part of POSIX, so compliance with it is unrelated to POSIX compliance; as previously quoted in the linked comment (emphasis added, from https://en.wikipedia.org/wiki/Single_UNIX_Specification#Non-registered_Unix-like_systems):

It is based on the POSIX specifications, the Single UNIX Specification, and other open standards, and also extends them in several areas; but there are some conflicts between the LSB and The POSIX standards."

• The same source also states: "very few Linux systems can be configured to be completely conformant."

---

So I think that establishes that $IsPosix is factually incorrect.

$IsPosixLike could work, but by your own reasoning, wouldn't that mean it should return $True even on _Windows_? That would certainly make the variable pointless.

I'll address the POSIX/Unix terminology question in a separate comment.

[dragonwolf83]

The name doesn't have to be 100% accurate, just accurate enough that any Linux, Windows, or Mac user will be able to understand the variable's intent. No grouping is going to be that accurate either way.

IsUnix alongside IsLinux, IsMacOs, and IsWindows is confusing. 3 of them are specific OS Platforms. There is no grouping. Now you add another OS Platform and then decide to group it for Linux and macOS. What if I thought that IsUnix meant any BSD-derivative of Unix, the Unix Clones instead of Linux?

What about Cygwin and WSL? Is it factually correct to call either one of them Unix or Linux?

IsPosix, the intent is much more clear. This is for any OS or Subsystem that implements the POSIX Standard. It seems to be generally understood that while Linux and others are not POSIX-certified, that they are implementing enough of the standard to be compliant for most use cases. It is much more accurate of a term since it is referencing a Standard and not an OS Platform. It covers the Cygwin and WSL use cases, alongside macOS, BSD, and Linux.

[mklement0]

@dragonwolf83:

To answer the rhetorical question first:

What about Cygwin and WSL? Is it factually correct to call either one of them Unix or Linux?

Yes to "Unix" in the sense of "Unix-like" (we needn't worry about "Linux" in the context of this discussion, but as an aside: WSL self-identifies as Linux (uname)), though you'd want to make clear that they're emulation environments with limitations.

With respect to Cygwin it is a moot point, however, because you cannot install PowerShell Core in it (as far as I'm aware; you can start the _Windows_ PowerShell Core executable, but then you're simply running on _Windows_)

With respect to WSL: I don't know if it's officially supported, but running PowerShell Core on WSL seems to work fine, at least the Ubuntu flavor.

There is no grouping.

"Unix" _is_ a grouping. No one should expect it to refer to a specific OS these days.

---

I agree that we don't have to be accurate - in fact, we cannot be.

Because $IsPosix references an actual standard - trademarked as POSIX® - my concern was that it suggests an unwarranted accuracy (unlike $IsUnix).

But, perhaps, as with the "Unix" / "UNIX®" distinction, there's also informal use of "Posix".

Strictly speaking, POSIX® is a _family_ of standards, and _selective_ certification is possible.
There is no _the_ POSIX® standard, but there are parts that matter in the context of PowerShell.
(I'll cover that in a separate comment.)

At the end of the day, if everyone's happy with $IsPosix and understands it to mean quasi-POSIX® rather than certified compliance, that's fine.

Without the compliance confusion, I agree that it's helpful to reference POSIX as a de-facto near-common denominator (even though that is quite fuzzy in itself, without naming parts or versions).

[BrucePay]

From my reading of this discussion (grumbles aside), it sounds like we have consensus on $IsPosix. So we can move forward on it now - right?

[mklement0]

@BrucePay:

Sounds good to me.

From the department of more-than-you-ever-wanted-to-know:

• POSIX® is a _family_ of certifications, jointly awarded by the Open Group and IEEE (the certification authority is the former, the license authority the latter).

  • Currently, certification can be obtained for 4 distinct standards, subdivided into 2 real-time-OS standards distinguished by system size (1003.13 PSE52 and 1003.13 PSE52), and a non-real-time standard in 2 publication-year editions (1003.1 2016 and 1003.1 2003)

  • _All_ POSIX standards mandate portability at the _source-code level_.

  • Only the 1003.1 (non-real-time) standards include standardization of a "command interpreter (or "shell"), and common utility programs" (volume _Shell and Utilities_).

  • The current list of certified products has surprisingly few entries - presumably, manufacturers go for the UNIX® certification instead.

  • Certification can be obtained without signing the trademark-licensing agreement, but only signing the latter entitles to use of the POSIX® trademark.

• The UNIX® trademark, awarded by the Open Group, is awarded based on compliance with the SUS (Single Unix Specification).

  • The non-real-time POSIX standards (1003.1) form the _core_ of the SUS, currently as part of the POSIX.1-2017 standard. POSIX.1-2017 aka IEEE Std 1003.1™-2017 aka The Open Group Technical Standard Base Specifications, Issue 7 is composed of 4 volumes, the following 2 of which are the non-real-time POSIX standards (1003.1) required for POSIX® certification, albeit in a _more recent_ edition:
  • XSH: System Interfaces: IEEE Std 1003.1-2017 (Revision of IEEE Std 1003.1-2008) (APIs, system functions)
  • XCU: Shell & Utilities: IEEE Std 1003.1-2017 (Revision of IEEE Std 1003.1-2008)

  • The other 2 volumes are:

    • XBD: Base Definitions (sections 2.1.1 and 2.1.2 of which the non-real-time POSIX® certifications also require compliance with)
      
      
      
      • XRAT: Rationale
      
      

  • Currently, 3 UNIX® certifications exist, depending on the (historical) version of the SUS: UNIX 03, UNIX 98, UNIX 95 - see https://www.opengroup.org/openbrand/register/, which also lists all currently certified operating systems.

The perhaps surprising thing is that while UNIX® certification _encompasses_ compliance with a specific POSIX _standard_, it is distinct from POSIX® _certification_ (with the latter seemingly of little real-world importance, given how few products are currently certified).

---

Open Group certifies operating systems as UNIX not POSIX

As stated above, the Open Group does _both_ (in the case of POSIX jointly with IEEE).

POSIX is a ABI specification.

The current POSIX standards _include_ - but aren't limited to - an API specification, not an ABI specification - compatibility is only required at the _source-code_ level, not at the _binary_ level.

Windows, through the POSIX subsystem, is certified as POSIX-compliant, as are a large number of operating systems.

No, Windows is _not_ currently POSIX-certified (and it's safe to assume won't ever be again).
(Unix-like platforms aren't technically POSIX-certified: some went for UNIX-certification, which, however _implies_ POSIX compliance, as detailed above.)

A brief history of POSIX on Windows:

• Windows NT - Windows 2000: [_defunct_] [Microsoft POSIX subsystem](https://en.wikipedia.org/wiki/Microsoft_POSIX_subsystem): "Windows NT versions 3.5, 3.51 and 4.0 were certified as compliant with FIPS 151-2.", a specific, FIPS-related POSIX standard that has since been withdrawn.

• Windows XP / Server 2003 - Windows 8/Windows Server 2012: "The POSIX subsystem was replaced in Windows XP / Windows Server 2003 by [_defunct_] ["Windows Services for UNIX", (SFU)](https://en.wikipedia.org/wiki/Windows_Services_for_UNIX) [...] SFU was removed from later versions of Windows 8 and Windows Server 2012" (ibid.)

• Windows 10 / Server 2016 - now: "SFU is logically (though not formally) replaced by the Windows Subsystem for Linux (WSL) in the Windows 10 Anniversary Update and Windows Server 2016 Version 1709 respectively." (ibid.)

  • Note that Microsoft conceives of WSL as "primarily a tool for developers – especially web developers and those who work on or with open source projects"

[SteveL-MSFT]

cc @JamesWTruher

[bergmeister]

After having read the thread again it seems there is still no conclusion. @SteveL-MSFT Should we therefore take the Commitee-Reviewed tag off?