Powershell: [System.Runtime.InteropServices.Marshal]::PtrToStringAuto failing to extract complete Binary String on macOS

[phatmandrake]

Steps to reproduce

$String = $("Hello World!" | ConvertTo-SecureString -AsPlainText -Force)
[System.Runtime.InteropServices.Marshal]::PtrToStringAuto(
[System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($string)
)

Expected behavior

Decrypts and returns the value of the Secure String. "Hellow world!"

Actual behavior

Decrypts only the first char "H" and then stops.

Environment data

Name | Value
-- | --
PSVersion | 7.0.0-rc.3
PSEdition | Core
GitCommitId | 7.0.0-rc.3
OS | Darwin 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64
Platform | Unix
PSCompatibleVersions | {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion | 2.3
SerializationVersion | 1.1.0.1
WSManStackVersion | 3.0

Documentation

https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.marshal.ptrtostringauto?view=netframework-4.8

According to the documentation, when specifying an integer, PtrToStringAuto:

Allocates a managed String and copies the specified number of characters from a string stored in unmanaged memory into it.

Specifying an int of 11 Returns "Hello", this is because every other char returned is Null. In this case, you must specify an int of 23 to return the complete string "Hello World!" using this method.

$String = $("Hello World!" | ConvertTo-SecureString -AsPlainText -Force)
[System.Runtime.InteropServices.Marshal]::PtrToStringAuto( [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($string), 23)

$String[0] Returns H
$String[1] Returns NULL
$String[2] Returns E
$String[3] Returns NULL
etc.... 

If no integer is specified, PtrToStringAuto:

Allocates a managed String and copies all characters up to the first null character from a string stored in unmanaged memory into it.

I believe this suggests that either the Secure String is being stored with NULL values, whereas in PS6 it was not, or that the behavior of the PtrToStringAuto function has changed, and now adheres to the behavior the documentation describes above.

This is only an issue on macOS; however, using PtrToStringBSTR in place of PtrToStringAuto to decrypt the Secure String works as expected across all versions, windows and macOS.

[mklement0]

@phatmandrake, based on our conversation at https://stackoverflow.com/a/60406968/45375 I think you can close this issue now.

To summarize:

The Auto in PtrToStringAuto() means that the _unmanaged_ input string is assumed to use a platform-appropriate character encoding, whereas BSTR is
a "Unicode" (UTF-16) string on _all_ platforms. On Windows, an unmanaged string is assumed to have UTF-16 encoding (which is why the code works), whereas on Unix-like platforms it is UTF-8 since .NET Core 3.0 (PowerShell [Core] 7.0 is based on .NET Core 3.1), which explains your symptoms: the NUL chars. in the BSTR instance's UTF-16 code units are interpreted as characters in their own right when (mis)interpreted as UTF-8. Note that .NET Core 2.x (which is what PowerShell [Core] 6.x is based on) (inappropriately) defaulted to UTF-16, which this PR fixed, amounting to a breaking change.

While using PtrToStringBSTR() instead of PtrToStringAuto() is therefore the immediate fix, it's worth nothing that [securestring] is generally not recommended for new code anymore and that it is especially problematic on Unix-like platforms, because _no encryption_ is used there.

More specifically, creating a [string] instance with a plain-text representation from a secure string defeats the purpose of using [securestring] to begin with.

Stack Overflow

Are you able to use PtrToStringAuto to decrypt a secure string in Powershell 7 on macOS?

I have had no success in getting the following code snippet to output "Hello World!" in PS7 $("Hello World!" | ConvertTo-SecureString -AsPlainText -Force) [System.Runtime.InteropServices.Marshal]::

[phatmandrake]

The loop has been made.