Powershell: Can not decrypt correctly in PowerShell 7.0.3 in CloudShell and Linux

[dolauli]

Steps to reproduce

PS /home/xiaogang> [int]$timeInt = $(Get-Date -UFormat '%s')
PS /home/xiaogang> $passwd = "bl0ckCh@!n$timeInt)" | ConvertTo-SecureString -AsPlainText -Force
PS /home/xiaogang> $psTxt = [System.Runtime.InteropServices.marshal]::PtrToStringAuto([System.Runtime.InteropServices.marshal]::SecureStringToBSTR($passwd))
PS /home/xiaogang> $psTxt
b

Expected behavior

PS C:\Users\xidi> [int]$timeInt = $(Get-Date -UFormat '%s')
PS C:\Users\xidi> $passwd = 'bl0ckCh@!n$timeInt)' | ConvertTo-SecureString -AsPlainText -Force
PS C:\Users\xidi> $psTxt = [System.Runtime.InteropServices.marshal]::PtrToStringAuto([System.Runtime.InteropServices.marshal]::SecureStringToBSTR($passwd))
PS C:\Users\xidi> $psTxt
bl0ckCh@!n$timeInt)

Actual behavior

b is return instead of bl0ckCh@!n$timeInt)

Environment data

Issue only happens in PowerShell 7.0.3 in CloudShell and Ubuntu 18.04, But in PowerShell 7.0.3, 7.0.0, 6.2.4 and Windows PowerShell in my local Windows machine, it is OK.

[jborean93]

When you create a secure string on non-Windows hosts the actual secure string is simply the UTF-16-LE encoding of the text you've specified. In this example we will be using the password café𝄞 to test out some of the encoding edge cases.

$pass = "café$([Char]::ConvertFromUTF32(0x0001D11E))" | ConvertTo-SecureString -AsPlainText -Force
$pass | ConvertFrom-SecureString

# 630061006600e90034d81edd

$bstrPtr = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($pass)
$bstrBytes = [byte[]]::new(14)
[System.Runtime.InteropServices.Marshal]::Copy($bstrPtr, $bstrBytes, 0, $bstrBytes.Length)
$bstrBytes | Format-Hex

#    Label: Byte (System.Byte) <095C5B21>
#
#           Offset Bytes                                           Ascii
#                  00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
#           ------ ----------------------------------------------- -----
# 0000000000000000 63 00 61 00 66 00 E9 00 34 D8 1E DD 00 00       c a f é 4Ø�Ý  

$uniPtr = [System.Runtime.InteropServices.Marshal]::SecureStringToGlobalAllocUnicode($pass)
$uniBytes = [byte[]]::new(14)
[System.Runtime.InteropServices.Marshal]::Copy($uniPtr, $uniBytes, 0, $uniBytes.Length)
$uniBytes | Format-Hex

#    Label: Byte (System.Byte) <3576ADA4>
#
#           Offset Bytes                                           Ascii
#                  00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
#           ------ ----------------------------------------------- -----
# 0000000000000000 63 00 61 00 66 00 E9 00 34 D8 1E DD 00 00       c a f é 4Ø�Ý 

# This will differ as the encoding used is based on [Text.Encoding]::Default
# This example was on Linux where the default encoding is UTF-8
$ansiPtr = [System.Runtime.InteropServices.Marshal]::SecureStringToGlobalAllocAnsi($pass)
$ansiBytes = [byte[]]::new(10)
[System.Runtime.InteropServices.Marshal]::Copy($ansiPtr, $ansiBytes, 0, $ansiBytes.Length)
$ansiBytes | Format-Hex

#    Label: Byte (System.Byte) <75B70A19>
#
#           Offset Bytes                                           Ascii
#                  00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
#           ------ ----------------------------------------------- -----
# 0000000000000000 63 61 66 C3 A9 F0 9D 84 9E 00                   café�

So we can see that the raw bytes when you use SecureStringToBSTR or SecureStringToGlobalAllocUnicode are the UTF-16-LE encoded bytes of your string. I personally don't know what the difference is between the 2 as the end result seems to be the same. The SecureStringToGlobalAllocAnsi method is dependent on the default encoding set for the system. You can get this value by running [Text.Encoding]::Default but this will definitely differ across Windows and non-Windows hosts. The key takeaway from this point is that the raw bytes in unmanaged memory is going to be your password encoded by UTF-16-LE.

There are 5 main functions that you can use to convert a ptr of unmanaged memory to a string

• PtrToStringAnsi
• PtrToStringAuto
• PtrToStringBSTR
• PtrToStringUni
• PtrToStringUTF8 - Added in .NET Core

When you call each of these functions without specifying the length it reads each byte until it comes across the first NULL char, i.e. \u0000 encoded to bytes. For example UTF-8 and other extended ASCII encodings would be 00, whereas Unicode/UTF-16 will be 00 00.

On Windows, PtrToStringAuto calls PtrToStringUni so it's reading all the bytes until it gets to 00 00. This is why you can use your function and get the proper string back.

On other platforms, PtrToStringAuto calls PtrToStringUTF8 so it's reading all the bytes until it gets to 00. It's only returning the first char because the 2nd byte is 00 when the function thinks it's the end of the string.

PowerShell can't really do anything about the change in behaviour on the different platforms. Ultimately what this means is that you should make sure you call SecureStringToGlobalAllocUnicode to get your pointer then PtrToStringUni to get the string from that pointer. This way you ensure you are always dealing with UTF-16-LE and don't need to deal with platform differences.

Another option is to avoid dealing with pointers altogether and use this method instead

$pass = 'abc' | ConvertTo-SecureString -AsPlainText -Force

[PSCredential]::new('dummy', $pass).GetNetworkCredential().Password

No need to worry about clearing any unmanaged memory and personally I think it's simpler.

[jborean93]

Thanks to @SeeminglyScience I now know what the difference between SecureStringToBSTR and SecureStringToGlobalAllocUnicode is. The former encodes the length of the string in 4 bytes just before the pointer. This is important if your string contains a null character as any of the other methods would see that as the end of the string.

$pass = "pass`0word" | ConvertTo-SecureString -AsPlainText -Force
$pass | ConvertFrom-SecureString
# 7000610073007300000077006f0072006400

$bstrPtr = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($pass)
$bstrBytes = [byte[]]::new(24)
[System.Runtime.InteropServices.Marshal]::Copy([IntPtr]::Add($bstrPtr, -4), $bstrBytes, 0, $bstrBytes.Length)

$bstrBytes | Format-Hex

#    Label: Byte (System.Byte) <630BBC81>
#
#           Offset Bytes                                           Ascii
#                  00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
#           ------ ----------------------------------------------- -----
# 0000000000000000 12 00 00 00 70 00 61 00 73 00 73 00 00 00 77 00 �   p a s s   w 
# 0000000000000010 6F 00 72 00 64 00 00 00                         o r d   

$plaintext = [System.Runtime.InteropServices.Marshal]::PtrToStringBSTR($bstrPtr)
$plaintext.Length
# 9

$plaintext
# password

$uniPtr = [System.Runtime.InteropServices.Marshal]::SecureStringToGlobalAllocUnicode($pass)
$uniBytes = [byte[]]::new(24)
[System.Runtime.InteropServices.Marshal]::Copy([IntPtr]::Add($uniPtr, -4), $uniBytes, 0, $uniBytes.Length)

$uniBytes | Format-Hex

#    Label: Byte (System.Byte) <4552DC2C>
#
#           Offset Bytes                                           Ascii
#                  00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
#           ------ ----------------------------------------------- -----
# 0000000000000000 00 00 00 00 70 00 61 00 73 00 73 00 00 00 77 00     p a s s   w 
# 0000000000000010 6F 00 72 00 64 00 00 00                         o r d   

$plaintext = [System.Runtime.InteropServices.Marshal]::PtrToStringUni($uniPtr)
$plaintext.Length
# 4

$plaintext
# pass

It also turns out that the GetNetworkCredential() method also fails when dealing with null bytes in the string so I would take back my recommendation, even for such a rare edge case. Ultimately you should be using SecureStringToBSTR and PtrToStringBSTR like so

$pass = "pass`0word" | ConvertTo-SecureString -AsPlainText -Force

$ssPtr = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($pass)
try {
    $plaintext = [System.Runtime.InteropServices.Marshal]::PtrToStringBSTR($ssPtr)
} finally {
    [System.Runtime.InteropServices.Marshal]::ZeroFreeBSTR($ssPtr)
}

[vexx32]

Yeah, there've been a few issues around this; fundamentally though, this apparent change was in the underlying API on .NET's end, so there's effectively nothing PowerShell could do. When working with the BSTR methods, the best bet is to use them all the way through, using PtrToStringBSTR rather than PtrToStringAuto.

That won't fix the _many_ blog posts recommending the latter, but it is a solution that works both in Windows PowerShell and pwsh; using PtrToStringAuto only works on Windows PowerShell (and even then may occasionally cause some issues, from what I've seen).

[mklement0]

Good info here, but this has indeed come up several times.
Here's the previous list of issues: https://github.com/PowerShell/PowerShell/issues?q=is%3Aissue+ptrtostringauto+is%3Aclosed+NOT+%2Fhome%2Fchythu%2Ftemp

GitHub

PowerShell/PowerShell

PowerShell for every system! Contribute to PowerShell/PowerShell development by creating an account on GitHub.

[msftbot[bot]]

This issue has been marked as answered and has not had any activity for 1 day. It has been closed for housekeeping purposes.