Vscode-powershell: Integrated Console - Failing to Execute GAM

[brentbain]

System Details

• Operating system name and version: Windows 10
• VS Code version: 1.22.2
• PowerShell extension version: 1.6.0
• Output from $PSVersionTable:
  ---- -----
  PSVersion 5.1.14393.2189
  PSEdition Desktop
  PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
  BuildVersion 10.0.14393.2189
  CLRVersion 4.0.30319.42000
  WSManStackVersion 3.0
  PSRemotingProtocolVersion 2.3
  SerializationVersion 1.1.0.1
  

  Issue Description

I am experiencing a problem with the open source project for Google API management. This same tool and powershell code work in ISE as well as when running from the non-integrated powershell console. The tool can be downloaded/installed from here:
https://github.com/jay0lee/GAM

It creates a dos executable which makes Google API calls - allowing powershell to interact with the Google API.

I get error messages returned which ultimately are tied back to permissions on a oauth token file which is stored in the folder where the GAM exe resides. If I run the same command from Powershell ISE - or run it from the non-integrated powershell console, everything works as expected. While I'd normally say this is an issue for GAM, since it works in both ISE and in the non-integrated console that starts up with VSCode, I'm hoping it might be something that can be fixed in VSCode Powershell.

Attached Logs

Follow the instructions in the README
about capturing and sending logs.
1523912748-a1e77963-f824-4a1c-8569-29acac8246f81523912745826.zip

[rjmholt]

Hi @brentbain, thanks for reporting this!

I know you've said the error messages are due to permissions errors, but your screenshot indicates an encoding problem. Do you know how the permissions problem is precipitating that error?

[rjmholt]

Also, for us to reproduce this (ideally without needing to download and run gam.exe), do you know what the permissions are on the OAuth file and how they differ from those on gam.exe?

[brentbain]

@rjmholt Thanks for following up...
I guess I do not know 100% that the issue is permissions errors - reading the GAM forum (also in github), the primary maintainer has answered a couple questions related to errors with the actual line (12881) as the first traceback - his answer was always to check file permissions on the oauth2.txt file - or to delete it and try recreating it from the console window that gam is failing in (gam oauth create).

I attempted this multiple times to no avail (both hardcoding permissions on the file and the directory) - as well as delete the oauth2.txt file and then trying to have it recreate the file, which also throws similar errors on the integrated console.

Currently, I have set the folder where gam.exe and oauth2.txt live to "Authenticated users - Full Permissions" as well as "Users - Full" - originally, it was just my account with full permissions. The permissions on the folder, according to documentation, need to be that the process executing gam needs to be able to read/create/modify the file, oauth2.txt.

Looking deeper into other errors, while none on the GAM forums, there are some other people who have noted python execution errors related to the encoding. All though the same question/problem still comes up - why does the "non-integrated" powershell console in VSCode execute it just fine, and the Integrated Console throw the error?

Possibly a duplicate of:
https://github.com/Microsoft/vscode/issues/30350

[rjmholt]

Sorry for not being more helpful. This is a rather mystifying bug though.

100% agree that the failing in the Integrated Console and not failing in the VSCode console is the categorising feature of where the fault lies.

The Integrated Console is a client/server process that communicates by JSON, so was thinking that there might be a process permissions thing going on, if permissions are the part that's not working.

But it may be an issue with the console's encoding mechanism instead. I noticed this StackOverflow issue mentions that it has to do with Python not using UTF-8 in Windows console environments.

Also seems to be occuring in https://github.com/conda/conda/issues/5448, https://github.com/Microsoft/WSL/issues/2009 and https://github.com/docker/compose/issues/2775. Also see this other StackOverflow issue.

So maybe try setting the PYTHONIOENCODING environment variable like in those posts above? Hopefully that works, but I can imagine the console's environment being a bit finnicky and hard to configure for things like this.

EDITED: Got it wrong and thought you'd said the non-integrated console isn't working...

We implement our own host for PowerShell to run in on the server side, which communicates input and output over JSON messages. So that might be where the encoding issue lies. Does using Python3 resolve this (not suggesting you move to it, just helpful to diagnose)?

[WToorenburghIntiveo]

I've encountered this as well. The encoding error arising from the client-host JSON structure of the plugin makes sense, insofar as my inexperience in these things can allow.

I don't know if it's possible to use Python3 with GAM, as for Windows, it comes in a packed exe. As far as I can tell, I think we might need to ask jay0lee to compile the tool using Python3.

[phatmandrake]

Has anyone found a viable workaround for this issue?

[TylerLeonhardt]

This seems like a bug, for sure. I'm wondering what you get when you set the PYTHONIOENCODING:

[Environment]::SetEnvironmentVariable("PYTHONIOENCODING", "UTF-8", "Process") 

I think there is a bug on PowerShell Editor Services to properly deal with encoding - as we've been seeing stuff like:
https://github.com/PowerShell/vscode-powershell/issues/1306 , https://github.com/PowerShell/vscode-powershell/issues/1319 ,
https://github.com/PowerShell/vscode-powershell/issues/1074

[WToorenburghIntiveo]

I finally got an opportunity to test that, and got the same output:

Traceback (most recent call last):
  File "gam.py", line 12881, in <module>
  File "gam.py", line 12367, in ProcessGAMCommand
  File "gam.py", line 10297, in doPrintUsers
LookupError: unknown encoding: cp65001
[11040] Failed to execute script gam

[mklement0]

@tylerl0706: From what I understand, setting PYTHONIOENCODING only helps with _stand-alone_ Python versions, not versions _embedded_ in executables.

The workaround discussed in https://github.com/PowerShell/PowerShell/issues/3819#issuecomment-306064583 _may_ help.

[TylerLeonhardt]

Can someone with this issue give this workaround a try that @mklement0 supplied:

$OutputEncoding = [console]::InputEncoding = [console]::OutputEncoding = New-Object System.Text.UTF8Encoding

[mklement0]

@tylerl0706: In this case I was thinking it's the gam ... | % ToString workaround that might help.

[WToorenburghIntiveo]

Aha! The workaround @mklement0 mentioned seems to work. Output appears as expected from the GAM tool inside the Powershell Integrated Console. Nice find!

---

Speaking of nice finds, because this thread is about using G Suite tools in Powershell, I feel like I must mention this module I found: https://github.com/scrthq/PSGSuite

It's _way_ better than the GAM tool, as it returns actual Powershell objects, so no weirdo CSV parsing or Python encoding to wrangle with. Sorry if this is off topic!

[scrthq]

@WToorenburghEasyMarkit cheers! 😄

[SydneyhSmith]

Closing this issue as resolved