Powershell: AWSPowershell.NetCore is not loaded in linux powershell automatically

[sg21210]

Hello, I'm facing an issue while loading the AWS SDK Libraries automatically on my Powershell, installed on Linux. At first I thought this is a problem in AWS Tool for Powershell, but I realized that it resides on the Powershell loading modules and not AWS SDK library.

I noticed this issue while making a linux docker image using powershell, but same issue were reported by AWS engineer using OS X also.

Steps to reproduce

1) Install powershell on Linux
2) Install AWS Tools for Powershell
3) Launch powershell and try some commands like Copy-S3Object, Get-IAMRole (not Get-IAMRoles)

Step 2) can be

pwsh -command "& {&'Install-Module' -Name AWSPowerShell.NetCore -AllowClobber -Force}"

or

pwsh -command "& {&'Install-Module' -Scope CurrentUser -Name AWSPowerShell.NetCore -Force}"

or any command that installs the module, even on running powershell.

Expected behavior

The aws module should be loaded and handle AWS commands without explicit import

PS /> Get-S3Object

cmdlet Get-S3Object at command pipeline position 1
Supply values for the following parameters:
BucketName:

Actual behavior

PS /> Get-S3Object
Get-S3Object : The term 'Get-S3Object' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ Get-S3Object
+ ~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (Get-S3Object:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException

Environment data

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      6.0.4
PSEdition                      Core
GitCommitId                    v6.0.4
OS                             Linux 4.9.93-linuxkit-aufs #1 SMP Wed Jun 6 16:55:56 UTC 2018
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Note

What makes me confused more is,

• When I type Get-IAM then press Tab, it shows

PS /> Get-IAM
Get-IAMAccountAuthorizationDetails  Get-IAMAttachedRolePolicies         Get-IAMGroupPolicies                Get-IAMInstanceProfiles             Get-IAMPolicies                     Get-IAMRolePolicies                 Get-IAMSAMLProviders                Get-IAMUserPolicies
Get-IAMAttachedGroupPolicies        Get-IAMAttachedUserPolicies         Get-IAMGroups                       Get-IAMOpenIDConnectProviders       Get-IAMPolicyVersions               Get-IAMRoles                        Get-IAMServerCertificates           Get-IAMUsers

• At this point, Get-IAMRole does not work. But Get-IAMRoles works.
• After typing Get-IAMRoles, it loads aws module, so autocomplete works fine. So now Get-IAMRole works

PS /> Get-IAMRole
Get-IAMRole : The term 'Get-IAMRole' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ Get-IAMRole
+ ~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (Get-IAMRole:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException

PS /> Get-IAMRoles
Get-IAMRoles : No credentials specified or obtained from persisted/shell defaults.
At line:1 char:1
+ Get-IAMRoles
+ ~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (Amazon.PowerShe...MRoleListCmdlet:GetIAMRoleListCmdlet) [Get-IAMRoleList], InvalidOperationException
+ FullyQualifiedErrorId : InvalidOperationException,Amazon.PowerShell.Cmdlets.IAM.GetIAMRoleListCmdlet

PS /> Get-IAMRole
Get-IAMRole : No credentials specified or obtained from persisted/shell defaults.
At line:1 char:1
+ Get-IAMRole
+ ~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (Amazon.PowerShe...etIAMRoleCmdlet:GetIAMRoleCmdlet) [Get-IAMRole], InvalidOperationException
+ FullyQualifiedErrorId : InvalidOperationException,Amazon.PowerShell.Cmdlets.IAM.GetIAMRoleCmdlet

• Now, repeat typing Get-IAM then press Tab,

PS /> Get-IAM
Get-IAMAccessKey                        Get-IAMAttachedRolePolicies             Get-IAMGroupForUser                     Get-IAMInstanceProfiles                 Get-IAMPolicyVersion                    Get-IAMSAMLProvider                     Get-IAMSSHPublicKey
Get-IAMAccessKeyLastUsed                Get-IAMAttachedRolePolicyList           Get-IAMGroupList                        Get-IAMLoginProfile                     Get-IAMPolicyVersionList                Get-IAMSAMLProviderList                 Get-IAMSSHPublicKeyList
Get-IAMAccountAlias                     Get-IAMAttachedUserPolicies             Get-IAMGroupPolicies                    Get-IAMMFADevice                        Get-IAMPolicyVersions                   Get-IAMSAMLProviders                    Get-IAMUser
Get-IAMAccountAuthorizationDetail       Get-IAMAttachedUserPolicyList           Get-IAMGroupPolicy                      Get-IAMOpenIDConnectProvider            Get-IAMRole                             Get-IAMServerCertificate                Get-IAMUserList
Get-IAMAccountAuthorizationDetails      Get-IAMContextKeysForCustomPolicy       Get-IAMGroupPolicyList                  Get-IAMOpenIDConnectProviderList        Get-IAMRoleList                         Get-IAMServerCertificateList            Get-IAMUserPolicies
Get-IAMAccountPasswordPolicy            Get-IAMContextKeysForPrincipalPolicy    Get-IAMGroups                           Get-IAMOpenIDConnectProviders           Get-IAMRolePolicies                     Get-IAMServerCertificates               Get-IAMUserPolicy
Get-IAMAccountSummary                   Get-IAMCredentialReport                 Get-IAMInstanceProfile                  Get-IAMPolicies                         Get-IAMRolePolicy                       Get-IAMServiceLinkedRoleDeletionStatus  Get-IAMUserPolicyList
Get-IAMAttachedGroupPolicies            Get-IAMEntitiesForPolicy                Get-IAMInstanceProfileForRole           Get-IAMPolicy                           Get-IAMRolePolicyList                   Get-IAMServiceSpecificCredentialList    Get-IAMUsers
Get-IAMAttachedGroupPolicyList          Get-IAMGroup                            Get-IAMInstanceProfileList              Get-IAMPolicyList                       Get-IAMRoles                            Get-IAMSigningCertificate               Get-IAMVirtualMFADevice

It seems having some problem on automatically loading some modules.

[MaximoTrinidad]

I normally install module from within the pwsh console:
Install-Module -Scope CurrentUser -Name AWSPowerShell.NetCore -Force;

Then, in Linux, I realized that you do 2 tabs (not once), in order to display the list of commands:
Get-IAM (tab)+(tab)

On the issue with Get-IAMRoles and Get-IAMRole:
Get-IAMRole is a Cmdlet.
Get-IAMRoles is an Alias for Get-IAMRoleList.

I tested this with WSL/Ubuntu, Ubuntu 18/04, and CentOS 7.

It did work for me until I exited and went back to pwsh.

I think this need to be reported to AWS PowerShell forum as I don't see any of the same behavior with other installed modules.

:)

[MaximoTrinidad]

Yes! That works but there's no need to do the import module. It's only happening with that particular command. Remember that Get-IAMRole is an Alias and not a Cmdlet.

I'm not seen this behavior on any other installed modules.

Something is not in Synch when auto-loading the AWS module. So. it's possible is in the AWS module and not PowerShell Core.
I think this should be also address at the AWS PowerShell forum.

[BrucePay]

Try running

Get-Module -ListAvailable -Refresh

This will refresh the module analysis cache which is what the autoloader uses to figure out what modules to load for a given command.

[rjmholt]

@sg21210 does importing the module in the console by itself raise any errors?

[sg21210]

I understand all your message. I've been using linux machine having powershell and AWS tool also.
Problem occurred while making a new docker image based on linux.

Here are some notes.

• After explicitly importing module by Import-Module, it works!
• But, after relaunching powershell, the shell does not import it again. Everytime I launch it, I should import it to use AWS tool.
• When I type Get-Module -ListAvailable (even without -Refresh tag),

PS /> Get-Module -ListAvailable


    Directory: /root/.local/share/powershell/Modules


ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Binary     3.3.343.0  AWSPowerShell.NetCore               {Add-ASInstances, Add-CTTag, Add-DPTags, Add-DSIpRoutes...}


    Directory: /opt/microsoft/powershell/6.0.4/Modules


ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Manifest   1.1.0.0    Microsoft.PowerShell.Archive        {Compress-Archive, Expand-Archive}
Manifest   3.0.0.0    Microsoft.PowerShell.Host           {Start-Transcript, Stop-Transcript}
Manifest   3.1.0.0    Microsoft.PowerShell.Management     {Add-Content, Clear-Content, Clear-ItemProperty, Join-Path...}
Manifest   3.0.0.0    Microsoft.PowerShell.Security       {Get-Credential, Get-ExecutionPolicy, Set-ExecutionPolicy, ConvertFrom-SecureString...}
Manifest   3.1.0.0    Microsoft.PowerShell.Utility        {Format-List, Format-Custom, Format-Table, Format-Wide...}
Script     1.1.7.0    PackageManagement                   {Find-Package, Get-Package, Get-PackageProvider, Get-PackageSource...}
Script     1.6.0      PowerShellGet                       {Install-Module, Find-Module, Save-Module, Update-Module...}
Script     0.0        PSDesiredStateConfiguration         {CheckResourceFound, Set-NodeResourceSource, Test-NodeResourceSource, Update-ModuleVersion...}
Script     1.2        PSReadLine                          {Get-PSReadlineKeyHandler, Set-PSReadlineKeyHandler, Remove-PSReadlineKeyHandler, Get-PSReadlineOption...}


PS /> Copy-S3Object
Copy-S3Object : The term 'Copy-S3Object' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ Copy-S3Object
+ ~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (Copy-S3Object:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException

• Module is correct, because when I type Get-Module -ListAvailable | Import-Module, it also works.

PS /> Get-Module -ListAvailable | Import-Module
WARNING: The names of some imported commands from the module 'PSDesiredStateConfiguration' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.
PS /> Copy-S3Object

cmdlet Copy-S3Object at command pipeline position 1
Supply values for the following parameters:
BucketName:

[sg21210]

Same problem is also occurred in OS X, and Ubuntu user. I'll attach the screenshot of Ubuntu case. I'm wondering if this could be a newer-version powershell/OS/SDK problem.

[SeeminglyScience]

In their module manifest they only explicitly list aliases. The CmdletsToExport field is a wildcard statement

# Cmdlets to export from this module
CmdletsToExport = '*-*'

They need to explicitly list exported cmdlets as well.

[steveataws]

As noted above, this is caused by us (AWS) currently not explicitly listing the cmdlets in the module in the manifest.

Until a couple months ago we did in fact enumerate the cmdlet names in the manifest but had to switch to using wildcards when we went through 4000 cmdlets in the module. This caused the gallery publishing to reject the module as it limited modules to under 4000 cmdlets in the export statement. We considered listing only 'the most important' cmdlets but couldn't come up with a single list - every service seemed to be important to someone :-). In the face of this, switching back to wildcards and noting the issue in the release notes at the time seemed the best approach.

Now, we realize 4000+ cmdlets is a lot - and the growth in service APIs since we shipped v1.0 has surprised us too. We have backlog plans to re-modularize to per-service modules but have not yet been able to schedule the work.

Interestingly, reverting to wildcards actually improved the import time for the module too - which now that we have PowerShell support in Lambda we're not too keen on making worse either. Realistically I think a fix will need to wait until we can get the re-modularization work scheduled. I'll bring this discussion to the wider attention of the team here at AWS and see what we can do to bump the priority.

[sg21210]

@steveataws Thank you. I didn't know the history so I couldn't understand what @SeeminglyScience mentioned above at first, but now I understand.

Hope to get fixed soon :)

[rjmholt]

Ah! Ok that all makes sense. Going to close as external resolution.

[LazerFX]

As a note for anyone accessing this in the future, you can run the following command and get access to the cmdlets (After installing the module):

Get-Command -Module AWSPowerShell.NetCore | Out-Null

The reason for the pipe into Out-Null is because (as noted above) there are more than 4000 exports. This allowed me to access Get-IAMAccessKey from within the core environment.

Hope this helps anyone Googling this in the future!

(Tagging @steveataws to bring this to his attention as a possible documentation improvement for the AWS team)

[scrthq]

Not to bring a closed issue back into discussion, but why would you force the module import with Get-Command instead of just running Import-Module AWSPowerShell.NetCore @LazerFX ? I see zero value in that from a high-level and you're ultimately costing more by doing that if you're not going to utilize the output of Get-Command. That's literally what Import-Module is for

[LazerFX]

@scrthq Performance. We have a small framework designed to run on a CI/CD platform... As we don't know which command is to be run, each has to ensure its own dependencies are captured... Calling get command is multiple times faster than get command, and enables the whole pipeline to run noticeably faster. If there's a better way, I'd love to know - not a PowerShell expert, simply a coder trying to keep up ;-)

[scrthq]

@LazerFX - the reason that Get-Command -Module $module works is because it's force importing the module underneath. You're effectively running Import-Module $module; Get-Command -Module $module, so if you don't need the output of Get-Command, then just simply run Import-Module.

In fact, running Import-Module is significantly faster than running Get-Command and piping to Out-Null due to no pipeline output that needs to subsequently be squashed:

"Get-Command time: {0}" -f (Measure-Command {
    pwsh -noprofile -C "Get-Command -Module AWSPowerShell.NetCore | Out-Null"
}).ToString()
"Import-Module time: {0}" -f (Measure-Command {
    pwsh -noprofile -C "Import-Mdule AWSPowerShell.NetCore"
}).ToString()

# Output:
Get-Command time: 00:00:06.7730225
Import-Module time: 00:00:00.8841475

[LazerFX]

That's weird, in Azure DevOps Pipelines, the time difference between import-module and get-command is significant, we trimmed nearly 1 minute off our runtime by using get-command. I don't profess to understand what's going on under the hood, and the results did seem odd, but we didn't have time to properly investigate (a speedup is a speedup). I'm a C# developer, and I'll freely admit PowerShell performance isn't something I've worked on a lot, so always happy to learn. We'll reinvestigate the performance once we've deployed the initial version, and see what we can do.

[scrthq]

Very strange! Speedup is a speedup indeed, if that works in your environment, so I get that.

I'd be curious how the following task script runs in Azure DevOps, will test shortly:

switch ($PSVersionTable.PSVersion.Major) {
    6 {
        $module = 'AWSPowerShell.NetCore'
        $engine = 'pwsh'
    }
    default {
        $module = 'AWSPowerShell'
        $engine = 'powershell'
    }
}
Install-Module $module -Repository PSGallery -Force
"Get-Command time: {0}" -f (Measure-Command {
    & $engine -noprofile -C "Get-Command -Module $module | Out-Null"
}).ToString()
"Import-Module time: {0}" -f (Measure-Command {
    & $engine -noprofile -C "Import-Module $module"
}).ToString()

[scrthq]

@LazerFX ¯\_(ツ)_/¯ - still getting better speed through Import-Module vs Get-Command in Azure Pipelines, both Windows PS and PS Core, for what it's worth:

[LazerFX]

Very interesting. Time for us to re-evaluate, I think. Thanks for the details - I'm on holiday this week, but we'll certainly look into it.

[ryanpagel]

I'm unable to get this to work even when running Get-Command -Module AWSPowerShell.NetCore | Out-Null. I'm running pwsh in a CodeBuild image in AWS. I'm getting this message: The term 'Get-S3Bucket' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again..

[scrthq]

@ryanpagel - probably best to bring that up on the AWS repo for that module, since it's not explicitly a PowerShell issue/bug - https://github.com/aws/aws-tools-for-powershell

GitHub

aws/aws-tools-for-powershell

The AWS Tools for PowerShell lets developers and administrators manage their AWS services from the PowerShell scripting environment. - aws/aws-tools-for-powershell