Aws-cli: How to describe instances from all regions?

[ApproximateIdentity]

I'm trying to switch my workflow over to the cli from the aws control panel, but I've run into a very odd issue. I know how to describe instances within a region, but how do I describe all of my instances irrespective of regions? Or at least get an output of all ec2 regions so I can do this programmatically one by one (i.e. something like aws ec2 list-regions).

It's very odd because the cli seems to use different names for regions than the control panel. For one, apparently all of my instances are being started in region "us-west-2" (at least so far), but I cannot find that string anywhere in the control panel. In fact, I can't seem to find anywhere listing my default region at all.

Is there a command inside of aws that does this?

[JordonPhillips]

The console doesn't allow you to list all instances irrespective of region, but it does hide the region identifiers with more descriptive text. us-west-2 is the datacenter located in Oregon, for instance. There's a drop-down in the top right of the console page that lets you select a region (pictured below).

For most services, there is not currently a programmatic way of discovering which regions are available, and so you will need to refer to this list. EC2 is one of the exceptions, however, as it provides the describe-regions function. Since the CLI is a wrapper over service APIs, we can't do multi-region searches unless the service supports it (S3, for instance, is mostly region-agnostic). You will have to provide the region with the --region argument to query in a different region.

Your default region is located in your configuration file (~/.aws/config on UNIX systems), and can be both shown and set with the aws configure command.

[nonbeing]

For people arriving here from Google, here's one way to list all your instances across all regions with the AWS CLI (based on the discussion above):

for region in `aws ec2 describe-regions --output text | cut -f3`
do
     echo -e "\nListing Instances in region:'$region'..."
     aws ec2 describe-instances --region $region
done

[Fitblip]

For those of you who also don't want to parse JSON with your eyeballs, here's an appended version that uses jq to give you a bit more-to-the-point summary.

for region in `aws ec2 describe-regions --output text | cut -f3`
do
     echo -e "\nListing Instances in region:'$region'..."
     aws ec2 describe-instances --region $region | jq '.Reservations[] | ( .Instances[] | {state: .State.Name, name: .KeyName, type: .InstanceType, key: .KeyName})'
done

Example output:

Listing Instances in region:'us-west-1'...
{
  "state": "stopped",
  "name": "scanner-1",
  "type": "t2.micro",
  "key": "scan1"
}
{
  "state": "stopped",
  "name": "cors",
  "type": "t2.micro",
  "key": "cors"
}
{
  "state": "stopped",
  "name": "scanner-2",
  "type": "t2.medium",
  "key": "scan2"
}

[JordonPhillips]

@nonbeing Nice! If you use that often, you should consider making it an alias.

[pcgeek86]

Another option would be to use the AWS PowerShell .NET Core module, which runs natively on Linux, Mac, and Windows.

foreach ($Region in (Get-AWSRegion)) {
  Get-EC2Instance -Region $Region.Region
}

Results

GroupNames    : {}
Groups        : {}
Instances     : {asdf}
OwnerId       : 123
RequesterId   : 123
ReservationId : r-asdf

GroupNames    : {}
Groups        : {}
Instances     : {asdf}
OwnerId       : 123
RequesterId   : 123
ReservationId : r-asdf

GroupNames    : {}
Groups        : {}
Instances     : {asdf}
OwnerId       : 123
RequesterId   : 123
ReservationId : r-asdf

Output using Format-Table

By default, in the current version of the AWS PowerShell module, the EC2 instances are returned in the PowerShell "list" format. However, this is easily changed to tabular formatting, using the Format-Table command.

$InstanceList = @()
foreach ($Region in (Get-AWSRegion)) {
  $InstanceList += Get-EC2Instance -Region $Region.Region -ProfileName CIDBTest
}
$InstanceList | Format-Table

Results

GroupNames Groups Instances   OwnerId      RequesterId  ReservationId       RunningInstance
---------- ------ ---------   -------      -----------  -------------       ---------------
{}         {}     {asdf222}   123          123                  r-123               {asdf222}
{}         {}     {asdf222}   123          123                  r-123               {asdf222}
{}         {}     {asdf222}   123          123                  r-123               {asdf222}

[jimdigriz]

You could use jq, if you like adding more dependencies, or just use the built in --query in AWS CLI:
aws --profile whatever ec2 describe-instances --query 'Reservations[].Instances[].[State.Name,InstanceType]' [ [ "running", "t2.small" ], [ "running", "t2.small" ], [ "running", "m3.medium" ] ]

[ygeo]

while read -r r ; echo start region : $r ; do while read -r i ; do echo instance-id: $i 2>&1 ; done < <(aws ec2 describe-instances --query "Reservations[].Instances[].InstanceId" --region $r --profile prd|perl -nl -e 's/\s+/\n/g;print'); echo "stop region: $r" ; done < <(aws ec2 describe-regions --output text --profile prd| cut -f 3)

[Selimcse98]

Include tags value to better understand your EC2 instances

for region in `aws ec2 describe-regions --output text | cut -f3`
do
     echo -e "\nListing Instances in region:'$region'..."
     aws ec2 describe-instances --region $region | jq '.Reservations[] | ( .Instances[] | {state: .State.Name, name: .KeyName, Tag_Name: .Tags[].Value,type: .InstanceType, key: .KeyName})'
done

[Ahmed-AG]

Just another solution I just wrote!
This small script can execute commands across all regions
Usage examples:
sudo ./aws-x.sh ec2 describe-instances
sudo ./aws-x.sh ec2 describe-instances | grep PrivateDnsName

!/bin/bash

echo "Getting AWS Avaiable Regions:"
sudo aws ec2 describe-regions --region eu-west-2 |grep RegionName |cut -d '"' -f 4 > /tmp/regions.txt
cat /tmp/regions.txt

Start executing

while read region; do
echo "Executing " $@ " on " $region ":"
aws --region $region $@
done

cleanup

rm /tmp/regions.txt

[nlsko]

!/bin/bash

echo Grabbing instances in all regions, please wait..
for region in aws ec2 describe-regions --output text | cut -f3
do
aws ec2 describe-instances --region $region --query 'Reservations[].Instances[].[State.Name,InstanceType,PublicIpAddress,Placement.AvailabilityZone]' --output text
done

------output-------

./get-instances.sh

Grabbing instances in all regions, please wait..

running t2.micro 54.93.120.174 eu-central-1b

running t2.micro 54.86.91.202 us-east-1d

[obijan42]

For those of you who also don't want to parse JSON with your eyeballs, here's an appended version that uses jq to give you a bit more-to-the-point summary.

aws cli supports the "query" parameter, which does this stuff as a build-in.

[Ahmed-AG]

Thanks obijan! I I started using jq few months back and it changed my life :)

[ajorpheus]

Note that the AWS Cli Docs do still recommend JQ for more advanced usage:

If you need more advanced features that might not be possible with --query, you can check out jq, a command line JSON processor. You can download it and find the official tutorial at http://stedolan.github.io/jq/

IMHO, --query is still worth looking if it works for your query use case since it is natively supported by AWS CLI. This option was added in aws cli v. 1.2.0 which was released on 2013/10/18, so you should have definitely have this option unless you have an ancient version of the CLI.

If you prefer to use the --query param, note that it uses JMESPath as its query language (the home page has a live JMESPath tester). The tutorial is worth a gander as well.

---

Sample Comparison of a similar query

The following queries list the RDS instances in the default region using the aws cli, parsing the json output from it using JQ and JMESPath

For example, a similar query seems to be far more readable OOTB in JMESPath:

# Using Jquery
aws  rds describe-db-instances --output json | jq -r '.DBInstances[] | "\(.DBInstanceIdentifier) -> \(.DBInstanceStatus) ( \(.DBInstanceClass), \(.Endpoint.Address):\(.Endpoint.Port)  ) "  '

## Outputs
someinstancename -> available ( db.t2.small, xxxxx.rds.amazonaws.com:1234 )

## Show a list of properties for DBInstances which have a status containing the word 'avail' 
aws rds describe-db-instances --output json --query "DBInstances[?contains(DBInstanceStatus, 'avail')].[DBInstanceIdentifier, DBInstanceClass, DBInstanceStatus, Endpoint.[Address, Port]]"

## Outputs
[
    [
        "someinstancename",
        "db.t2.small",
        "available",
        [
            "xxxxx.rds.amazonaws.com",
            1234
        ]
    ]
]

[PatMyron]

If opt-in-not-required is the 3rd column of output when you run aws ec2 describe-regions --output text, you can change cut -f3 to cut -f4 like so:

for region in `aws ec2 describe-regions --output text | cut -f4`
do

done

[coltoncat]

Probably a more robust way is to use a JMESPath query as follows:

for region in $(aws ec2 describe-regions --query "Regions[*].RegionName" --output text)
do

done

[genyrosk]

You could also do some further parsing to filter to the regions that you want with grep.
For instance, after installing awscli-aliases, add the following to ~/.aws/cli/alias to list your instances:

list-instances =
  !f() {
    echo "Filtering by \`$1\`"
    for region in $(aws ec2 describe-regions --query "Regions[*].RegionName" --output text | tr "\t" "\n" | grep "$1")
      do
        echo "\nListing Instances in region:'$region'..."
        aws ec2 describe-instances --region $region \
          --output table \
          --query "Reservations[*].Instances[*].{
                    Instance:InstanceId,
                    Type:InstanceType,
                    AZ:Placement.AvailabilityZone,
                    KeyName:KeyName,
                    Name:Tags[?Key==\`Name\`]|[0].Value,
                    Project:Tags[?Key==\`project\`]|[0].Value,
                    IP:PublicIpAddress,
                    State:State.Name,
                    CPUcores:CpuOptions.CoreCount,
                    CPUThreads:CpuOptions.ThreadsPerCore
                }"
      done
  }; f

Usage:

$ aws list-instances

Apply a filter to the region code, e.g. us for US regions:

$ aws list-instances 'us'

[Johnjosh11]

for region in aws ec2 describe-regions --output text | cut -f3
do
echo -e "\nListing Instances in region:'$region'..."
aws ec2 describe-instances --region $region | grep running > file.txt
cat file.txt | wc -l
done