Boto3: instances.filter(InstanceIds=[]) returns all instances, rather than an empty set

[goodcode]

I know that returning all instances might not be wrong, but because of that behavior we terminated all instances in two of our regions.
I believe that passing an empty ID whitelist should result in getting an empty instance list. In practice, this would cause less sad stories like ours.

[JordonPhillips]

The filter command is mapped directly to the service's filtering, and unfortunately S3 interprets an empty list as a catch-all. You could try raising the issue in the S3 forums, but unfortunately we can't do anything about it.

Sorry about your instances, I know that's gotta be a big blow. :(

[goodcode]

I understand... Thank you for the feedback. Will raise the issue to them!

[nchammas]

@goodcode - Did you end up raising the issue there? Please link to the thread from here if so.

@JordonPhillips - Do you have any suggestions as to how we should be handling situations like this?

If you're dynamically filtering instances based on some criteria, you typically want an empty list of instances to match 0 instances, not all instances. Does Boto3 have some pattern for that, or should we just have an if/else branch every time we use filter()?

[nchammas]

This problem seems to affect all collections that offer a filter() method. Passing an empty list like this, for example:

ec2.security_groups.filter(GroupIds=[])

also returns all security groups.

[nchammas]

OK, I think I found a good work-around for this issue.

Instead of using the named filter parameters like InstanceIds or GroupIds, always use Filters.

For example, to filter instances:

>>> list(ec2.instances.filter(InstanceIds=[]))
<returns all instances>
>>> list(ec2.instances.filter(Filters=[{'Name': 'instance-id', 'Values': []}]))
[]

And to filter security groups:

>>> list(ec2.security_groups.filter(GroupIds=[]))
<returns all security groups>
>>> list(ec2.security_groups.filter(Filters=[{'Name': 'group-id', 'Values': []}]))
[]

I still think it really sucks that InstanceIds=[] returns all instances. It's surprising and, since dynamically determined filters can always return an empty list, it's also dangerous. (i.e. You aren't normally going to explicitly specify [] as your filter. But [] can always be the result of, e.g. a list comprehension.)

However, at least we have a different pattern to use that avoids this issue.