What is the correct ways to write Boto3 filters to

Question

I am trying to list the instances on tag values of different tag keys For eg> one tag key - Environment, other tag key - Role. My code is given below :

import argparse
import boto3

AWS_ACCESS_KEY_ID = '<Access Key>'
AWS_SECRET_ACCESS_KEY = '<Secret Key>'

def get_ec2_instances(Env,Role):
    ec2 = boto3.client("ec2", region)
    reservations = ec2.describe_instances(Filters={"tag:environment" :   Env, "tag:role" : Role})
    for reservation in reservations["Reservations"] :
        for instance in reservation["Instances"]:
             print  "%s" % (instance.tags['Name'])

if  __name__ == '__main__':

    regions = ['us-east-1','us-west-1','us-west-2','eu-west-1','sa-east-1',
               'ap-southeast-1','ap-southeast-2','ap-northeast-1']
    parser = argparse.ArgumentParser()
    parser.add_argument('Env', default="environment", help='value for   tag:environment');
    parser.add_argument('Role', default="role", help='value for tag:role');
    args = parser.parse_args()

    for region in regions: get_ec2_instances(args.Env, args.Role)

After running this script : python script.py arg1 arg2

I am getting following error

Traceback (most recent call last):
  File "script.py", line 27, in <module>
    for region in regions: get_ec2_instances(args.Env, args.Role)
  File "script.py", line 10, in get_ec2_instances
    reservations = ec2.describe_instances(Filters={"tag:environment" :  Env, "tag:role" : Role})
  File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 258, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 524, in _make_api_call
    api_params, operation_model, context=request_context)
  File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 577, in _convert_to_request_dict
    api_params, operation_model)
  File "/usr/local/lib/python2.7/dist-packages/botocore/validate.py", line 270, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter Filters, value: {'tag:role': 'arg1', 'tag:environment': 'arg2'}, type: <type 'dict'>, valid types: <type 'list'>, <type 'tuple'>

Answer 1

Fix the Env and Role, as I am not sure mine or mootmoot's answer will work because the Array for Values expects strings.

reservervations = ec2.describe_instances(
        Filters=[
           {'Name': 'tag:environment', 'Values': ['%s'], 'Name': 'tag:role', 'Values': ['%s']} % (Env, Role),
        ]]
    ).get(
        'Reservations', []
    )  

Answer 2

This looks familiar, did I modify this for somebody somewhere ;-) . Actually the code I wrote is in rush and not tested properly (And I don't bother to amend the % string formating and replace it with str.format() ) . In fact,using Filters parameter is not properly documented in AWS.

Please refer to Russell Ballestrini blog Filtering AWS resources with Boto3 to learn more about correct boto Filters method.

1. Filters accept list value, and info inside the tag should be dict. thus [{}]
2. Boto3 documentation is pretty ambiguous on how to use specify the tag name. It is confusing without examples when they say you may use tag:key. So many people will just do [{"tag:keyname","Values": [""] }] and it doesn't work. (Actually the origin code I assume the developer know how the filters works, so I just amend the structure only).
3. Actually, You MUST explicitly specify "Name" and "Values" pair. So the correct way to specify tag name is [{"Name" :"tag:keyname", "Values":[""] }]. It is tricky.

So the correct way of formatting a filters if you want to use for your example

filters = [{'Name':'tag:environment', 'Values':[Env]},
           {'Name':'tag:role', 'Values':[Role]}
          ]

(Update) And to make sure argparse take up string value, you just enforce the argument to take string values

parser.add_argument('Env', type=str, default="environment",
                    help='value for   tag:environment');
parser.add_argument('Role', type=str,default="role",
                    help='value for tag:role');

Answer 3

In my own python script I use the following:

import boto3
ec2client = boto3.client('ec2','us-east-1')
response = ec2client.describe_instances(Filters=[{'Name' : 'instance-state-name','Values' : ['running']}])