Localhost Endpoint to DynamoDB Local with Boto3

Question

Although Amazon provides documentation regarding how to connect to dynamoDB local with Java, PHP and .Net, there is no description of how to connect to localhost:8000 using Python. Existing documentation on the web points to the use of the DynamoDBConnection method inside boto.dynamodb2.layer1, but this creates an incompatibility between live and test environments that use the boto3 protocol to manage dynamoDB.

In boto3, you can make a request to dynamo using the following constructor and variables set into the environment:

client = boto3.client('dynamodb')
table = client.list_tables()

Whereas the boto.dynamodb2.layer1 package requires you to construct the following:

client = DynamoDBConnection(
    host='localhost',
    port=8000,
    aws_access_key_id='anything',
    aws_secret_access_key='anything',
    is_secure=False)
table = client.list_tables()

Although it is possible to create logic which determines the proper constructor based upon the local environment, I am wary of building a set of methods which treat each constructor as the same. Instead, I would prefer to use boto3 for everything and to be able to set the endpoint for dynamoDB in the environmental variables. Unfortunately, that option does not appear to be currently be available.

Is there any way to use boto3 to define a dynamoDB local endpoint (like the other languages)? Or any chance that Amazon will plan to support this feature?

Answer 1

This is the tutorial python DynamoDb. It depicts how to connect to local instance.

http://docs.aws.amazon.com/amazondynamodb/latest/gettingstartedguide/GettingStarted.Python.01.html

It seems like the minimum required parameters are the following with the help of aws configuration (below).

dynamodb = boto3.resource('dynamodb', endpoint_url='http://localhost:8000/')

The region, access key and secret key parameters can be omitted when configure profile parameters using aws configure command (require install aws cli). However, you can create aws configuration files manually in your home (in case, you don't want to use aws cli).

file ~/.aws/config

[default]
output = json
region = anywhere

file ~/.aws/credentials

[default]
aws_access_key_id = whatever_id
aws_secret_access_key = whatever_key 

You can consult the aws configuration in http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html

Note in the local DynamoDb development region, aws_access_key_id and aws_secret_access_key values in those files can be anything. But if you want to use aws cli with the AWS then you must put the valid region, valid id and keys. They are available when you register to the AWS services.

More information, when you call

db = boto3.client('dynamodb')

The host the boto3 connect will base on the region parameter e.g. region=us-west-1 when call above api, it will connect to dynamodb.us-west-1.amazonaws.com. However, when pass parameter endpoint_url the region will not be used. For more AWS endpoint list please go to http://docs.aws.amazon.com/general/latest/gr/rande.html.

Answer 2

use dummy access key and id otherwise it will throw an exception on running the methods.

import boto3

dynamodb = boto3.session('dynamodb',
                          aws_access_key_id="anything",
                          aws_secret_access_key="anything",
                          region_name="us-west-2",
                          endpoint_url="http://localhost:8000")

Answer 3

It does support DynamoDB Local. You just need to set the appropriate endpoint such as you can do with other language SDKs

Here is a code snippet of how you can use boto3's client and resource interface via DynamoDB Local:

import boto3

# For a Boto3 client.
ddb = boto3.client('dynamodb', endpoint_url='http://localhost:8000')
response = ddb.list_tables()
print(response)

# For a Boto3 service resource
ddb = boto3.resource('dynamodb', endpoint_url='http://localhost:8000')
print(list(ddb.tables.all()))

Answer 4

Note: You will want to extend the above response to include region. I have appended to Kyle's code above. If your initial attempt is greeted with a region error, this will return the appropriate '[]' response.

import boto3

## For a Boto3 client ('client' is for low-level access to Dynamo service API)
ddb1 = boto3.client('dynamodb', endpoint_url='http://localhost:8000', region_name='us-west-2')
response = ddb1.list_tables()
print(response)

# For a Boto3 service resource ('resource' is for higher-level, abstracted access to Dynamo)
ddb2 = boto3.resource('dynamodb', endpoint_url='http://localhost:8000', region_name='us-west-2')
print(list(ddb2.tables.all()))