Basic interaction between boto3 and dynamoDB under python and how to backup and restore tables

Reference material: http://boto3.readthedocs.io/en/latest/reference/services/dynamodb.html

Recently, due to work needs, I studied the dynamoDB part of boto3 and gained some insights. Let’s sum it up here.

The first is the installation of boto3. On a machine equipped with python and pip, run

sudo pip install boto3

In the official website documentation, boto3 provides and The interfaces for dynamoDB interaction include the following:

batch_get_item()
batch_write_item()
can_paginate()
create_table()
delete_item()
delete_table()
describe_limits()
describe_table()
describe_time_to_live()
generate_presigned_url()
get_item()
get_paginator()
get_waiter()
list_tables()
list_tags_of_resource()
put_item()
query()
scan()
tag_resource()
untag_resource()
update_item()
update_table()
update_time_to_live()

To put it bluntly, it means adding, deleting, checking, and modifying tables and records. This article mainly describes the interfaces I have used recently.

To use boto3 in python, you must first import boto3. Of course, this is nonsense. For ease of use, I first wrote a configuration file in json format, as follows:

{"region_name":"xxx","aws_access_key_id":"xxx","aws_secret_access_key":"xxx"}

Then I encapsulated a class specifically used to operate dynamoDB. What is currently None

class dynamodb_operation():

It needs a method to read json files:

    def load_json(self,path):try:
            with open(path) as json_file:
                data = json.load(json_file)except Exception as e:print 'ERROR: no such file like ' + path
            exit(-1)else:return data

Since the file read in may not be in json format, here I am I just want him to report an error and quit. If you don't want it to exit, just change it in except.

Then, I hope that this class has a private member client, and the connection is established when I instantiate the object, so I have the following initialization method:

    def __init__(self,path):
        conf = self.load_json(path)
        self.client = boto3.client('dynamodb',region_name=conf['region_name'],aws_access_key_id=conf['aws_access_key_id'], aws_secret_access_key=conf['aws_secret_access_key'])

Corresponds to the previous configuration file.

With this foundation, you can encapsulate the method you want to use. The descriptions of each method on the official website will not be copied.

1. List all tables in dynamoDB

    def list_all_table(self):
        page=1LastEvaluationTableName = ""while True:if page == 1:
                response = self.client.list_tables()else:
                response = self.client.list_tables(
                        ExclusiveStartTableName=LastEvaluationTableName
                        )
            TableNames = response['TableNames']for table in TableNames:print tableif response.has_key('LastEvaluatedTableName'):
                LastEvaluationTableName = response["LastEvaluatedTableName"]else:breakpage += 1

list_table() method can only obtain the table names of up to 100 tables at a time, and returns each time When , the value with key "LastEvaluatedTableName" is the table name of the last table, which can be used as a parameter in the next request. In this loop call, all table names can be obtained. If there is no table later, there will be no LastEvaluatedTableName in the response. Here I just want to print the table name to the terminal. If you want to save it, you can also do it.

2. Get the information of a certain table describe_table()

    def get_table_desc_only(self,table):try:
            response = self.client.describe_table(TableName=table)except Exception as e:print 'ERROR: no such table like ' + table
            exit(-1)else:return response["Table"]

Here we just return the response["Table"] as it is without any other processing.

If I want to know the size of a table, I can:

    def get_table_size(self,table):
        response = self.get_table_desc_only(table)
        stastic = {}
        stastic['TableSizeBytes'] = response['TableSizeBytes']
        stastic['ItemCount'] = response['ItemCount']return stastic

If I want to know other information and only want to know that information, I can also write corresponding method.

3. Create a table

    def create_table(self,tablename,keySchema,attributeDefinitions,provisionedThroughput):
        table = self.client.create_table(
                TableName=tablename,
                KeySchema=keySchema,
                AttributeDefinitions=attributeDefinitions,
                ProvisionedThroughput=provisionedThroughput
                )# Wait until the table exists.self.client.get_waiter('table_exists').wait(TableName=tablename)

        response = self.client.describe_table(TableName=tablename)print response

This is creating a table without an index. Creating a table takes time, so the get_waiter() method is used.

4. Insert data

    def put_item(self,tableName,item):try:
            self.client.put_item(
                    TableName=tableName,
                    Item=item
                    )except Exception as e:print 'ERROR: put item fail. msg: ' + str(e)
            exit(-1)else:return

This encapsulated method needs to pass in a correctly formatted json, and the key must correspond to the table. For example:

{'uid':{'N':'999'},'aid':{'N':'999'},'sid':{'N':'999'},'ksid':{'N':'999'}}

5, delete table

    def delete_table(self,table):try:
            self.client.delete_table(
                    TableName=table
                    )   except Exception as e:print 'ERROR: delete table ' + table + ' fail. msg: ' + str(e)else:print 'delete table ' + table + ' succ'

To be continued...

The above is the detailed content of Basic interaction between boto3 and dynamoDB under python and how to backup and restore tables. For more information, please follow other related articles on the PHP Chinese website!