Dynamodb 简明教程

DynamoDB - Quick Guide

DynamoDB - Overview

DynamoDB 允许用户创建能够存储和检索任意数据量并服务于任意流量的数据库。它在服务器上自动分配数据和流量,以动态管理每个客户的请求,并且还保持了快速性能。

DynamoDB allows users to create databases capable of storing and retrieving any amount of data, and serving any amount of traffic. It automatically distributes data and traffic over servers to dynamically manage each customer’s requests, and also maintains fast performance.

DynamoDB vs. RDBMS

DynamoDB 使用 NoSQL 模型,这意味着它使用非关系系统。下表重点介绍了 DynamoDB 和 RDBMS 之间的差异 −

DynamoDB uses a NoSQL model, which means it uses a non-relational system. The following table highlights the differences between DynamoDB and RDBMS −

Common Tasks

RDBMS

DynamoDB

Connect to the Source

It uses a persistent connection and SQL commands.

It uses HTTP requests and API operations

Create a Table

Its fundamental structures are tables, and must be defined.

It only uses primary keys, and no schema on creation. It uses various data sources.

Get Table Info

All table info remains accessible

Only primary keys are revealed.

Load Table Data

It uses rows made of columns.

In tables, it uses items made of attributes

Read Table Data

It uses SELECT statements and filtering statements.

It uses GetItem, Query, and Scan.

Manage Indexes

It uses standard indexes created through SQL statements. Modifications to it occur automatically on table changes.

It uses a secondary index to achieve the same function. It requires specifications (partition key and sort key).

Modify Table Data

It uses an UPDATE statement.

It uses an UpdateItem operation.

Delete Table Data

It uses a DELETE statement.

It uses a DeleteItem operation.

Delete a Table

It uses a DROP TABLE statement.

It uses a DeleteTable operation.

Advantages

DynamoDB 的两个主要优点是可伸缩性与灵活性。它不要求使用特定数据源与结构,这让用户能在几乎所有内容上操作,但以统一的方式。

The two main advantages of DynamoDB are scalability and flexibility. It does not force the use of a particular data source and structure, allowing users to work with virtually anything, but in a uniform way.

其设计也支持从较轻任务与操作 到 要求苛刻的企业功能 的广泛用途。它还允许使用多种语言:Ruby、Java、Python、C#、Erlang、PHP 和 Perl。

Its design also supports a wide range of use from lighter tasks and operations to demanding enterprise functionality. It also allows simple use of multiple languages: Ruby, Java, Python, C#, Erlang, PHP, and Perl.

Limitations

但是,DynamoDB 确实存在某些限制,不过这些限制不会一定造成大问题或妨碍稳固的开发。

DynamoDB does suffer from certain limitations, however, these limitations do not necessarily create huge problems or hinder solid development.

您可以从以下观点对其进行审阅 −

You can review them from the following points −

  1. Capacity Unit Sizes − A read capacity unit is a single consistent read per second for items no larger than 4KB. A write capacity unit is a single write per second for items no bigger than 1KB.

  2. Provisioned Throughput Min/Max − All tables and global secondary indices have a minimum of one read and one write capacity unit. Maximums depend on region. In the US, 40K read and write remains the cap per table (80K per account), and other regions have a cap of 10K per table with a 20K account cap.

  3. Provisioned Throughput Increase and Decrease − You can increase this as often as needed, but decreases remain limited to no more than four times daily per table.

  4. Table Size and Quantity Per Account − Table sizes have no limits, but accounts have a 256 table limit unless you request a higher cap.

  5. Secondary Indexes Per Table − Five local and five global are permitted.

  6. Projected Secondary Index Attributes Per Table − DynamoDB allows 20 attributes.

  7. Partition Key Length and Values − Their minimum length sits at 1 byte, and maximum at 2048 bytes, however, DynamoDB places no limit on values.

  8. Sort Key Length and Values − Its minimum length stands at 1 byte, and maximum at 1024 bytes, with no limit for values unless its table uses a local secondary index.

  9. Table and Secondary Index Names − Names must conform to a minimum of 3 characters in length, and a maximum of 255. They use the following characters: AZ, a-z, 0-9, “_”, “-”, and “.”.

  10. Attribute Names − One character remains the minimum, and 64KB the maximum, with exceptions for keys and certain attributes.

  11. Reserved Words − DynamoDB does not prevent the use of reserved words as names.

  12. Expression Length − Expression strings have a 4KB limit. Attribute expressions have a 255-byte limit. Substitution variables of an expression have a 2MB limit.

DynamoDB - Basic Concepts

在使用 DynamoDB 之前,你必须让自己熟悉它的基本组件和生态系统。在 DynamoDB 生态系统中,你可以使用表、属性和条目。表包含一组条目,条目包含一组属性。属性是无需进一步分解的数据的基本元素,即字段。

Before using DynamoDB, you must familiarize yourself with its basic components and ecosystem. In the DynamoDB ecosystem, you work with tables, attributes, and items. A table holds sets of items, and items hold sets of attributes. An attribute is a fundamental element of data requiring no further decomposition, i.e., a field.

Primary Key

主键用作表条目的唯一标识符,辅助索引提供查询灵活性。DynamoDB 流通过修改表数据记录事件。

The Primary Keys serve as the means of unique identification for table items, and secondary indexes provide query flexibility. DynamoDB streams record events by modifying the table data.

表创建不仅需要设置名称,还需要设置主键;它标识表条目。没有两条条目共享同一个键。DynamoDB 使用两种类型的主键 −

The Table Creation requires not only setting a name, but also the primary key; which identifies table items. No two items share a key. DynamoDB uses two types of primary keys −

  1. Partition Key − This simple primary key consists of a single attribute referred to as the “partition key.” Internally, DynamoDB uses the key value as input for a hash function to determine storage.

  2. Partition Key and Sort Key − This key, known as the “Composite Primary Key”, consists of two attributes. The partition key and The sort key. DynamoDB applies the first attribute to a hash function, and stores items with the same partition key together; with their order determined by the sort key. Items can share partition keys, but not sort keys.

主键属性仅允许标量(单个)值;和字符串、数字或二进制数据类型。非键属性没有这些约束。

The Primary Key attributes only allow scalar (single) values; and string, number, or binary data types. The non-key attributes do not have these constraints.

Secondary Indexes

这些索引允许你使用备用键查询表数据。尽管 DynamoDB 并不强制使用它们,但它们会优化查询。

These indexes allow you to query table data with an alternate key. Though DynamoDB does not force their use, they optimize querying.

DynamoDB 使用两种类型的二级索引 −

DynamoDB uses two types of secondary indexes −

  1. Global Secondary Index − This index possesses partition and sort keys, which can differ from table keys.

  2. Local Secondary Index − This index possesses a partition key identical to the table, however, its sort key differs.

API

DynamoDB 提供的 API 操作包括控制平面、数据平面(例如创建、读取、更新和删除)和流的操作。在控制平面操作中,你可以使用以下工具创建和管理表 −

The API operations offered by DynamoDB include those of the control plane, data plane (e.g., creation, reading, updating, and deleting), and streams. In control plane operations, you create and manage tables with the following tools −

  1. CreateTable

  2. DescribeTable

  3. ListTables

  4. UpdateTable

  5. DeleteTable

在数据平面中,你可以使用以下工具执行 CRUD 操作 −

In the data plane, you perform CRUD operations with the following tools −

Create

Read

Update

Delete

PutItem BatchWriteItem

GetItem BatchGetItem Query Scan

UpdateItem

DeleteItem BatchWriteItem

流操作控制表流。你可以查看以下流工具 −

The stream operations control table streams. You can review the following stream tools −

  1. ListStreams

  2. DescribeStream

  3. GetShardIterator

  4. GetRecords

Provisioned Throughput

在表创建中,你可以指定预置吞吐量,它为读取和写入预留了资源。你可以使用容量单位来衡量和设置吞吐量。

In table creation, you specify provisioned throughput, which reserves resources for reads and writes. You use capacity units to measure and set throughput.

当应用程序超出设置的吞吐量时,请求会失败。DynamoDB GUI 控制台允许监控已设置和已使用的吞吐量,以实现更好和更动态的预置。

When applications exceed the set throughput, requests fail. The DynamoDB GUI console allows monitoring of set and used throughput for better and dynamic provisioning.

Read Consistency

DynamoDB 使用 eventually consistentstrongly consistent 读取来支持动态应用程序需求。最终一致性读取并不总能提供当前数据。

DynamoDB uses eventually consistent and strongly consistent reads to support dynamic application needs. Eventually consistent reads do not always deliver current data.

强一致性读取始终提供当前数据(设备故障或网络问题除外)。最终一致性读取是默认设置,要求在 ConsistentRead 参数中设置 true 来更改它。

The strongly consistent reads always deliver current data (with the exception of equipment failure or network problems). Eventually consistent reads serve as the default setting, requiring a setting of true in the ConsistentRead parameter to change it.

Partitions

DynamoDB 使用分区存储数据。这些表的存储分配具有 SSD 支持并跨区域自动复制。DynamoDB 管理所有分区任务,无需用户参与。

DynamoDB uses partitions for data storage. These storage allocations for tables have SSD backing and automatically replicate across zones. DynamoDB manages all the partition tasks, requiring no user involvement.

在创建表时,表进入分配分区的 CREATING 状态。当它进入 ACTIVE 状态时,你可以执行操作。系统在容量达到最大值或更改吞吐量时更改分区。

In table creation, the table enters the CREATING state, which allocates partitions. When it reaches ACTIVE state, you can perform operations. The system alters partitions when its capacity reaches maximum or when you change throughput.

DynamoDB - Environment

DynamoDB 环境仅包含使用你的亚马逊网络服务帐户来访问 DynamoDB GUI 控制台,不过你也可以执行本地安装。

The DynamoDB Environment only consists of using your Amazon Web Services account to access the DynamoDB GUI console, however, you can also perform a local install.

导航至以下网站 − https://aws.amazon.com/dynamodb/

Navigate to the following website − https://aws.amazon.com/dynamodb/

单击“使用亚马逊 DynamoDB 入门”按钮,或在没有亚马逊网络服务帐户时单击“创建 AWS 帐户”按钮。简单的引导流程将告知你所有相关费用和要求。

Click the “Get Started with Amazon DynamoDB” button, or the “Create an AWS Account” button if you do not have an Amazon Web Services account. The simple, guided process will inform you of all the related fees and requirements.

执行该流程的所有必要步骤后,你将具有访问权限。只需登录 AWS 控制台,然后导航至 DynamoDB 控制台。

After performing all the necessary steps of the process, you will have the access. Simply sign in to the AWS console, and then navigate to the DynamoDB console.

请务必删除未使用或不必要的素材,以免产生相关费用。

Be sure to delete unused or unnecessary material to avoid associated fees.

Local Install

AWS(亚马逊网络服务)提供了用于本地安装的 DynamoDB 版本。它支持在没有 Web 服务或连接的情况下创建应用程序。通过允许使用本地数据库,它也降低了预置吞吐量、数据存储和传输费用。本指南假定为本地安装。

The AWS (Amazon Web Service) provides a version of DynamoDB for local installations. It supports creating applications without the web service or a connection. It also reduces provisioned throughput, data storage, and transfer fees by allowing a local database. This guide assumes a local install.

准备就绪时,你可以对应用程序进行一些微小的调整,以使其转换为 AWS 使用。

When ready for deployment, you can make a few small adjustments to your application to convert it to AWS use.

安装文件是 .jar executable 。它在 Linux、Unix、Windows 和任何其他支持 Java 的操作系统中运行。使用以下链接之一下载该文件 −

The install file is a .jar executable. It runs in Linux, Unix, Windows, and any other OS with Java support. Download the file by using one of the following links −

Note − 其他存储库提供该文件,但不一定是最新的版本。使用上述链接获取最新的安装文件。另外,确保你具有 Java 运行时引擎 (JRE) 6.x 或更高版本。较低版本上无法运行 DynamoDB。

Note − Other repositories offer the file, but not necessarily the latest version. Use the links above for up-to-date install files. Also, ensure you have Java Runtime Engine (JRE) version 6.x or a newer version. DynamoDB cannot run with older versions.

下载适当的存档后,解压缩其目录 (DynamoDBLocal.jar) 并将其放置在所需位置。

After downloading the appropriate archive, extract its directory (DynamoDBLocal.jar) and place it in the desired location.

然后,你可以通过打开命令提示符、导航至包含 DynamoDBLocal.jar 的目录并输入以下命令来启动 DynamoDB −

You can then start DynamoDB by opening a command prompt, navigating to the directory containing DynamoDBLocal.jar, and entering the following command −

java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -sharedDb

你也可以通过关闭用于启动它的命令提示符来停止 DynamoDB。

You can also stop the DynamoDB by closing the command prompt used to start it.

Working Environment

你可以使用 JavaScript 外壳程序、GUI 控制台和多种语言来使用 DynamoDB。可用语言包括 Ruby、Java、Python、C#、Erlang、PHP 和 Perl。

You can use a JavaScript shell, a GUI console, and multiple languages to work with DynamoDB. The languages available include Ruby, Java, Python, C#, Erlang, PHP, and Perl.

在本教程中,我们使用 Java 和 GUI 控制台示例,以实现概念和代码清晰。安装 Java IDE、适用于 Java 的 AWS SDK,并为 Java SDK 设置 AWS 安全凭证,以利用 Java。

In this tutorial, we use Java and GUI console examples for conceptual and code clarity. Install a Java IDE, the AWS SDK for Java, and setup AWS security credentials for the Java SDK in order to utilize Java.

Conversion from Local to Web Service Code

准备就绪后,你需要更改你的代码。调整取决于代码语言和其他因素。主要更改仅包含将 endpoint 从本地点更改为 AWS 区域。其他更改需要对你的应用程序进行更深入的分析。

When ready for deployment, you will need to alter your code. The adjustments depend on code language and other factors. The main change merely consists of changing the endpoint from a local point to an AWS region. Other changes require deeper analysis of your application.

本地安装与 Web 服务在许多方面存在差异,包括但不限于以下主要差异 −

A local install differs from the web service in many ways including, but not limited to the following key differences −

  1. The local install creates tables immediately, but the service takes much longer.

  2. The local install ignores throughput.

  3. The deletion occurs immediately in a local install.

  4. The reads/writes occur quickly in local installs due to the absence of network overhead.

DynamoDB - Operations Tools

DynamoDB 提供三种执行操作的选择:基于 Web 的 GUI 控制台、JavaScript shell 和您选择的编程语言。

DynamoDB provides three options for performing operations: a web-based GUI console, a JavaScript shell, and a programming language of your choice.

在本教程中,我们将重点介绍为清晰和概念理解而使用 GUI 控制台和 Java 语言。

In this tutorial, we will focus on using the GUI console and Java language for clarity and conceptual understanding.

GUI Console

Amazon DynamoDB 的 GUI 控制台或 AWS 管理控制台可在以下地址找到 − https://console.aws.amazon.com/dynamodb/home

The GUI console or the AWS Management Console for Amazon DynamoDB can be found at the following address − https://console.aws.amazon.com/dynamodb/home

它允许您执行以下任务 −

It allows you to perform the following tasks −

  1. CRUD

  2. View Table Items

  3. Perform Table Queries

  4. Set Alarms for Table Capacity Monitoring

  5. View Table Metrics in Real-Time

  6. View Table Alarms

gui console

如果您的 DynamoDB 账户没有表,则在访问时,它会指导您创建表。其主屏幕提供了三个执行常见操作的快捷方式 −

If your DynamoDB account has no tables, on access, it guides you through creating a table. Its main screen offers three shortcuts for performing common operations −

  1. Create Tables

  2. Add and Query Tables

  3. Monitor and Manage Tables

The JavaScript Shell

DynamoDB 包含一个交互式 JavaScript shell。shell 在 Web 浏览器中运行,推荐的浏览器包括 Firefox 和 Chrome。

DynamoDB includes an interactive JavaScript shell. The shell runs inside a web browser, and the recommended browsers include Firefox and Chrome.

javascript shell

Note − 使用其他浏览器可能会导致错误。

Note − Using other browsers may result in errors.

通过打开 Web 浏览器并输入以下地址访问 shell − http://localhost:8000/shell

Access the shell by opening a web browser and entering the following address −http://localhost:8000/shell

使用 shell 的方法是在左窗格中输入 JavaScript,然后单击左窗格右上角的“播放”图标按钮,该按钮运行代码。代码结果显示在右窗格中。

Use the shell by entering JavaScript in the left pane, and clicking the “Play” icon button in the top right corner of the left pane, which runs the code. The code results display in the right pane.

DynamoDB and Java

通过使用您的 Java 开发环境来将 Java 与 DynamoDB 一起使用。操作与常规 Java 语法和结构一致。

Use Java with DynamoDB by utilizing your Java development environment. Operations confirm to normal Java syntax and structure.

DynamoDB - Data Types

DynamoDB 支持的数据类型包括特定于属性、操作和您选择的编码语言的数据类型。

Data types supported by DynamoDB include those specific to attributes, actions, and your coding language of choice.

Attribute Data Types

DynamoDB 支持大量表属性数据类型。每种数据类型都属于以下三个类别之一 −

DynamoDB supports a large set of data types for table attributes. Each data type falls into one of the three following categories −

  1. Scalar − These types represent a single value, and include number, string, binary, Boolean, and null.

  2. Document − These types represent a complex structure possessing nested attributes, and include lists and maps.

  3. Set − These types represent multiple scalars, and include string sets, number sets, and binary sets.

记住 DynamoDB 是无模式、无 SQL 数据库,创建表时不需要属性或数据类型定义。它仅需要主键属性数据类型,而关系数据库管理系统 (RDBMS) 在创建表时则需要列数据类型。

Remember DynamoDB as a schemaless, NoSQL database that does not need attribute or data type definitions when creating a table. It only requires a primary key attribute data types in contrast to RDBMS, which require column data types on table creation.

Scalars

  1. Numbers − They are limited to 38 digits, and are either positive, negative, or zero.

  2. String − They are Unicode using UTF-8, with a minimum length of >0 and maximum of 400KB.

  3. Binary − They store any binary data, e.g., encrypted data, images, and compressed text. DynamoDB views its bytes as unsigned.

  4. Boolean − They store true or false.

  5. Null − They represent an unknown or undefined state.

Document

  1. List − It stores ordered value collections, and uses square ([…​]) brackets.

  2. Map − It stores unordered name-value pair collections, and uses curly ({…​}) braces.

Set

无论是否为数字、字符串或二进制,集合必须包含相同类型的值。针对集合设置的唯一限制涉及 400KB 项目大小限制,并且每个元素必须唯一。

Sets must contain elements of the same type whether number, string, or binary. The only limits placed on sets consist of the 400KB item size limit, and each element being unique.

Action Data Types

DynamoDB API 保存由操作使用的各种数据类型。你可以查看以下一些关键类型 -

DynamoDB API holds various data types used by actions. You can review a selection of the following key types −

  1. AttributeDefinition − It represents key table and index schema.

  2. Capacity − It represents the quantity of throughput consumed by a table or index.

  3. CreateGlobalSecondaryIndexAction − It represents a new global secondary index added to a table.

  4. LocalSecondaryIndex − It represents local secondary index properties.

  5. ProvisionedThroughput − It represents the provisioned throughput for an index or table.

  6. PutRequest − It represents PutItem requests.

  7. TableDescription − It represents table properties.

Supported Java Datatypes

DynamoDB 为 Java 提供对原始数据类型、集合设置和任意类型的支持。

DynamoDB provides support for primitive data types, Set collections, and arbitrary types for Java.

DynamoDB - Create Table

创建表通常包括生成表、为其命名、建立其主键属性以及设置属性数据类型。

Creating a table generally consists of spawning the table, naming it, establishing its primary key attributes, and setting attribute data types.

利用 GUI 控制台、Java 或其他选项执行这些任务。

Utilize the GUI Console, Java, or another option to perform these tasks.

Create Table using the GUI Console

通过访问 https://console.aws.amazon.com/dynamodb 中的控制台创建表。然后选择“创建表”选项。

Create a table by accessing the console at https://console.aws.amazon.com/dynamodb. Then choose the “Create Table” option.

gui console

我们的示例生成一个包含产品信息的表,其中具有唯一属性的产品由 ID 号(数字属性)标识。在 Create Table 屏幕中,在表名字段输入表名;在分区键字段中输入主键 (ID);输入数据类型的“数字”。

Our example generates a table populated with product information, with products of unique attributes identified by an ID number (numeric attribute). In the Create Table screen, enter the table name within the table name field; enter the primary key (ID) within the partition key field; and enter “Number” for the data type.

create table

输入所有信息后,选择 Create

After entering all information, select Create.

Create Table using Java

使用 Java 创建相同的表。其主键包含以下两个属性 -

Use Java to create the same table. Its primary key consists of the following two attributes −

  1. ID − Use a partition key, and the ScalarAttributeType N, meaning number.

  2. Nomenclature − Use a sort key, and the ScalarAttributeType S, meaning string.

Java 使用 createTable method 生成一个表;并在调用中,指定表名、主键属性和属性数据类型。

Java uses the createTable method to generate a table; and within the call, table name, primary key attributes, and attribute data types are specified.

您可以查看以下示例 -

You can review the following example −

import java.util.Arrays;

import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Table;

import com.amazonaws.services.dynamodbv2.model.AttributeDefinition;
import com.amazonaws.services.dynamodbv2.model.KeySchemaElement;
import com.amazonaws.services.dynamodbv2.model.KeyType;
import com.amazonaws.services.dynamodbv2.model.ProvisionedThroughput;
import com.amazonaws.services.dynamodbv2.model.ScalarAttributeType;

public class ProductsCreateTable {
   public static void main(String[] args) throws Exception {
      AmazonDynamoDBClient client = new AmazonDynamoDBClient()
         .withEndpoint("http://localhost:8000");

      DynamoDB dynamoDB = new DynamoDB(client);
      String tableName = "Products";
      try {
         System.out.println("Creating the table, wait...");
         Table table = dynamoDB.createTable (tableName,
            Arrays.asList (
               new KeySchemaElement("ID", KeyType.HASH), // the partition key
                                                         // the sort key
               new KeySchemaElement("Nomenclature", KeyType.RANGE)
            ),
            Arrays.asList (
               new AttributeDefinition("ID", ScalarAttributeType.N),
               new AttributeDefinition("Nomenclature", ScalarAttributeType.S)
            ),
            new ProvisionedThroughput(10L, 10L)
         );
         table.waitForActive();
         System.out.println("Table created successfully.  Status: " +
            table.getDescription().getTableStatus());

      } catch (Exception e) {
         System.err.println("Cannot create the table: ");
         System.err.println(e.getMessage());
      }
   }
}

在上述示例中,请注意端点: .withEndpoint

In the above example, note the endpoint: .withEndpoint.

它指示使用 localhost 使用本地安装。另外,请注意所需的 ProvisionedThroughput parameter ,本地安装将其忽略。

It indicates the use of a local install by using the localhost. Also, note the required ProvisionedThroughput parameter, which the local install ignores.

DynamoDB - Load Table

加载表通常包含创建源文件、确保源文件符合与 DynamoDB 兼容的语法、将源文件发送到目的地,然后确认成功填充。

Loading a table generally consists of creating a source file, ensuring the source file conforms to a syntax compatible with DynamoDB, sending the source file to the destination, and then confirming a successful population.

利用 GUI 控制台、Java 或其他选项执行任务。

Utilize the GUI console, Java, or another option to perform the task.

Load Table using GUI Console

结合使用命令行和控制台加载数据。您可以通过多种方式加载数据,其中一些方式如下 -

Load data using a combination of the command line and console. You can load data in multiple ways, some of which are as follows −

  1. The Console

  2. The Command Line

  3. Code and also

  4. Data Pipeline (a feature discussed later in the tutorial)

然而,为了速度,此示例同时使用 shell 和控制台。首先,使用以下语法将源数据加载到目标中 −

However, for speed, this example uses both the shell and console. First, load the source data into the destination with the following syntax −

aws dynamodb batch-write-item -–request-items file://[filename]

例如 -

For example −

aws dynamodb batch-write-item -–request-items file://MyProductData.json

通过访问 − 中的控制台验证操作的成功

Verify the success of the operation by accessing the console at −

从导航窗格选择 Tables ,并从表格列表中选择目标表格。

Choose Tables from the navigation pane, and select the destination table from the table list.

选择 Items 选项卡以检查用于填充表格的数据。选择 Cancel 以返回到表格列表。

Select the Items tab to examine the data you used to populate the table. Select Cancel to return to the table list.

Load Table using Java

首先创建一个源文件以使用 Java。我们的源文件使用 JSON 格式。每个产品有两个主键属性(Id 和命名法)和一个 JSON 映射(Stat)。

Employ Java by first creating a source file. Our source file uses JSON format. Each product has two primary key attributes (ID and Nomenclature) and a JSON map (Stat) −

[
   {
      "ID" : ... ,
      "Nomenclature" : ... ,
      "Stat" : { ... }
   },
   {
      "ID" : ... ,
      "Nomenclature" : ... ,
      "Stat" : { ... }
   },
    ...
]

您可以查看以下示例 -

You can review the following example −

{
   "ID" : 122,
   "Nomenclature" : "Particle Blaster 5000",
   "Stat" : {
      "Manufacturer" : "XYZ Inc.",
      "sales" : "1M+",
      "quantity" : 500,
      "img_src" : "http://www.xyz.com/manuals/particleblaster5000.jpg",
      "description" : "A laser cutter used in plastic manufacturing."
   }
}

下一步是将文件放在您的应用程序使用的目录中。

The next step is to place the file in the directory used by your application.

Java 主要使用 putItempath methods 来执行加载。

Java primarily uses the putItem and path methods to perform the load.

您可以查看以下代码示例来处理文件并加载它−

You can review the following code example for processing a file and loading it −

import java.io.File;
import java.util.Iterator;

import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.Table;

import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.databind.node.ObjectNode;

public class ProductsLoadData {
   public static void main(String[] args) throws Exception {
      AmazonDynamoDBClient client = new AmazonDynamoDBClient()
         .withEndpoint("http://localhost:8000");

      DynamoDB dynamoDB = new DynamoDB(client);
      Table table = dynamoDB.getTable("Products");
      JsonParser parser = new JsonFactory()
         .createParser(new File("productinfo.json"));

      JsonNode rootNode = new ObjectMapper().readTree(parser);
      Iterator<JsonNode> iter = rootNode.iterator();
      ObjectNode currentNode;

      while (iter.hasNext()) {
         currentNode = (ObjectNode) iter.next();
         int ID = currentNode.path("ID").asInt();
         String Nomenclature = currentNode.path("Nomenclature").asText();

         try {
            table.putItem(new Item()
               .withPrimaryKey("ID", ID, "Nomenclature", Nomenclature)
               .withJSON("Stat", currentNode.path("Stat").toString()));
            System.out.println("Successful load: " + ID + " " + Nomenclature);
         } catch (Exception e) {
            System.err.println("Cannot add product: " + ID + " " + Nomenclature);
            System.err.println(e.getMessage());
            break;
         }
      }
      parser.close();
   }
}

DynamoDB - Query Table

查询一个表主要需要选择一个表,指定一个分区键并执行查询;可以使用二级索引和通过扫描操作执行更深层次的过滤。

Querying a table primarily requires selecting a table, specifying a partition key, and executing the query; with the options of using secondary indexes and performing deeper filtering through scan operations.

使用 GUI 控制台、Java 或其他选项来执行任务。

Utilize the GUI Console, Java, or another option to perform the task.

Query Table using the GUI Console

使用先前创建的表执行一些简单查询。首先,在 https://console.aws.amazon.com/dynamodb 处打开控制台。

Perform some simple queries using the previously created tables. First, open the console at https://console.aws.amazon.com/dynamodb

从导航窗格中选择 Tables 并从表格列表中选择 Reply 。然后选择 Items 选项卡以查看已加载的数据。

Choose Tables from the navigation pane and select Reply from the table list. Then select the Items tab to see the loaded data.

选择 Create Item 按钮下方的数据过滤链接(“扫描:[表] 答复”)。

Select the data filtering link (“Scan: [Table] Reply”) beneath the Create Item button.

query table using the gui console

在过滤界面中,为操作选择查询。输入适当的分区键值,然后单击 Start

In the filtering screen, select Query for the operation. Enter the appropriate partition key value, and click Start.

然后, Reply 表返回匹配的项目。

The Reply table then returns matching items.

reply table

Query Table using Java

在 Java 中使用查询方法来执行数据检索操作。它需要指定分区键值,排序键为可选项。

Use the query method in Java to perform data retrieval operations. It requires specifying the partition key value, with the sort key as optional.

首先创建一个描述参数的 querySpec object 以对 Java 查询进行编码。然后将对象传递给查询方法。我们使用前面示例中的分区键。

Code a Java query by first creating a querySpec object describing parameters. Then pass the object to the query method. We use the partition key from the previous examples.

您可以查看以下示例 -

You can review the following example −

import java.util.HashMap;
import java.util.Iterator;

import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.ItemCollection;
import com.amazonaws.services.dynamodbv2.document.QueryOutcome;
import com.amazonaws.services.dynamodbv2.document.Table;
import com.amazonaws.services.dynamodbv2.document.spec.QuerySpec;
import com.amazonaws.services.dynamodbv2.document.utils.NameMap;

public class ProductsQuery {
   public static void main(String[] args) throws Exception {
      AmazonDynamoDBClient client = new AmazonDynamoDBClient()
         .withEndpoint("http://localhost:8000");

      DynamoDB dynamoDB = new DynamoDB(client);
      Table table = dynamoDB.getTable("Products");
      HashMap<String, String> nameMap = new HashMap<String, String>();
      nameMap.put("#ID", "ID");
      HashMap<String, Object> valueMap = new HashMap<String, Object>();
      valueMap.put(":xxx", 122);
      QuerySpec querySpec = new QuerySpec()
         .withKeyConditionExpression("#ID = :xxx")
         .withNameMap(new NameMap().with("#ID", "ID"))
         .withValueMap(valueMap);

      ItemCollection<QueryOutcome> items = null;
      Iterator<Item> iterator = null;
      Item item = null;
      try {
         System.out.println("Product with the ID 122");
         items = table.query(querySpec);
         iterator = items.iterator();

         while (iterator.hasNext()) {
            item = iterator.next();
            System.out.println(item.getNumber("ID") + ": "
               + item.getString("Nomenclature"));
         }
      } catch (Exception e) {
         System.err.println("Cannot find products with the ID number 122");
         System.err.println(e.getMessage());
      }
   }
}

请注意,查询使用分区键,但是二级索引为查询提供了另一种备选方法。它们的灵活性允许查询非键属性,本教程稍后将讨论该主题。

Note that the query uses the partition key, however, secondary indexes provide another option for queries. Their flexibility allows querying of non-key attributes, a topic which will be discussed later in this tutorial.

扫描方法还通过收集所有表数据来支持检索操作。 optional .withFilterExpression 阻止指定条件之外的项目出现在结果中。

The scan method also supports retrieval operations by gathering all the table data. The optional .withFilterExpression prevents items outside of specified criteria from appearing in results.

在接下来的教程中,我们将详细讨论 scanning 。现在,看看下面的例子−

Later in this tutorial, we will discuss scanning in detail. Now, take a look at the following example −

import java.util.Iterator;

import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.ItemCollection;
import com.amazonaws.services.dynamodbv2.document.ScanOutcome;
import com.amazonaws.services.dynamodbv2.document.Table;
import com.amazonaws.services.dynamodbv2.document.spec.ScanSpec;
import com.amazonaws.services.dynamodbv2.document.utils.NameMap;
import com.amazonaws.services.dynamodbv2.document.utils.ValueMap;

public class ProductsScan {
   public static void main(String[] args) throws Exception {
      AmazonDynamoDBClient client = new AmazonDynamoDBClient()
         .withEndpoint("http://localhost:8000");

      DynamoDB dynamoDB = new DynamoDB(client);
      Table table = dynamoDB.getTable("Products");
      ScanSpec scanSpec = new ScanSpec()
         .withProjectionExpression("#ID, Nomenclature , stat.sales")
         .withFilterExpression("#ID between :start_id and :end_id")
         .withNameMap(new NameMap().with("#ID",  "ID"))
         .withValueMap(new ValueMap().withNumber(":start_id", 120)
         .withNumber(":end_id", 129));

      try {
         ItemCollection<ScanOutcome> items = table.scan(scanSpec);
         Iterator<Item> iter = items.iterator();

         while (iter.hasNext()) {
            Item item = iter.next();
            System.out.println(item.toString());
         }
      } catch (Exception e) {
         System.err.println("Cannot perform a table scan:");
         System.err.println(e.getMessage());
      }
   }
}

DynamoDB - Delete Table

在本章中,我们将讨论如何删除表格以及删除表格的不同方法。

In this chapter, we will discuss regarding how we can delete a table and also the different ways of deleting a table.

表删除是一个简单的操作,只需要表名就可以进行。使用 GUI 控制台、Java 或其他任何选项执行此任务。

Table deletion is a simple operation requiring little more than the table name. Utilize the GUI console, Java, or any other option to perform this task.

Delete Table using the GUI Console

通过首先访问控制台执行删除操作:

Perform a delete operation by first accessing the console at −

从导航窗格中选择 Tables ,并从表列表中选择要删除的表,如下图所示。

Choose Tables from the navigation pane, and choose the table desired for deletion from the table list as shown in the following screeenshot.

delete table using the gui console

最后,选择 Delete Table 。选择“删除表”后,会出现一个确认对话框。然后,你的表就被删除了。

Finally, select Delete Table. After choosing Delete Table, a confirmation appears. Your table is then deleted.

Delete Table using Java

使用 delete 方法删除表。下面给出示例以更好地解释该概念。

Use the delete method to remove a table. An example is given below to explain the concept better.

import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Table;

public class ProductsDeleteTable {
   public static void main(String[] args) throws Exception {
      AmazonDynamoDBClient client = new AmazonDynamoDBClient()
         .withEndpoint("http://localhost:8000");

      DynamoDB dynamoDB = new DynamoDB(client);
      Table table = dynamoDB.getTable("Products");
      try {
         System.out.println("Performing table delete, wait...");
         table.delete();
         table.waitForDelete();
         System.out.print("Table successfully deleted.");
      } catch (Exception e) {
         System.err.println("Cannot perform table delete: ");
         System.err.println(e.getMessage());
      }
   }
}

DynamoDB - API Interface

DynamoDB 提供了一系列强大的 API 工具,用于表操作、数据读取和数据修改。

DynamoDB offers a wide set of powerful API tools for table manipulation, data reads, and data modification.

Amazon 建议使用 AWS SDKs (例如,Java SDK)而不是调用低级别的 API。此类库消除了直接与低级别 API 交互的必要性。这些库简化了常见的任务,如认证、序列化和连接。

Amazon recommends using AWS SDKs (e.g., the Java SDK) rather than calling low-level APIs. The libraries make interacting with low-level APIs directly unnecessary. The libraries simplify common tasks such as authentication, serialization, and connections.

Manipulate Tables

DynamoDB 提供了五种用于表管理的低级别操作:

DynamoDB offers five low-level actions for Table Management −

  1. CreateTable − This spawns a table and includes throughput set by the user. It requires you to set a primary key, whether composite or simple. It also allows one or multiple secondary indexes.

  2. ListTables − This provides a list of all tables in the current AWS user’s account and tied to their endpoint.

  3. UpdateTable − This alters throughput, and global secondary index throughput.

  4. DescribeTable − This provides table metadata; for example, state, size, and indices.

  5. DeleteTable − This simply erases the table and its indices.

Read Data

DynamoDB 提供了四种用于数据读取的低级别操作:

DynamoDB offers four low-level actions for data reading −

  1. GetItem − It accepts a primary key and returns attributes of the associated item. It permits changes to its default eventually consistent read setting.

  2. BatchGetItem − It executes several GetItem requests on multiple items through primary keys, with the option of one or multiple tables. Its returns no more than 100 items and must remain under 16MB. It permits eventually consistent and strongly consistent reads.

  3. Scan − It reads all the table items and produces an eventually consistent result set. You can filter results through conditions. It avoids the use of an index and scans the entire table, so do not use it for queries requiring predictability.

  4. Query − It returns a single or multiple table items or secondary index items. It uses a specified value for the partition key, and permits the use of comparison operators to narrow scope. It includes support for both types of consistency, and each response obeys a 1MB limit in size.

Modify Data

DynamoDB 为数据修改提供了四个底层操作 −

DynamoDB offers four low-level actions for data modification −

  1. PutItem − This spawns a new item or replaces existing items. On discovery of identical primary keys, by default, it replaces the item. Conditional operators allow you to work around the default, and only replace items under certain conditions.

  2. BatchWriteItem − This executes both multiple PutItem and DeleteItem requests, and over several tables. If one request fails, it does not impact the entire operation. Its cap sits at 25 items, and 16MB in size.

  3. UpdateItem − It changes the existing item attributes, and permits the use of conditional operators to execute updates only under certain conditions.

  4. DeleteItem − It uses the primary key to erase an item, and also allows the use of conditional operators to specify the conditions for deletion.

DynamoDB - Creating Items

在 DynamoDB 中创建项目主要包括项目和属性规范,以及指定条件的选项。每个项目都作为一个属性集存在,每个属性都命名并分配了特定类型的某个值。

Creating an item in DynamoDB consists primarily of item and attribute specification, and the option of specifying conditions. Each item exists as a set of attributes, with each attribute named and assigned a value of a certain type.

值类型包括标量、文档或集合。项目的尺寸限制为 400KB,但在该限制内可以容纳的属性数量不受限制。名称和值大小(二进制和 UTF-8 长度)决定项目大小。使用短属性名称有助于减小项目大小。

Value types include scalar, document, or set. Items carry a 400KB size limit, with the possibility of any amount of attributes capable of fitting within that limit. Name and value sizes (binary and UTF-8 lengths) determine item size. Using short attribute names aids in minimizing item size.

Note − 您必须指定所有主键属性,而主键只需分区键即可;复合键则需要分区键和排序键。

Note − You must specify all primary key attributes, with primary keys only requiring the partition key; and composite keys requiring both the partition and sort key.

另外,请记住表不拥有预定义架构。您可以在一张表中存储截然不同的数据集。

Also, remember tables possess no predefined schema. You can store dramatically different datasets in one table.

使用 GUI 控制台、Java 或其他工具来执行此任务。

Use the GUI console, Java, or another tool to perform this task.

How to Create an Item Using the GUI Console?

导航到控制台。在左侧导航窗格中,选择 Tables 。选择用作目标的表名,然后选择 Items 选项卡,如下面的屏幕截图所示。

Navigate to the console. In the navigation pane on the left side, select Tables. Choose the table name for use as the destination, and then select the Items tab as shown in the following screenshot.

create item

选择 Create Item 。创建项目屏幕提供了一个用于输入所需属性值的界面。还必须输入任何辅助索引。

Select Create Item. The Create Item screen provides an interface for entering the required attribute values. Any secondary indices must also be entered.

select create item

如果您需要更多属性,请选择 Message 左边的操作菜单。然后选择 Append 和所需的数据类型。

If you require more attributes, select the action menu on the left of the Message. Then select Append, and the desired data type.

message

在输入所有必要信息后,选择 Save 以添加项目。

After entering all essential information, select Save to add the item.

How to Use Java in Item Creation?

在项目创建操作中使用 Java 包括创建 DynamoDB 类实例、Table 类实例、Item 类实例,以及指定要创建的项目的主键和属性。然后使用 putItem 方法添加新项目。

Using Java in item creation operations consists of creating a DynamoDB class instance, Table class instance, Item class instance, and specifying the primary key and attributes of the item you will create. Then add your new item with the putItem method.

Example

DynamoDB dynamoDB = new DynamoDB (new AmazonDynamoDBClient(
   new ProfileCredentialsProvider()));
Table table = dynamoDB.getTable("ProductList");

// Spawn a related items list
List<Number> RELItems = new ArrayList<Number>();
RELItems.add(123);
RELItems.add(456);
RELItems.add(789);

//Spawn a product picture map
Map<String, String> photos = new HashMap<String, String>();
photos.put("Anterior", "http://xyz.com/products/101_front.jpg");
photos.put("Posterior", "http://xyz.com/products/101_back.jpg");
photos.put("Lateral", "http://xyz.com/products/101_LFTside.jpg");

//Spawn a product review map
Map<String, List<String>> prodReviews = new HashMap<String, List<String>>();
List<String> fiveStarRVW = new ArrayList<String>();
fiveStarRVW.add("Shocking high performance.");
fiveStarRVW.add("Unparalleled in its market.");
prodReviews.put("5 Star", fiveStarRVW);
List<String> oneStarRVW = new ArrayList<String>();
oneStarRVW.add("The worst offering in its market.");
prodReviews.put("1 Star", oneStarRVW);

// Generate the item
Item item = new Item()
   .withPrimaryKey("Id", 101)
   .withString("Nomenclature", "PolyBlaster 101")
   .withString("Description", "101 description")
   .withString("Category", "Hybrid Power Polymer Cutter")
   .withString("Make", "Brand – XYZ")
   .withNumber("Price", 50000)
   .withString("ProductCategory", "Laser Cutter")
   .withBoolean("Availability", true)
   .withNull("Qty")
   .withList("ItemsRelated", RELItems)
   .withMap("Images", photos)
   .withMap("Reviews", prodReviews);

// Add item to the table
PutItemOutcome outcome = table.putItem(item);

您还可以查看以下更大的示例。

You can also look at the following larger example.

Note − 以下示例可能假设之前创建过数据源。在尝试执行之前,获取支持库并创建必要的数据源(具有所需特性的表,或其他参考源)。

Note − The following sample may assume a previously created data source. Before attempting to execute, acquire supporting libraries and create necessary data sources (tables with required characteristics, or other referenced sources).

以下示例还使用 Eclipse IDE、AWS 凭证文件以及 Eclipse AWS Java 项目中的 AWS Toolkit。

The following sample also uses Eclipse IDE, an AWS credentials file, and the AWS Toolkit within an Eclipse AWS Java Project.

package com.amazonaws.codesamples.document;

import java.io.IOException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DeleteItemOutcome;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.Table;

import com.amazonaws.services.dynamodbv2.document.UpdateItemOutcome;
import com.amazonaws.services.dynamodbv2.document.spec.DeleteItemSpec;
import com.amazonaws.services.dynamodbv2.document.spec.UpdateItemSpec;
import com.amazonaws.services.dynamodbv2.document.utils.NameMap;
import com.amazonaws.services.dynamodbv2.document.utils.ValueMap;
import com.amazonaws.services.dynamodbv2.model.ReturnValue;

public class CreateItemOpSample {
   static DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient (
      new ProfileCredentialsProvider()));
   static String tblName = "ProductList";

   public static void main(String[] args) throws IOException {
      createItems();
      retrieveItem();

      // Execute updates
      updateMultipleAttributes();
      updateAddNewAttribute();
      updateExistingAttributeConditionally();

      // Item deletion
      deleteItem();
   }
   private static void createItems() {
      Table table = dynamoDB.getTable(tblName);
      try {
         Item item = new Item()
            .withPrimaryKey("ID", 303)
            .withString("Nomenclature", "Polymer Blaster 4000")
            .withStringSet( "Manufacturers",
            new HashSet<String>(Arrays.asList("XYZ Inc.", "LMNOP Inc.")))
            .withNumber("Price", 50000)
            .withBoolean("InProduction", true)
            .withString("Category", "Laser Cutter");

         table.putItem(item);
         item = new Item()
            .withPrimaryKey("ID", 313)
            .withString("Nomenclature", "Agitatatron 2000")
            .withStringSet( "Manufacturers",
            new HashSet<String>(Arrays.asList("XYZ Inc,", "CDE Inc.")))
            .withNumber("Price", 40000)
            .withBoolean("InProduction", true)
            .withString("Category", "Agitator");

         table.putItem(item);
      } catch (Exception e) {
         System.err.println("Cannot create items.");
         System.err.println(e.getMessage());
      }
   }
}

DynamoDB - Getting Items

在 DynamoDB 中检索项目需要使用 GetItem,并指定表名和项目主键。请务必包含一个完整的主键,而不是省略一部分。

Retrieving an item in DynamoDB requires using GetItem, and specifying the table name and item primary key. Be sure to include a complete primary key rather than omitting a portion.

例如,省略复合键的排序键。

For example, omitting the sort key of a composite key.

GetItem 行为符合三个默认值−

GetItem behaviour conforms to three defaults −

  1. It executes as an eventually consistent read.

  2. It provides all attributes.

  3. It does not detail its capacity unit consumption.

这些参数允许您覆盖默认的 GetItem 行为。

These parameters allow you to override the default GetItem behaviour.

Retrieve an Item

DynamoDB 通过在多个服务器上维护多个副本确保可靠性。每次成功的写操作都会创建这些副本,但执行需要大量时间;这意味着最终一致性。这意味着您不能在写一个条目后立即尝试读取。

DynamoDB ensures reliability through maintaining multiple copies of items across multiple servers. Each successful write creates these copies, but takes substantial time to execute; meaning eventually consistent. This means you cannot immediately attempt a read after writing an item.

然而,您可以更改 GetItem 的默认最终一致读取,但获取最新数据的成本仍然是消耗更多的预留单元;具体来说,消耗的资源是原来的两倍。请注意,DynamoDB 通常在一秒内使每个副本保持一致。

You can change the default eventually consistent read of GetItem, however, the cost of more current data remains consumption of more capacity units; specifically, two times as much. Note DynamoDB typically achieves consistency across every copy within a second.

您可以使用 GUI 控制台、Java 或其他工具执行此任务。

You can use the GUI console, Java, or another tool to perform this task.

Item Retrieval Using Java

在项目检索操作中使用 Java 需要创建一个 DynamoDB 类实例、表类实例,并调用表的 getItem 方法。然后指定项目的主键。

Using Java in item retrieval operations requires creating a DynamoDB Class Instance, Table Class Instance, and calling the Table instance’s getItem method. Then specify the primary key of the item.

您可以查看以下示例 -

You can review the following example −

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
   new ProfileCredentialsProvider()));
Table table = dynamoDB.getTable("ProductList");
Item item = table.getItem("IDnum", 109);

在某些情况下,您需要为此操作指定参数。

In some cases, you need to specify the parameters for this operation.

以下示例使用 .withProjectionExpressionGetItemSpec 作为检索规范−

The following example uses .withProjectionExpression and GetItemSpec for retrieval specifications −

GetItemSpec spec = new GetItemSpec()
   .withPrimaryKey("IDnum", 122)
   .withProjectionExpression("IDnum, EmployeeName, Department")
   .withConsistentRead(true);

Item item = table.getItem(spec);
System.out.println(item.toJSONPretty());

您还可以查看以下更大的示例以更好地理解。

You can also review a the following bigger example for better understanding.

Note − 以下示例可能假设之前创建过数据源。在尝试执行之前,获取支持库并创建必要的数据源(具有所需特性的表,或其他参考源)。

Note − The following sample may assume a previously created data source. Before attempting to execute, acquire supporting libraries and create necessary data sources (tables with required characteristics, or other referenced sources).

此示例还使用 Eclipse IDE、AWS 凭据文件和 Eclipse AWS Java 项目中的 AWS 工具包。

This sample also uses Eclipse IDE, an AWS credentials file, and the AWS Toolkit within an Eclipse AWS Java Project.

package com.amazonaws.codesamples.document;

import java.io.IOException
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DeleteItemOutcome;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.Table;

import com.amazonaws.services.dynamodbv2.document.UpdateItemOutcome;
import com.amazonaws.services.dynamodbv2.document.spec.DeleteItemSpec;
import com.amazonaws.services.dynamodbv2.document.spec.UpdateItemSpec;
import com.amazonaws.services.dynamodbv2.document.utils.NameMap;
import com.amazonaws.services.dynamodbv2.document.utils.ValueMap;
import com.amazonaws.services.dynamodbv2.model.ReturnValue;

public class GetItemOpSample {
   static DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
      new ProfileCredentialsProvider()));

   static String tblName = "ProductList";
   public static void main(String[] args) throws IOException {
      createItems();
      retrieveItem();

      // Execute updates
      updateMultipleAttributes();
      updateAddNewAttribute();
      updateExistingAttributeConditionally();

      // Item deletion
      deleteItem();
   }
   private static void createItems() {
      Table table = dynamoDB.getTable(tblName);
      try {
         Item item = new Item()
            .withPrimaryKey("ID", 303)
            .withString("Nomenclature", "Polymer Blaster 4000")
            .withStringSet( "Manufacturers",
            new HashSet<String>(Arrays.asList("XYZ Inc.", "LMNOP Inc.")))
            .withNumber("Price", 50000)
            .withBoolean("InProduction", true)
            .withString("Category", "Laser Cutter");
            table.putItem(item);

         item = new Item()
            .withPrimaryKey("ID", 313)
            .withString("Nomenclature", "Agitatatron 2000")
            .withStringSet( "Manufacturers",
            new HashSet<String>(Arrays.asList("XYZ Inc,", "CDE Inc.")))
            .withNumber("Price", 40000)
            .withBoolean("InProduction", true)
            .withString("Category", "Agitator");

         table.putItem(item);
      } catch (Exception e) {
         System.err.println("Cannot create items.");
         System.err.println(e.getMessage());
      }
   }
   private static void retrieveItem() {
      Table table = dynamoDB.getTable(tableName);
      try {
         Item item = table.getItem("ID", 303, "ID, Nomenclature, Manufacturers", null);
         System.out.println("Displaying retrieved items...");
         System.out.println(item.toJSONPretty());
      } catch (Exception e) {
         System.err.println("Cannot retrieve items.");
         System.err.println(e.getMessage());
      }
   }
}

DynamoDB - Update Items

在 DynamoDB 中更新一个项目主要包括为项目指定完整的主键和表名。您需要修改的每个属性都需要一个新值。该操作使用 UpdateItem ,它修改现有项目或在发现缺少项目时创建它们。

Updating an item in DynamoDB mainly consists of specifying the full primary key and table name for the item. It requires a new value for each attribute you modify. The operation uses UpdateItem, which modifies the existing items or creates them on discovery of a missing item.

更新中,您可能希望通过在操作之前和之后显示原始值和新值来跟踪更改。UpdateItem 使用 ReturnValues 参数来实现此目的。

In updates, you might want to track the changes by displaying the original and new values, before and after the operations. UpdateItem uses the ReturnValues parameter to achieve this.

Note − 该操作不报告预留单元消耗,但您可以使用 ReturnConsumedCapacity 参数。

Note − The operation does not report capacity unit consumption, but you can use the ReturnConsumedCapacity parameter.

使用 GUI 控制台、Java 或任何其他工具执行此任务。

Use the GUI console, Java, or any other tool to perform this task.

How to Update Items Using GUI Tools?

导航到控制台。在左侧的导航窗格中,选择 Tables 。选择所需的表,然后选择 Items 选项卡。

Navigate to the console. In the navigation pane on the left side, select Tables. Choose the table needed, and then select the Items tab.

update items using gui tools

选择希望更新的项目,然后选择 Actions | Edit

Choose the item desired for an update, and select Actions | Edit.

choose item

Edit Item 窗口中修改任何必要的属性或值。

Modify any attributes or values necessary in the Edit Item window.

Update Items Using Java

在项目更新操作中使用 Java 就需要创建一个表类实例,并调用其 updateItem 方法。然后,指定项目的键并提供详细的 UpdateExpression 属性修改。

Using Java in the item update operations requires creating a Table class instance, and calling its updateItem method. Then you specify the item’s primary key, and provide an UpdateExpression detailing attribute modifications.

以下是一个相同的示例 −

The Following is an example of the same −

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
   new ProfileCredentialsProvider()));

Table table = dynamoDB.getTable("ProductList");

Map<String, String> expressionAttributeNames = new HashMap<String, String>();
expressionAttributeNames.put("#M", "Make");
expressionAttributeNames.put("#P", "Price
expressionAttributeNames.put("#N", "ID");

Map<String, Object> expressionAttributeValues = new HashMap<String, Object>();
expressionAttributeValues.put(":val1",
   new HashSet<String>(Arrays.asList("Make1","Make2")));
expressionAttributeValues.put(":val2", 1);       //Price

UpdateItemOutcome outcome =  table.updateItem(
   "internalID",                                 // key attribute name
   111,                                          // key attribute value
   "add #M :val1 set #P = #P - :val2 remove #N", // UpdateExpression
   expressionAttributeNames,
   expressionAttributeValues);

updateItem 方法还允许指定条件,可以在以下示例中看到 −

The updateItem method also allows for specifying conditions, which can be seen in the following example −

Table table = dynamoDB.getTable("ProductList");
Map<String, String> expressionAttributeNames = new HashMap<String, String>();
expressionAttributeNames.put("#P", "Price");

Map<String, Object> expressionAttributeValues = new HashMap<String, Object>();
expressionAttributeValues.put(":val1", 44);  // change Price to 44
expressionAttributeValues.put(":val2", 15);  // only if currently 15

UpdateItemOutcome outcome = table.updateItem (new PrimaryKey("internalID",111),
   "set #P = :val1",                        // Update
   "#P = :val2",                            // Condition
   expressionAttributeNames,
   expressionAttributeValues);

Update Items Using Counters

DynamoDB 允许原子计数器,这意味着可以使用 UpdateItem 递增/递减属性值,而不会影响其他请求;此外,计数器总是更新。

DynamoDB allows atomic counters, which means using UpdateItem to increment/decrement attribute values without impacting other requests; furthermore, the counters always update.

以下是解释如何执行此操作的示例。

The following is an example that explains how it can be done.

Note − 以下示例可能假设之前创建过数据源。在尝试执行之前,获取支持库并创建必要的数据源(具有所需特性的表,或其他参考源)。

Note − The following sample may assume a previously created data source. Before attempting to execute, acquire supporting libraries and create necessary data sources (tables with required characteristics, or other referenced sources).

此示例还使用 Eclipse IDE、AWS 凭据文件和 Eclipse AWS Java 项目中的 AWS 工具包。

This sample also uses Eclipse IDE, an AWS credentials file, and the AWS Toolkit within an Eclipse AWS Java Project.

package com.amazonaws.codesamples.document;

import java.io.IOException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DeleteItemOutcome;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.Table;

import com.amazonaws.services.dynamodbv2.document.UpdateItemOutcome;
import com.amazonaws.services.dynamodbv2.document.spec.DeleteItemSpec;
import com.amazonaws.services.dynamodbv2.document.spec.UpdateItemSpec;
import com.amazonaws.services.dynamodbv2.document.utils.NameMap;
import com.amazonaws.services.dynamodbv2.document.utils.ValueMap;
import com.amazonaws.services.dynamodbv2.model.ReturnValue;

public class UpdateItemOpSample {
   static DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
      new ProfileCredentialsProvider()));
   static String tblName = "ProductList";

   public static void main(String[] args) throws IOException {
      createItems();
      retrieveItem();

      // Execute updates
      updateMultipleAttributes();
      updateAddNewAttribute();
      updateExistingAttributeConditionally();

      // Item deletion
      deleteItem();
   }
   private static void createItems() {
      Table table = dynamoDB.getTable(tblName);
      try {
         Item item = new Item()
            .withPrimaryKey("ID", 303)
            .withString("Nomenclature", "Polymer Blaster 4000")
            .withStringSet( "Manufacturers",
            new HashSet<String>(Arrays.asList("XYZ Inc.", "LMNOP Inc.")))
            .withNumber("Price", 50000)
            .withBoolean("InProduction", true)
            .withString("Category", "Laser Cutter");
            table.putItem(item);

         item = new Item()
            .withPrimaryKey("ID", 313)
            .withString("Nomenclature", "Agitatatron 2000")
            .withStringSet( "Manufacturers",
            new HashSet<String>(Arrays.asList("XYZ Inc,", "CDE Inc.")))
            .withNumber("Price", 40000)
            .withBoolean("InProduction", true)
            .withString("Category", "Agitator");
            table.putItem(item);
      } catch (Exception e) {
         System.err.println("Cannot create items.");
         System.err.println(e.getMessage());
      }
   }
   private static void updateAddNewAttribute() {
      Table table = dynamoDB.getTable(tableName);
      try {
         Map<String, String> expressionAttributeNames = new HashMap<String, String>();
         expressionAttributeNames.put("#na", "NewAttribute");
         UpdateItemSpec updateItemSpec = new UpdateItemSpec()
            .withPrimaryKey("ID", 303)
            .withUpdateExpression("set #na = :val1")
            .withNameMap(new NameMap()
            .with("#na", "NewAttribute"))
            .withValueMap(new ValueMap()
            .withString(":val1", "A value"))
            .withReturnValues(ReturnValue.ALL_NEW);
            UpdateItemOutcome outcome =  table.updateItem(updateItemSpec);

         // Confirm
         System.out.println("Displaying updated item...");
         System.out.println(outcome.getItem().toJSONPretty());
      } catch (Exception e) {
         System.err.println("Cannot add an attribute in " + tableName);
         System.err.println(e.getMessage());
      }
   }
}

DynamoDB - Delete Items

在 DynamoDB 中删除项目只需要提供表名和项目键。强烈建议使用条件表达式,这对于避免删除错误的项目是必要的。

Deleting an item in the DynamoDB only requires providing the table name and the item key. It is also strongly recommended to use of a conditional expression which will be necessary to avoid deleting the wrong items.

与往常一样,可以使用 GUI 控制台、Java 或任何其他必需的工具来执行此任务。

As usual, you can either use the GUI console, Java, or any other needed tool to perform this task.

Delete Items Using the GUI Console

导航至控制台。在左侧的导航窗格中,选择 Tables 。然后选择表名和 Items 选项卡。

Navigate to the console. In the navigation pane on the left side, select Tables. Then select the table name, and the Items tab.

delete items using gui console

选择要删除的项目并选择 Actions | Delete

Choose the items desired for deletion, and select Actions | Delete.

select actions

Delete Item(s) 对话框随后显示,如以下屏幕截图所示。选择“删除”进行确认。

A Delete Item(s) dialog box then appears as shown in the following screeshot. Choose “Delete” to confirm.

delete item

How to Delete Items Using Java?

在项目删除操作中使用 Java 只涉及创建 DynamoDB 客户端实例并使用项目的键来调用 deleteItem 方法。

Using Java in item deletion operations merely involves creating a DynamoDB client instance, and calling the deleteItem method through using the item’s key.

你可以查看以下示例,其中有详细的解释。

You can see the following example, where it has been explained in detail.

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
   new ProfileCredentialsProvider()));

Table table = dynamoDB.getTable("ProductList");
DeleteItemOutcome outcome = table.deleteItem("IDnum", 151);

你还可以指定参数来防止错误删除。只需使用 ConditionExpression

You can also specify the parameters to protect against incorrect deletion. Simply use a ConditionExpression.

例如 -

For example −

Map<String,Object> expressionAttributeValues = new HashMap<String,Object>();
expressionAttributeValues.put(":val", false);

DeleteItemOutcome outcome = table.deleteItem("IDnum",151,
   "Ship = :val",
   null,                   // doesn't use ExpressionAttributeNames
   expressionAttributeValues);

以下是一个更大的示例,以便更好地理解。

The following is a larger example for better understanding.

Note − 以下示例可能假设之前创建过数据源。在尝试执行之前,获取支持库并创建必要的数据源(具有所需特性的表,或其他参考源)。

Note − The following sample may assume a previously created data source. Before attempting to execute, acquire supporting libraries and create necessary data sources (tables with required characteristics, or other referenced sources).

此示例还使用 Eclipse IDE、AWS 凭据文件和 Eclipse AWS Java 项目中的 AWS 工具包。

This sample also uses Eclipse IDE, an AWS credentials file, and the AWS Toolkit within an Eclipse AWS Java Project.

package com.amazonaws.codesamples.document;

import java.io.IOException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DeleteItemOutcome;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.Table;

import com.amazonaws.services.dynamodbv2.document.UpdateItemOutcome;
import com.amazonaws.services.dynamodbv2.document.spec.DeleteItemSpec;
import com.amazonaws.services.dynamodbv2.document.spec.UpdateItemSpec;
import com.amazonaws.services.dynamodbv2.document.utils.NameMap;
import com.amazonaws.services.dynamodbv2.document.utils.ValueMap;
import com.amazonaws.services.dynamodbv2.model.ReturnValue;

public class DeleteItemOpSample {
   static DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
      new ProfileCredentialsProvider()));

   static String tblName = "ProductList";
   public static void main(String[] args) throws IOException {
      createItems();
      retrieveItem();

      // Execute updates
      updateMultipleAttributes();
      updateAddNewAttribute();
      updateExistingAttributeConditionally();

      // Item deletion
      deleteItem();
   }
   private static void createItems() {
      Table table = dynamoDB.getTable(tblName);
      try {
         Item item = new Item()
            .withPrimaryKey("ID", 303)
            .withString("Nomenclature", "Polymer Blaster 4000")
            .withStringSet( "Manufacturers",
            new HashSet<String>(Arrays.asList("XYZ Inc.", "LMNOP Inc.")))
            .withNumber("Price", 50000)
            .withBoolean("InProduction", true)
            .withString("Category", "Laser Cutter");
            table.putItem(item);

         item = new Item()
            .withPrimaryKey("ID", 313)
            .withString("Nomenclature", "Agitatatron 2000")
            .withStringSet( "Manufacturers",
            new HashSet<String>(Arrays.asList("XYZ Inc,", "CDE Inc.")))
            .withNumber("Price", 40000)
            .withBoolean("InProduction", true)
            .withString("Category", "Agitator");
            table.putItem(item);
      } catch (Exception e) {
         System.err.println("Cannot create items.");
         System.err.println(e.getMessage());
      }
   }
   private static void deleteItem() {
      Table table = dynamoDB.getTable(tableName);
      try {
         DeleteItemSpec deleteItemSpec = new DeleteItemSpec()
            .withPrimaryKey("ID", 303)
            .withConditionExpression("#ip = :val")
            .withNameMap(new NameMap()
            .with("#ip", "InProduction"))
            .withValueMap(new ValueMap()
            .withBoolean(":val", false))
            .withReturnValues(ReturnValue.ALL_OLD);
         DeleteItemOutcome outcome = table.deleteItem(deleteItemSpec);

         // Confirm
         System.out.println("Displaying deleted item...");
         System.out.println(outcome.getItem().toJSONPretty());
      } catch (Exception e) {
         System.err.println("Cannot delete item in " + tableName);
         System.err.println(e.getMessage());
      }
   }
}

DynamoDB - Batch Writing

批量写入操作通过创建或删除多个项目在多个项目上进行操作。这些操作使用 BatchWriteItem ,其限制不超过 16MB 的写入和 25 个请求。每个项目遵守 400KB 的大小限制。批量写入也不能执行项目更新。

Batch writing operates on multiple items by creating or deleting several items. These operations utilize BatchWriteItem, which carries the limitations of no more than 16MB writes and 25 requests. Each item obeys a 400KB size limit. Batch writes also cannot perform item updates.

What is Batch Writing?

批量写入可以在多个表中操作项目。操作调用发生在每个单独的请求中,这意味着操作不会相互影响,并且允许异构混合;例如,一个批处理中的一个 PutItem 和三个 DeleteItem 请求,其中 PutItem 请求的失败不会影响其他请求。失败的请求会导致操作返回与每个失败的请求有关的信息(键和数据)。

Batch writes can manipulate items across multiple tables. Operation invocation happens for each individual request, which means operations do not impact each other, and heterogeneous mixes are permitted; for example, one PutItem and three DeleteItem requests in a batch, with the failure of the PutItem request not impacting the others. Failed requests result in the operation returning information (keys and data) pertaining to each failed request.

Note − 如果 DynamoDB 返回任何未处理的项目,请重试它们;但是,请使用退避方法来避免由于过载而导致的另一个请求失败。

Note − If DynamoDB returns any items without processing them, retry them; however, use a back-off method to avoid another request failure based on overloading.

当以下一个或多个语句被证明为真时,DynamoDB 会拒绝批量写入操作 −

DynamoDB rejects a batch write operation when one or more of the following statements proves to be true −

  1. The request exceeds the provisioned throughput.

  2. The request attempts to use BatchWriteItems to update an item.

  3. The request performs several operations on a single item.

  4. The request tables do not exist.

  5. The item attributes in the request do not match the target.

  6. The requests exceed size limits.

批量写入需要某些 RequestItem 参数−

Batch writes require certain RequestItem parameters −

  1. Deletion operations need DeleteRequest key subelements meaning an attribute name and value.

  2. The PutRequest items require an Item subelement meaning an attribute and attribute value map.

Response −成功操作产生 HTTP 200 响应,它指示已消耗的容量单位、表处理指标和任何未处理的项目等特征。

Response − A successful operation results in an HTTP 200 response, which indicates characteristics like capacity units consumed, table processing metrics, and any unprocessed items.

Batch Writes with Java

通过创建一个 DynamoDB 类实例、一个描述所有操作的 TableWriteItems 类实例,并调用 batchWriteItem 方法来使用 TableWriteItems 对象来执行批量写入。

Perform a batch write by creating a DynamoDB class instance, a TableWriteItems class instance describing all operations, and calling the batchWriteItem method to use the TableWriteItems object.

Note −必须为批处理中的每一张表创建 TableWriteItems 实例。另外,检查请求响应以了解任何未处理的请求。

Note − You must create a TableWriteItems instance for every table in a batch write to multiple tables. Also, check your request response for any unprocessed requests.

您可以查看以下批量写入示例−

You can review the following example of a batch write −

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
   new ProfileCredentialsProvider()));

TableWriteItems forumTableWriteItems = new TableWriteItems("Forum")
   .withItemsToPut(
   new Item()
   .withPrimaryKey("Title", "XYZ CRM")
   .withNumber("Threads", 0));

TableWriteItems threadTableWriteItems = new TableWriteItems(Thread)
   .withItemsToPut(
   new Item()
   .withPrimaryKey("ForumTitle","XYZ CRM","Topic","Updates")
   .withHashAndRangeKeysToDelete("ForumTitle","A partition key value",
   "Product Line 1", "A sort key value"));

BatchWriteItemOutcome outcome = dynamoDB.batchWriteItem (
   forumTableWriteItems, threadTableWriteItems);

以下程序是另一个较大的示例,用于更好地理解 Java 中如何进行批量写入。

The following program is another bigger example for better understanding of how a batch writes with Java.

Note −以下示例可能假定已经创建了数据源。在尝试执行之前,获取支持库并创建必要的数据源(具有所需特征的表或其他引用源)。

Note − The following example may assume a previously created data source. Before attempting to execute, acquire supporting libraries and create necessary data sources (tables with required characteristics, or other referenced sources).

此示例还使用了 Eclipse IDE、AWS 凭据文件和 Eclipse AWS Java 项目中的 AWS Toolkit。

This example also uses Eclipse IDE, an AWS credentials file, and the AWS Toolkit within an Eclipse AWS Java Project.

package com.amazonaws.codesamples.document;

import java.io.IOException;
import java.util.Arrays;
import java.util.HashSet;
import java.util.List;
import java.util.Map;

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.BatchWriteItemOutcome;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.TableWriteItems;
import com.amazonaws.services.dynamodbv2.model.WriteRequest;

public class BatchWriteOpSample {
   static DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
      new ProfileCredentialsProvider()));
   static String forumTableName = "Forum";
   static String threadTableName = "Thread";

   public static void main(String[] args) throws IOException {
      batchWriteMultiItems();
   }
   private static void batchWriteMultiItems() {
      try {
         // Place new item in Forum
         TableWriteItems forumTableWriteItems = new TableWriteItems(forumTableName)
                                                                       //Forum
            .withItemsToPut(new Item()
            .withPrimaryKey("Name", "Amazon RDS")
            .withNumber("Threads", 0));

         // Place one item, delete another in Thread
         // Specify partition key and range key
         TableWriteItems threadTableWriteItems = new TableWriteItems(threadTableName)
            .withItemsToPut(new Item()
            .withPrimaryKey("ForumName","Product
            Support","Subject","Support Thread 1")
            .withString("Message", "New OS Thread 1 message")
            .withHashAndRangeKeysToDelete("ForumName","Subject", "Polymer Blaster",
            "Support Thread 100"));

         System.out.println("Processing request...");
         BatchWriteItemOutcome outcome = dynamoDB.batchWriteItem (
               forumTableWriteItems, threadTableWriteItems);
         do {
            // Confirm no unprocessed items
            Map<String, List<WriteRequest>> unprocessedItems
               = outcome.getUnprocessedItems();

            if (outcome.getUnprocessedItems().size() == 0) {
               System.out.println("All items processed.");
            } else {
               System.out.println("Gathering unprocessed items...");
               outcome = dynamoDB.batchWriteItemUnprocessed(unprocessedItems);
            }
         } while (outcome.getUnprocessedItems().size() > 0);
      } catch (Exception e) {
         System.err.println("Could not get items: ");
         e.printStackTrace(System.err);
      }
   }
}

DynamoDB - Batch Retrieve

批量获取操作返回单个或多个项目的属性。这些操作通常包括使用主键来识别所需项目。 BatchGetItem 操作受限于个别操作及其自身独特的约束。

Batch Retrieve operations return attributes of a single or multiple items. These operations generally consist of using the primary key to identify the desired item(s). The BatchGetItem operations are subject to the limits of individual operations as well as their own unique constraints.

批量获取操作中的以下请求导致拒绝−

The following requests in batch retrieval operations result in rejection −

  1. Make a request for more than 100 items.

  2. Make a request exceeding throughput.

批量检索操作可对可能会超出限制的请求进行部分处理。

Batch retrieve operations perform partial processing of requests carrying the potential to exceed limits.

For example − 检索多个项目大小足以超出限制的请求会部分处理该请求,并显示一个错误消息,注明未处理部分。在返还未处理项目时,要创建回退算法解决方案来管理此问题,而不是限制表格。

For example − a request to retrieve multiple items large enough in size to exceed limits results in part of the request processing, and an error message noting the unprocessed portion. On return of unprocessed items, create a back-off algorithm solution to manage this rather than throttling tables.

BatchGet 操作最终使用一致读取进行,需要对强一致读取进行修改。它们还并行执行检索。

The BatchGet operations perform eventually with consistent reads, requiring modification for strongly consistent ones. They also perform retrievals in parallel.

Note − 返还项目的顺序。DynamoDB 不会对项目进行排序。它也不会指示所请求项目缺失。此外,这些请求会使用容量单位。

Note − The order of the returned items. DynamoDB does not sort the items. It also does not indicate the absence of the requested items. Furthermore, those requests consume capacity units.

所有 BatchGet 操作都需要 RequestItems 参数,例如读取一致性、属性名称和主键。

All the BatchGet operations require RequestItems parameters such as the read consistency, attribute names, and primary keys.

Response −成功操作产生 HTTP 200 响应,它指示已消耗的容量单位、表处理指标和任何未处理的项目等特征。

Response − A successful operation results in an HTTP 200 response, which indicates characteristics like capacity units consumed, table processing metrics, and any unprocessed items.

Batch Retrievals with Java

在 BatchGet 操作中使用 Java 要求创建一个 DynamoDB 类实例、 TableKeysAndAttributes 类实例(描述项目的哈希值列表),然后将 TableKeysAndAttributes 对象传递给 BatchGetItem 方法。

Using Java in BatchGet operations requires creating a DynamoDB class instance, TableKeysAndAttributes class instance describing a primary key values list for the items, and passing the TableKeysAndAttributes object to the BatchGetItem method.

以下是 BatchGet 操作的一个示例 −

The following is an example of a BatchGet operation −

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient (
   new ProfileCredentialsProvider()));

TableKeysAndAttributes forumTableKeysAndAttributes = new TableKeysAndAttributes
   (forumTableName);

forumTableKeysAndAttributes.addHashOnlyPrimaryKeys (
   "Title",
   "Updates",
   "Product Line 1"
);
TableKeysAndAttributes threadTableKeysAndAttributes = new TableKeysAndAttributes (
   threadTableName);

threadTableKeysAndAttributes.addHashAndRangePrimaryKeys (
   "ForumTitle",
   "Topic",
   "Product Line 1",
   "P1 Thread 1",
   "Product Line 1",
   "P1 Thread 2",
   "Product Line 2",
   "P2 Thread 1"
);
BatchGetItemOutcome outcome = dynamoDB.batchGetItem (
   forumTableKeysAndAttributes, threadTableKeysAndAttributes);

for (String tableName : outcome.getTableItems().keySet()) {
   System.out.println("Table items " + tableName);
   List<Item> items = outcome.getTableItems().get(tableName);
   for (Item item : items) {
      System.out.println(item);
   }
}

你可以查阅以下更全面的示例。

You can review the following larger example.

Note − 以下程序可能会假设有一个已创建的数据源。在尝试执行之前,获得支持库并创建必要的数据源(具备所需特征的表格或其他引用的源)。

Note − The following program may assume a previously created data source. Before attempting to execute, acquire supporting libraries and create necessary data sources (tables with required characteristics, or other referenced sources).

此程序还使用 Eclipse IDE、AWS 凭证文件以及 Eclipse AWS Java 项目中的 AWS 工具包。

This program also uses Eclipse IDE, an AWS credentials file, and the AWS Toolkit within an Eclipse AWS Java Project.

package com.amazonaws.codesamples.document;

import java.io.IOException;
import java.util.List;
import java.util.Map;

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.BatchGetItemOutcome;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.TableKeysAndAttributes;
import com.amazonaws.services.dynamodbv2.model.KeysAndAttributes;

public class BatchGetOpSample {
   static DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient (
      new ProfileCredentialsProvider()));

   static String forumTableName = "Forum";
   static String threadTableName = "Thread";

   public static void main(String[] args) throws IOException {
      retrieveMultipleItemsBatchGet();
   }
   private static void retrieveMultipleItemsBatchGet() {
      try {
         TableKeysAndAttributes forumTableKeysAndAttributes =
            new TableKeysAndAttributes(forumTableName);

         //Create partition key
         forumTableKeysAndAttributes.addHashOnlyPrimaryKeys (
            "Name",
            "XYZ Melt-O-tron",
            "High-Performance Processing"
         );
         TableKeysAndAttributes threadTableKeysAndAttributes =
            new TableKeysAndAttributes(threadTableName);

         //Create partition key and sort key
         threadTableKeysAndAttributes.addHashAndRangePrimaryKeys (
            "ForumName",
            "Subject",
            "High-Performance Processing",
            "HP Processing Thread One",
            "High-Performance Processing",
            "HP Processing Thread Two",
            "Melt-O-Tron",
            "MeltO Thread One"
         );
         System.out.println("Processing...");
         BatchGetItemOutcome outcome = dynamoDB.batchGetItem(forumTableKeysAndAttributes,
            threadTableKeysAndAttributes);

         Map<String, KeysAndAttributes> unprocessed = null;
         do {
            for (String tableName : outcome.getTableItems().keySet()) {
               System.out.println("Table items for " + tableName);
               List<Item> items = outcome.getTableItems().get(tableName);

               for (Item item : items) {
                  System.out.println(item.toJSONPretty());
               }
            }
            // Confirm no unprocessed items
            unprocessed = outcome.getUnprocessedKeys();

            if (unprocessed.isEmpty()) {
               System.out.println("All items processed.");
            } else {
               System.out.println("Gathering unprocessed items...");
               outcome = dynamoDB.batchGetItemUnprocessed(unprocessed);
            }
         } while (!unprocessed.isEmpty());
      } catch (Exception e) {
         System.err.println("Could not get items.");
         System.err.println(e.getMessage());
      }
   }
}

DynamoDB - Querying

查询会通过主键查找项目或辅助索引。执行查询要求使用分区键和特定值,或排序键和值;可以选择通过比较进行筛选。查询的默认执行过程是针对与提供的主键关联的项目返回每个属性。但你可以使用 ProjectionExpression 参数指定所需的属性。

Queries locate items or secondary indices through primary keys. Performing a query requires a partition key and specific value, or a sort key and value; with the option to filter with comparisons. The default behavior of a query consists of returning every attribute for items associated with the provided primary key. However, you can specify the desired attributes with the ProjectionExpression parameter.

查询使用 KeyConditionExpression 参数来选择项目,这要求以相等条件的形式提供分区键名称和值。你还可以选择为任何存在的排序键提供其他条件。

A query utilizes the KeyConditionExpression parameters to select items, which requires providing the partition key name and value in the form of an equality condition. You also have the option to provide an additional condition for any sort keys present.

下面列出了排序键条件的几个示例 −

A few examples of the sort key conditions are −

Sr.No

Condition & Description

1

x = y It evaluates to true if the attribute x equals y.

2

x < y It evaluates to true if x is less than y.

3

x ⇐ y It evaluates to true if x is less than or equal to y.

4

x > y It evaluates to true if x is greater than y.

5

x >= y It evaluates to true if x is greater than or equal to y.

6

x BETWEEN y AND z It evaluates to true if x is both >= y, and ⇐ z.

DynamoDB 还支持以下函数: begins_with (x, substr)

DynamoDB also supports the following functions: begins_with (x, substr)

如果属性 x 以指定字符串开头,则求值为真。

It evaluates to true if attribute x starts with the specified string.

以下条件必须符合某些要求 −

The following conditions must conform to certain requirements −

  1. Attribute names must start with a character within the a-z or A-Z set.

  2. The second character of an attribute name must fall in the a-z, A-Z, or 0-9 set.

  3. Attribute names cannot use reserved words.

不符合以上约束的属性名可以定义占位符。

Attribute names out of compliance with the constraints above can define a placeholder.

查询通过按排序键顺序执行检索,并使用任何条件和筛选器表达式来处理。查询总是返回一个结果集,如果没有匹配项,则返回一个空集。

The query processes by performing retrievals in sort key order, and using any condition and filter expressions present. Queries always return a result set, and on no matches, it returns an empty one.

结果始终按排序键顺序返回,并且基于数据类型顺序,修改后的默认值为升序。

The results always return in sort key order, and data type based order with the modifiable default as the ascending order.

Querying with Java

Java 中的查询允许您查询表和二级索引。它们要求指定分区键和相等条件,并可以选择指定排序键和条件。

Queries in Java allow you to query tables and secondary indices. They require specification of partition keys and equality conditions, with the option to specify sort keys and conditions.

在 Java 中进行查询的常规必需步骤包括:创建 DynamoDB 类实例、目标表的 Table 类实例,并调用 Table 实例的 query 方法以接收查询对象。

The general required steps for a query in Java include creating a DynamoDB class instance, Table class instance for the target table, and calling the query method of the Table instance to receive the query object.

对查询的响应包含一个 ItemCollection 对象,该对象提供所有返回的项目。

The response to the query contains an ItemCollection object providing all the returned items.

以下示例演示了详细查询 −

The following example demonstrates detailed querying −

DynamoDB dynamoDB = new DynamoDB (
   new AmazonDynamoDBClient(new ProfileCredentialsProvider()));

Table table = dynamoDB.getTable("Response");
   QuerySpec spec = new QuerySpec()
   .withKeyConditionExpression("ID = :nn")
.withValueMap(new ValueMap()
   .withString(":nn", "Product Line 1#P1 Thread 1"));

ItemCollection<QueryOutcome> items = table.query(spec);
Iterator<Item> iterator = items.iterator();
Item item = null;

while (iterator.hasNext()) {
   item = iterator.next();
   System.out.println(item.toJSONPretty());
}

查询方法支持广泛的可选参数。以下示例演示如何利用这些参数 −

The query method supports a wide variety of optional parameters. The following example demonstrates how to utilize these parameters −

Table table = dynamoDB.getTable("Response");
QuerySpec spec = new QuerySpec()
   .withKeyConditionExpression("ID = :nn and ResponseTM > :nn_responseTM")
   .withFilterExpression("Author = :nn_author")
   .withValueMap(new ValueMap()
   .withString(":nn", "Product Line 1#P1 Thread 1")
   .withString(":nn_responseTM", twoWeeksAgoStr)
   .withString(":nn_author", "Member 123"))
   .withConsistentRead(true);

ItemCollection<QueryOutcome> items = table.query(spec);
Iterator<Item> iterator = items.iterator();

while (iterator.hasNext()) {
   System.out.println(iterator.next().toJSONPretty());
}

您还可以查看以下更大的示例。

You can also review the following larger example.

Note − 以下程序可能会假设有一个已创建的数据源。在尝试执行之前,获得支持库并创建必要的数据源(具备所需特征的表格或其他引用的源)。

Note − The following program may assume a previously created data source. Before attempting to execute, acquire supporting libraries and create necessary data sources (tables with required characteristics, or other referenced sources).

此示例还使用了 Eclipse IDE、AWS 凭据文件和 Eclipse AWS Java 项目中的 AWS Toolkit。

This example also uses Eclipse IDE, an AWS credentials file, and the AWS Toolkit within an Eclipse AWS Java Project.

package com.amazonaws.codesamples.document;

import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Iterator;

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.ItemCollection;

import com.amazonaws.services.dynamodbv2.document.Page;
import com.amazonaws.services.dynamodbv2.document.QueryOutcome;
import com.amazonaws.services.dynamodbv2.document.Table;
import com.amazonaws.services.dynamodbv2.document.spec.QuerySpec;
import com.amazonaws.services.dynamodbv2.document.utils.ValueMap;

public class QueryOpSample {
   static DynamoDB dynamoDB = new DynamoDB(
      new AmazonDynamoDBClient(new ProfileCredentialsProvider()));
   static String tableName = "Reply";

   public static void main(String[] args) throws Exception {
      String forumName = "PolyBlaster";
      String threadSubject = "PolyBlaster Thread 1";
      getThreadReplies(forumName, threadSubject);
   }
   private static void getThreadReplies(String forumName, String threadSubject) {
      Table table = dynamoDB.getTable(tableName);
      String replyId = forumName + "#" + threadSubject;
      QuerySpec spec = new QuerySpec()
         .withKeyConditionExpression("Id = :v_id")
         .withValueMap(new ValueMap()
         .withString(":v_id", replyId));

      ItemCollection<QueryOutcome> items = table.query(spec);
      System.out.println("\ngetThreadReplies results:");
      Iterator<Item> iterator = items.iterator();

      while (iterator.hasNext()) {
         System.out.println(iterator.next().toJSONPretty());
      }
   }
}

DynamoDB - Scan

扫描操作会读取所有表项或二级索引。其默认功能会返回索引或表中所有项目的全部数据属性。使用 ProjectionExpression 参数对属性进行筛选。

Scan Operations read all table items or secondary indices. Its default function results in returning all data attributes of all items within an index or table. Employ the ProjectionExpression parameter in filtering attributes.

每次扫描都会返回一个结果集,即使没有找到匹配项,也会返回一个空集。扫描检索不到 1MB 的数据,并可以选择对数据进行筛选。

Every scan returns a result set, even on finding no matches, which results in an empty set. Scans retrieve no more than 1MB, with the option to filter data.

Note − 扫描的参数和筛选也适用于查询。

Note − The parameters and filtering of scans also apply to querying.

Types of Scan Operations

Filtering − 扫描操作通过过滤器表达式提供了精细筛选功能,该表达式在扫描或查询后修改数据,在返回结果之前。表达式使用比较运算符。它们的语法与条件表达式的语法类似,但键属性除外,因为过滤器表达式不允许键属性。你不能在过滤器表达式中使用分区键或排序键。

Filtering − Scan operations offer fine filtering through filter expressions, which modify data after scans, or queries; before returning results. The expressions use comparison operators. Their syntax resembles condition expressions with the exception of key attributes, which filter expressions do not permit. You cannot use a partition or sort key in a filter expression.

Note − 在应用任何过滤之前,都适用于 1MB 限制。

Note − The 1MB limit applies prior to any application of filtering.

Throughput Specifications − 扫描会消耗吞吐量,然而,消耗主要关注于项目大小,而不是返回的数据。无论你请求所有属性还是仅仅请求几个属性,消耗都保持不变。另外,使用或不使用过滤器表达式也不会影响消耗。

Throughput Specifications − Scans consume throughput, however, consumption focuses on item size rather than returned data. The consumption remains the same whether you request every attribute or only a few, and using or not using a filter expression also does not impact consumption.

Pagination − DynamoDB 对结果进行分页,从而将结果分成特定页面。1MB 限制适用于返回的结果,当你超过它时,需要进行另一项扫描才能收集其余数据。 LastEvaluatedKey 值允许你执行此后续扫描。只需将该值应用到 ExclusiveStartkey 。当 LastEvaluatedKey 值变为 null 时,该操作已完成所有数据页面。不过,非 null 值并不自动表示还存在更多数据。只有 null 值才表示状态。

Pagination − DynamoDB paginates results causing division of results into specific pages. The 1MB limit applies to returned results, and when you exceed it, another scan becomes necessary to gather the rest of the data. The LastEvaluatedKey value allows you to perform this subsequent scan. Simply apply the value to the ExclusiveStartkey. When the LastEvaluatedKey value becomes null, the operation has completed all pages of data. However, a non-null value does not automatically mean more data remains. Only a null value indicates status.

The Limit Parameter − 限制参数管理结果大小。DynamoDB 使用它来确定在返回数据之前处理的项目数,并且不适用于范围之外。如果你设置值为 x,DynamoDB 将返回前 x 个匹配项。

The Limit Parameter − The limit parameter manages the result size. DynamoDB uses it to establish the number of items to process before returning data, and does not work outside of the scope. If you set a value of x, DynamoDB returns the first x matching items.

在限制参数产生部分结果的情况下,LastEvaluatedKey 值也适用。使用它来完成扫描。

The LastEvaluatedKey value also applies in cases of limit parameters yielding partial results. Use it to complete scans.

Result Count − 对查询和扫描的响应还包括与 ScannedCount 和计数相关的信息,这些信息对扫描/查询的项目进行量化,并对返回的项目进行量化。如果你不进行过滤,它们的值是相同的。当你超过 1MB 时,计数仅表示已处理的部分。

Result Count − Responses to queries and scans also include information related to ScannedCount and Count, which quantify scanned/queried items and quantify items returned. If you do not filter, their values are identical. When you exceed 1MB, the counts represent only the portion processed.

Consistency − 查询结果和扫描结果最终是一致的读取结果,然而,你也可以设置强一致性读取。使用 ConsistentRead 参数更改此设置。

Consistency − Query results and scan results are eventually consistent reads, however, you can set strongly consistent reads as well. Use the ConsistentRead parameter to change this setting.

Note − 一致读取设置会影响消耗,因为在设为强一致性时会使用两倍的容量单位。

Note − Consistent read settings impact consumption by using double the capacity units when set to strongly consistent.

Performance − 查询比扫描提供更好的性能,因为扫描遍历了整个表或辅助索引,从而导致响应迟缓并大量消耗吞吐量。对于小表和过滤较少的搜索,扫描效果最佳。不过,你可以遵循一些最佳实践来设计精简扫描,例如避免突然加速读取活动并利用并行扫描。

Performance − Queries offer better performance than scans due to scans crawling the full table or secondary index, resulting in a sluggish response and heavy throughput consumption. Scans work best for small tables and searches with less filters, however, you can design lean scans by obeying a few best practices such as avoiding sudden, accelerated read activity and exploiting parallel scans.

查询根据给定条件查找某个键范围,其性能取决于它检索的数据量,而不是键的数量。操作参数和匹配项的数量会明确影响性能。

A query finds a certain range of keys satisfying a given condition, with performance dictated by the amount of data it retrieves rather than the volume of keys. The parameters of the operation and the number of matches specifically impact performance.

Parallel Scan

默认情况下,扫描操作顺序执行处理。然后,它们以 1MB 的部分返回数据,这会提示应用程序获取下一部分数据。对于大型表和索引,这会导致长时间扫描。

Scan operations perform processing sequentially by default. Then they return data in 1MB portions, which prompts the application to fetch the next portion. This results in long scans for large tables and indices.

此特性还意味着扫描可能并不总是充分利用可用吞吐量。DynamoDB 将表数据分布到多个分区上,而扫描吞吐量由于它的单分区操作而仅限于一个分区。

This characteristic also means scans may not always fully exploit the available throughput. DynamoDB distributes table data across multiple partitions; and scan throughput remains limited to a single partition due to its single-partition operation.

针对此问题的一个解决方案是从逻辑上将表或索引分成段。然后,“工作程序”并行(同时)扫描段。它使用段和 TotalSegments 的参数来指定由特定工作程序扫描的段并指定处理的段的总数。

A solution for this problem comes from logically dividing tables or indices into segments. Then “workers” parallel (concurrently) scan segments. It uses the parameters of Segment and TotalSegments to specify segments scanned by certain workers and specify the total quantity of segments processed.

Worker Number

你必须尝试不同的工作程序值(段参数)以实现最佳应用程序性能。

You must experiment with worker values (Segment parameter) to achieve the best application performance.

Note − 拥有大量工作程序的并行扫描会影响吞吐量,可能消耗所有吞吐量。通过限制参数管理此问题,你可以使用该参数阻止单个工作程序消耗所有吞吐量。

Note − Parallel scans with large sets of workers impacts throughput by possibly consuming all throughput. Manage this issue with the Limit parameter, which you can use to stop a single worker from consuming all throughput.

以下是深度扫描的示例。

The following is a deep scan example.

Note − 以下程序可能会假设有一个已创建的数据源。在尝试执行之前,获得支持库并创建必要的数据源(具备所需特征的表格或其他引用的源)。

Note − The following program may assume a previously created data source. Before attempting to execute, acquire supporting libraries and create necessary data sources (tables with required characteristics, or other referenced sources).

此示例还使用了 Eclipse IDE、AWS 凭据文件和 Eclipse AWS Java 项目中的 AWS Toolkit。

This example also uses Eclipse IDE, an AWS credentials file, and the AWS Toolkit within an Eclipse AWS Java Project.

import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.ItemCollection;
import com.amazonaws.services.dynamodbv2.document.ScanOutcome;
import com.amazonaws.services.dynamodbv2.document.Table;

public class ScanOpSample {
   static DynamoDB dynamoDB = new DynamoDB(
      new AmazonDynamoDBClient(new ProfileCredentialsProvider()));
   static String tableName = "ProductList";

   public static void main(String[] args) throws Exception {
      findProductsUnderOneHun();                       //finds products under 100 dollars
   }
   private static void findProductsUnderOneHun() {
      Table table = dynamoDB.getTable(tableName);
      Map<String, Object> expressionAttributeValues = new HashMap<String, Object>();
      expressionAttributeValues.put(":pr", 100);

      ItemCollection<ScanOutcome> items = table.scan (
         "Price < :pr",                                  //FilterExpression
         "ID, Nomenclature, ProductCategory, Price",     //ProjectionExpression
         null,                                           //No ExpressionAttributeNames
         expressionAttributeValues);

      System.out.println("Scanned " + tableName + " to find items under $100.");
      Iterator<Item> iterator = items.iterator();

      while (iterator.hasNext()) {
         System.out.println(iterator.next().toJSONPretty());
      }
   }
}

DynamoDB - Indexes

DynamoDB 使用主键属性的索引来改进访问。它们加速了应用程序访问和数据检索,并且通过减少应用程序滞后支持更好的性能。

DynamoDB uses indexes for primary key attributes to improve accesses. They accelerate application accesses and data retrieval, and support better performance by reducing application lag.

Secondary Index

辅助索引包含属性子集和备用键。你可以通过查询或扫描操作(以索引为目标)使用它。

A secondary index holds an attribute subset and an alternate key. You use it through either a query or scan operation, which targets the index.

其内容包括您映射或复制的属性。在创建时,您为索引定义另一个键以及任何您希望在索引中映射的属性。然后,DynamoDB 将这些属性复制到索引中,包括源自表的主键属性。执行这些任务后,您只需使用查询/扫描,如同对表执行操作一样即可。

Its contents include attributes you project or copy. In creation, you define an alternate key for the index, and any attributes you wish to project in the index. DynamoDB then performs a copy of the attributes into the index, including primary key attributes sourced from the table. After performing these tasks, you simply use a query/scan as if performing on a table.

DynamoDB 自动维护所有二级索引。对于诸如添加或删除这样的条目操作,它会更新目标表上的任何索引。

DynamoDB automatically maintains all secondary indices. On item operations, such as adding or deleting, it updates any indexes on the target table.

DynamoDB 提供两种类型的二级索引 -

DynamoDB offers two types of secondary indexes −

  1. Global Secondary Index − This index includes a partition key and sort key, which may differ from the source table. It uses the label “global” due to the capability of queries/scans on the index to span all table data, and over all partitions.

  2. Local Secondary Index − This index shares a partition key with the table, but uses a different sort key. Its “local” nature results from all of its partitions scoping to a table partition with identical partition key value.

使用的最佳索引类型取决于应用程序的需要。思考表中呈现的两种索引之间的以下差异 -

The best type of index to use depends on application needs. Consider the differences between the two presented in the following table −

Quality

Global Secondary Index

Local Secondary Index

Key Schema

It uses a simple or composite primary key.

It always uses a composite primary key.

Key Attributes

The index partition key and sort key can consist of string, number, or binary table attributes.

The partition key of the index is an attribute shared with the table partition key. The sort key can be string, number, or binary table attributes.

Size Limits Per Partition Key Value

They carry no size limitations.

It imposes a 10GB maximum limit on total size of indexed items associated with a partition key value.

Online Index Operations

You can spawn them at table creation, add them to existing tables, or delete existing ones.

You must create them at table creation, but cannot delete them or add them to existing tables.

Queries

It allows queries covering the entire table, and every partition.

They address single partitions through the partition key value provided in the query.

Consistency

Queries of these indices only offer the eventually consistent option.

Queries of these offer the options of eventually consistent or strongly consistent.

Throughput Cost

It includes throughput settings for reads and writes. Queries/scans consume capacity from the index, not the table, which also applies to table write updates.

Queries/scans consume table read capacity. Table writes update local indexes, and consume table capacity units.

Projection

Queries/scans can only request attributes projected into the index, with no retrievals of table attributes.

Queries/scans can request those attributes not projected; furthermore, automatic fetches of them occur.

使用辅助索引创建多个表时,按顺序创建;也就是说,先创建一个表,然后等待它变为 ACTIVE 状态后,再创建另一个并再次等待。DynamoDB 不允许并发创建。

When creating multiple tables with secondary indexes, do it sequentially; meaning make a table and wait for it to reach ACTIVE state before creating another and again waiting. DynamoDB does not permit concurrent creation.

每个辅助索引都需要某些规范 −

Each secondary index requires certain specifications −

  1. Type − Specify local or global.

  2. Name − It uses naming rules identical to tables.

  3. Key Schema − Only top level string, number, or binary type are permitted, with index type determining other requirements.

  4. Attributes for Projection − DynamoDB automatically projects them, and allows any data type.

  5. Throughput − Specify read/write capacity for global secondary indexes.

索引的限制仍然是每个表 5 个全局索引和 5 个本地索引。

The limit for indexes remains 5 global and 5 local per table.

你可以使用 DescribeTable 访问有关索引的详细信息。它会返回名称、大小和项目计数。

You can access the detailed information about indexes with DescribeTable. It returns the name, size, and item count.

Note − 这些值每 6 小时更新一次。

Note − These values updates every 6 hours.

在用于访问索引数据的查询或扫描中,提供表和索引名称、结果所需的属性以及任何条件语句。DynamoDB 提供了以升序或降序返回结果的选项。

In queries or scans used to access index data, provide the table and index names, desired attributes for the result, and any conditional statements. DynamoDB offers the option to return results in either ascending or descending order.

Note − 删除表也会删除所有索引。

Note − The deletion of a table also deletes all indexes.

DynamoDB - Global Secondary Indexes

需要使用不同属性执行各种查询类型的应用程序可以在执行这些详细查询时使用一个或多个全局辅助索引。

Applications requiring various query types with different attributes can use a single or multiple global secondary indexes in performing these detailed queries.

For example − 一个跟踪用户、他们的登录状态以及他们登录时间的系统。上一个例子的增长会减慢对其数据的查询速度。

For example − A system keeping a track of users, their login status, and their time logged in. The growth of the previous example slows queries on its data.

全局辅助索引通过按表中的特定属性组织数据来加速查询。它们使用主键对数据进行排序,不需要密钥表属性或与表相同的密钥架构。

Global secondary indexes accelerate queries by organizing a selection of attributes from a table. They employ primary keys in sorting data, and require no key table attributes, or key schema identical to the table.

所有全局二级索引都必须包含一个分区键,并可以包含一个排序键。索引键模式可以不同于表,索引键属性可以使用任何顶级字符串、数字或二进制表属性。

All the global secondary indexes must include a partition key, with the option of a sort key. The index key schema can differ from the table, and index key attributes can use any top-level string, number, or binary table attributes.

在投影中,您可以使用其他表属性,但是查询不会从父表中检索。

In a projection, you can use other table attributes, however, queries do not retrieve from parent tables.

Attribute Projections

投影包含从表复制到二级索引的一组属性。投影始终与表分区键和排序键一起发生。在查询中,投影允许 DynamoDB 访问投影的任何属性;它们本质上作为自己的表而存在。

Projections consist of an attribute set copied from table to secondary index. A Projection always occurs with the table partition key and sort key. In queries, projections allow DynamoDB access to any attribute of the projection; they essentially exist as their own table.

在二级索引创建中,您必须指定要投影的属性。DynamoDB 提供了三种执行此任务的方法−

In a secondary index creation, you must specify attributes for projection. DynamoDB offers three ways to perform this task −

  1. KEYS_ONLY − All index items consist of table partition and sort key values, and index key values. This creates the smallest index.

  2. INCLUDE − It includes KEYS_ONLY attributes and specified non-key attributes.

  3. ALL − It includes all source table attributes, creating the largest possible index.

请注意将属性投影到全局二级索引中的权衡,这涉及吞吐量和存储成本。

Note the tradeoffs in projecting attributes into a global secondary index, which relate to throughput and storage cost.

考虑以下几点:

Consider the following points −

  1. If you only need access to a few attributes, with low latency, project only those you need. This reduces storage and write costs.

  2. If an application frequently accesses certain non-key attributes, project them because the storage costs pale in comparison to scan consumption.

  3. You can project large sets of attributes frequently accessed, however, this carries a high storage cost.

  4. Use KEYS_ONLY for infrequent table queries and frequent writes/updates. This controls size, but still offers good performance on queries.

Global Secondary Index Queries and Scans

您可以使用查询来访问索引中的单个或多个项。您必须指定索引和表名称、所需属性和条件;可以选择按升序或降序返回结果。

You can utilize queries for accessing a single or multiple items in an index. You must specify index and table name, desired attributes, and conditions; with the option to return results in ascending or descending order.

您还可以使用扫描来获取所有索引数据。它需要表和索引名称。您利用筛选器表达式检索特定数据。

You can also utilize scans to get all index data. It requires table and index name. You utilize a filter expression to retrieve specific data.

Table and Index Data Synchronization

DynamoDB 会自动对索引与其父表进行同步。对项的每个修改操作都会导致异步更新,但是应用程序不会直接写入索引。

DynamoDB automatically performs synchronization on indexes with their parent table. Each modifying operation on items causes asynchronous updates, however, applications do not write to indexes directly.

您需要了解 DynamoDB 维护对索引的影响。在创建索引时,您指定键属性和数据类型,这意味着在写入时,这些数据类型必须与键模式数据类型匹配。

You need to understand the impact of DynamoDB maintenance on indices. On creation of an index, you specify key attributes and data types, which means on a write, those data types must match key schema data types.

在创建或删除项目时,索引将以最终一致的方式更新,但是数据更新会在几分之一秒内传播(除非系统发生某种类型的故障)。您必须在应用程序中考虑此延迟。

On item creation or deletion, indexes update in an eventually consistent manner, however, updates to data propagate in a fraction of a second (unless system failure of some type occurs). You must account for this delay in applications.

Throughput Considerations in Global Secondary Indexes −多个全局二级索引会影响吞吐量。索引创建需要容量单位规范,该规范与表分开存在,导致操作消耗索引容量单位而不是表单位。

Throughput Considerations in Global Secondary Indexes − Multiple global secondary indexes impact throughput. Index creation requires capacity unit specifications, which exist separate from the table, resulting in operations consuming index capacity units rather than table units.

如果查询或写入超出预置吞吐量,可能导致限制。使用 DescribeTable 查看吞吐量设置。

This can result in throttling if a query or write exceeds provisioned throughput. View throughput settings by using DescribeTable.

Read Capacity − 全局二级索引提供最终一致性。在查询中,DynamoDB 根据用于表的计算执行预置计算,唯一不同之处在于使用索引条目大小而不是条目大小。一个查询返回的限制仍然是 1MB,包括每个返回条目的属性名称大小和值。

Read Capacity − Global secondary indexes deliver eventual consistency. In queries, DynamoDB performs provision calculations identical to that used for tables, with a lone difference of using index entry size rather than item size. The limit of a query returns remains 1MB, which includes attribute name size and values across every returned item.

Write Capacity

当执行写入操作时,受影响的索引会消耗写入单元。写入吞吐量成本是表格写入中消耗的写入容量单元和索引更新中消耗的单元的总和。成功的写入操作需要有足够的容量,否则会导致节流。

When write operations occur, the affected index consumes write units. Write throughput costs are the sum of write capacity units consumed in table writes and units consumed in index updates. A successful write operation requires sufficient capacity, or it results in throttling.

写入成本还依赖于某些因素,其中一些如下所述 −

Write costs also remain dependent on certain factors, some of which are as follows −

  1. New items defining indexed attributes or item updates defining undefined indexed attributes use a single write operation to add the item to the index.

  2. Updates changing indexed key attribute value use two writes to delete an item and write a new one.

  3. A table write triggering deletion of an indexed attribute uses a single write to erase the old item projection in the index.

  4. Items absent in the index prior to and after an update operation use no writes.

  5. Updates changing only projected attribute value in the index key schema, and not indexed key attribute value, use one write to update values of projected attributes into the index.

所有这些因素都假定条目大小小于或等于 1KB。

All these factors assume an item size of less than or equal to 1KB.

Global Secondary Index Storage

在条目写入时,DynamoDB 会自动将正确的属性集复制到必须存在属性的任何索引中。这会影响您的账户,因为它会为表格条目存储和属性存储向其收费。使用结果空间由以下数量的总和得出 −

On an item write, DynamoDB automatically copies the right set of attributes to any indices where the attributes must exist. This impacts your account by charging it for table item storage and attribute storage. The space used results from the sum of these quantities −

  1. Byte size of table primary key

  2. Byte size of index key attribute

  3. Byte size of projected attributes

  4. 100 byte-overhead per index item

您可以通过估算平均条目大小并将此值乘以具有全局二级索引键属性的表格条目的数量来估算存储需求。

You can estimate storage needs through estimating average item size and multiplying by the quantity of the table items with the global secondary index key attributes.

DynamoDB 不会为定义为索引分区或排序键的未定义属性的表格条目写入条目数据。

DynamoDB does not write item data for a table item with an undefined attribute defined as an index partition or sort key.

Global Secondary Index Crud

使用 CreateTable 操作与 GlobalSecondaryIndexes 参数配对创建包含全局二级索引的表格。您必须指定一个属性作为索引分区键,或使用另一个属性作为索引排序键。所有索引键属性必须是字符串、数字或二进制标量。您还必须提供吞吐量设置,包括 ReadCapacityUnitsWriteCapacityUnits

Create a table with global secondary indexes by using the CreateTable operation paired with the GlobalSecondaryIndexes parameter. You must specify an attribute to serve as the index partition key, or use another for the index sort key. All index key attributes must be string, number, or binary scalars. You must also provide throughput settings, consisting of ReadCapacityUnits and WriteCapacityUnits.

再次使用 GlobalSecondaryIndexes 参数使用 UpdateTable 向现有表格添加全局二级索引。

Use UpdateTable to add global secondary indexes to existing tables using the GlobalSecondaryIndexes parameter once again.

在此操作中,您必须提供以下输入 -

In this operation, you must provide the following inputs −

  1. Index name

  2. Key schema

  3. Projected attributes

  4. Throughput settings

通过添加全局二级索引,可能会花费大量时间处理具有大量表格的情况,这是因为项体量、预计属性量、写入量和写活动。使用 CloudWatch 度量监控此进程。

By adding a global secondary index, it may take a substantial time with large tables due to item volume, projected attributes volume, write capacity, and write activity. Use CloudWatch metrics to monitor the process.

使用 DescribeTable 提取全局二级索引的状态信息。它会返回全局二级索引的一个四种 IndexStatus 之一 -

Use DescribeTable to fetch status information for a global secondary index. It returns one of four IndexStatus for GlobalSecondaryIndexes −

  1. CREATING − It indicates the build stage of the index, and its unavailability.

  2. ACTIVE − It indicates the readiness of the index for use.

  3. UPDATING − It indicates the update status of throughput settings.

  4. DELETING − It indicates the delete status of the index, and its permanent unavailability for use.

在加载/重新填充阶段(DynamoDB 将属性写入索引并跟踪添加/删除/更新的项)更新全局二级索引预配的吞吐量设置。使用 UpdateTable 执行此操作。

Update global secondary index provisioned throughput settings during the loading/backfilling stage (DynamoDB writing attributes to an index and tracking added/deleted/updated items). Use UpdateTable to perform this operation.

请记住,您不能在重新填充阶段添加/删除其他索引。

You should remember that you cannot add/delete other indices during the backfilling stage.

使用 UpdateTable 删除全局二级索引。它只允许每次操作删除一个索引,然而,您可以同时运行多个操作(最多五个)。删除过程不影响父表读/写活动,但在操作完成前您不能添加/删除其他索引。

Use UpdateTable to delete global secondary indexes. It permits deletion of only one index per operation, however, you can run multiple operations concurrently, up to five. The deletion process does not affect the read/write activities of the parent table, but you cannot add/delete other indices until the operation completes.

Using Java to Work with Global Secondary Indexes

通过 CreateTable 使用索引创建表格。只需创建一个 DynamoDB 类实例,一个 CreateTableRequest 类实例用于请求信息,并将请求对象传递给 CreateTable 方法。

Create a table with an index through CreateTable. Simply create a DynamoDB class instance, a CreateTableRequest class instance for request information, and pass the request object to the CreateTable method.

以下程序是一个小例子 -

The following program is a short example −

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient (
   new ProfileCredentialsProvider()));

// Attributes
ArrayList<AttributeDefinition> attributeDefinitions = new
   ArrayList<AttributeDefinition>();
attributeDefinitions.add(new AttributeDefinition()
   .withAttributeName("City")
   .withAttributeType("S"));

attributeDefinitions.add(new AttributeDefinition()
   .withAttributeName("Date")
   .withAttributeType("S"));

attributeDefinitions.add(new AttributeDefinition()
   .withAttributeName("Wind")
   .withAttributeType("N"));

// Key schema of the table
ArrayList<KeySchemaElement> tableKeySchema = new ArrayList<KeySchemaElement>();
tableKeySchema.add(new KeySchemaElement()
   .withAttributeName("City")
   .withKeyType(KeyType.HASH));              //Partition key

tableKeySchema.add(new KeySchemaElement()
   .withAttributeName("Date")
   .withKeyType(KeyType.RANGE));             //Sort key

// Wind index
GlobalSecondaryIndex windIndex = new GlobalSecondaryIndex()
   .withIndexName("WindIndex")
   .withProvisionedThroughput(new ProvisionedThroughput()
   .withReadCapacityUnits((long) 10)
   .withWriteCapacityUnits((long) 1))
   .withProjection(new Projection().withProjectionType(ProjectionType.ALL));

ArrayList<KeySchemaElement> indexKeySchema = new ArrayList<KeySchemaElement>();
indexKeySchema.add(new KeySchemaElement()
   .withAttributeName("Date")
   .withKeyType(KeyType.HASH));              //Partition key

indexKeySchema.add(new KeySchemaElement()
   .withAttributeName("Wind")
   .withKeyType(KeyType.RANGE));             //Sort key

windIndex.setKeySchema(indexKeySchema);
CreateTableRequest createTableRequest = new CreateTableRequest()
   .withTableName("ClimateInfo")
   .withProvisionedThroughput(new ProvisionedThroughput()
   .withReadCapacityUnits((long) 5)
   .withWriteCapacityUnits((long) 1))
   .withAttributeDefinitions(attributeDefinitions)
   .withKeySchema(tableKeySchema)
   .withGlobalSecondaryIndexes(windIndex);
Table table = dynamoDB.createTable(createTableRequest);
System.out.println(table.getDescription());

使用 DescribeTable 检索索引信息。首先,创建一个 DynamoDB 类实例。然后创建一个类实例,以关注一个索引。最后,将表格传递给 describe 方法。

Retrieve the index information with DescribeTable. First, create a DynamoDB class instance. Then create a Table class instance to target an index. Finally, pass the table to the describe method.

下面是一个小例子 -

Here is a short example −

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient (
   new ProfileCredentialsProvider()));

Table table = dynamoDB.getTable("ClimateInfo");
TableDescription tableDesc = table.describe();
Iterator<GlobalSecondaryIndexDescription> gsiIter =
   tableDesc.getGlobalSecondaryIndexes().iterator();

while (gsiIter.hasNext()) {
   GlobalSecondaryIndexDescription gsiDesc = gsiIter.next();
   System.out.println("Index data " + gsiDesc.getIndexName() + ":");
   Iterator<KeySchemaElement> kse7Iter = gsiDesc.getKeySchema().iterator();

   while (kseIter.hasNext()) {
      KeySchemaElement kse = kseIter.next();
      System.out.printf("\t%s: %s\n", kse.getAttributeName(), kse.getKeyType());
   }
   Projection projection = gsiDesc.getProjection();
   System.out.println("\tProjection type: " + projection.getProjectionType());

   if (projection.getProjectionType().toString().equals("INCLUDE")) {
      System.out.println("\t\tNon-key projected attributes: "
         + projection.getNonKeyAttributes());
   }
}

使用 Query 执行索引查询,就像表格查询一样。只需创建一个 DynamoDB 类实例,一个类实例用于目标索引,一个类实例用于具体索引,并将索引和查询对象传递给 query 方法。

Use Query to perform an index query as with a table query. Simply create a DynamoDB class instance, a Table class instance for the target index, an Index class instance for the specific index, and pass the index and query object to the query method.

查看以下代码以更好地理解 -

Take a look at the following code to understand better −

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient (
   new ProfileCredentialsProvider()));

Table table = dynamoDB.getTable("ClimateInfo");
Index index = table.getIndex("WindIndex");
QuerySpec spec = new QuerySpec()
   .withKeyConditionExpression("#d = :v_date and Wind = :v_wind")
   .withNameMap(new NameMap()
   .with("#d", "Date"))
   .withValueMap(new ValueMap()
   .withString(":v_date","2016-05-15")
   .withNumber(":v_wind",0));

ItemCollection<QueryOutcome> items = index.query(spec);
Iterator<Item> iter = items.iterator();

while (iter.hasNext()) {
   System.out.println(iter.next().toJSONPretty());
}

以下程序是一个较大的例子,以便更好地理解:

The following program is a bigger example for better understanding −

Note − 以下程序可能会假设有一个已创建的数据源。在尝试执行之前,获得支持库并创建必要的数据源(具备所需特征的表格或其他引用的源)。

Note − The following program may assume a previously created data source. Before attempting to execute, acquire supporting libraries and create necessary data sources (tables with required characteristics, or other referenced sources).

此示例还使用了 Eclipse IDE、AWS 凭据文件和 Eclipse AWS Java 项目中的 AWS Toolkit。

This example also uses Eclipse IDE, an AWS credentials file, and the AWS Toolkit within an Eclipse AWS Java Project.

import java.util.ArrayList;
import java.util.Iterator;

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Index;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.ItemCollection;
import com.amazonaws.services.dynamodbv2.document.QueryOutcome;
import com.amazonaws.services.dynamodbv2.document.Table;
import com.amazonaws.services.dynamodbv2.document.spec.QuerySpec;
import com.amazonaws.services.dynamodbv2.document.utils.ValueMap;

import com.amazonaws.services.dynamodbv2.model.AttributeDefinition;
import com.amazonaws.services.dynamodbv2.model.CreateTableRequest;
import com.amazonaws.services.dynamodbv2.model.GlobalSecondaryIndex;
import com.amazonaws.services.dynamodbv2.model.KeySchemaElement;
import com.amazonaws.services.dynamodbv2.model.KeyType;
import com.amazonaws.services.dynamodbv2.model.Projection;
import com.amazonaws.services.dynamodbv2.model.ProvisionedThroughput;

public class GlobalSecondaryIndexSample {
   static DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient (
      new ProfileCredentialsProvider()));
   public static String tableName = "Bugs";
   public static void main(String[] args) throws Exception {
      createTable();
      queryIndex("CreationDateIndex");
      queryIndex("NameIndex");
      queryIndex("DueDateIndex");
   }
   public static void createTable() {
      // Attributes
      ArrayList<AttributeDefinition> attributeDefinitions = new
         ArrayList<AttributeDefinition>();
      attributeDefinitions.add(new AttributeDefinition()
         .withAttributeName("BugID")
         .withAttributeType("S"));

      attributeDefinitions.add(new AttributeDefinition()
         .withAttributeName("Name")
         .withAttributeType("S"));

      attributeDefinitions.add(new AttributeDefinition()
         .withAttributeName("CreationDate")
         .withAttributeType("S"));

      attributeDefinitions.add(new AttributeDefinition()
         .withAttributeName("DueDate")
         .withAttributeType("S"));

      // Table Key schema
      ArrayList<KeySchemaElement> tableKeySchema = new ArrayList<KeySchemaElement>();
      tableKeySchema.add (new KeySchemaElement()
         .withAttributeName("BugID")
         .withKeyType(KeyType.HASH));              //Partition key

      tableKeySchema.add (new KeySchemaElement()
         .withAttributeName("Name")
         .withKeyType(KeyType.RANGE));             //Sort key

      // Indexes' initial provisioned throughput
      ProvisionedThroughput ptIndex = new ProvisionedThroughput()
         .withReadCapacityUnits(1L)
         .withWriteCapacityUnits(1L);

      // CreationDateIndex
      GlobalSecondaryIndex creationDateIndex = new GlobalSecondaryIndex()
         .withIndexName("CreationDateIndex")
         .withProvisionedThroughput(ptIndex)
         .withKeySchema(new KeySchemaElement()
         .withAttributeName("CreationDate")
         .withKeyType(KeyType.HASH),               //Partition key
         new KeySchemaElement()
         .withAttributeName("BugID")
         .withKeyType(KeyType.RANGE))              //Sort key
         .withProjection(new Projection()
         .withProjectionType("INCLUDE")
         .withNonKeyAttributes("Description", "Status"));

      // NameIndex
      GlobalSecondaryIndex nameIndex = new GlobalSecondaryIndex()
         .withIndexName("NameIndex")
         .withProvisionedThroughput(ptIndex)
         .withKeySchema(new KeySchemaElement()
         .withAttributeName("Name")
         .withKeyType(KeyType.HASH),                  //Partition key
         new KeySchemaElement()
         .withAttributeName("BugID")
         .withKeyType(KeyType.RANGE))                 //Sort key
         .withProjection(new Projection()
         .withProjectionType("KEYS_ONLY"));

      // DueDateIndex
      GlobalSecondaryIndex dueDateIndex = new GlobalSecondaryIndex()
         .withIndexName("DueDateIndex")
         .withProvisionedThroughput(ptIndex)
         .withKeySchema(new KeySchemaElement()
         .withAttributeName("DueDate")
         .withKeyType(KeyType.HASH))               //Partition key
         .withProjection(new Projection()
         .withProjectionType("ALL"));

      CreateTableRequest createTableRequest = new CreateTableRequest()
         .withTableName(tableName)
         .withProvisionedThroughput( new ProvisionedThroughput()
         .withReadCapacityUnits( (long) 1)
         .withWriteCapacityUnits( (long) 1))
         .withAttributeDefinitions(attributeDefinitions)
         .withKeySchema(tableKeySchema)
         .withGlobalSecondaryIndexes(creationDateIndex, nameIndex, dueDateIndex);
         System.out.println("Creating " + tableName + "...");
         dynamoDB.createTable(createTableRequest);

      // Pause for active table state
      System.out.println("Waiting for ACTIVE state of " + tableName);
      try {
         Table table = dynamoDB.getTable(tableName);
         table.waitForActive();
      } catch (InterruptedException e) {
         e.printStackTrace();
      }
   }
   public static void queryIndex(String indexName) {
      Table table = dynamoDB.getTable(tableName);
      System.out.println
      ("\n*****************************************************\n");
      System.out.print("Querying index " + indexName + "...");
      Index index = table.getIndex(indexName);
      ItemCollection<QueryOutcome> items = null;
      QuerySpec querySpec = new QuerySpec();

      if (indexName == "CreationDateIndex") {
         System.out.println("Issues filed on 2016-05-22");
         querySpec.withKeyConditionExpression("CreationDate = :v_date and begins_with
            (BugID, :v_bug)")
            .withValueMap(new ValueMap()
            .withString(":v_date","2016-05-22")
            .withString(":v_bug","A-"));
         items = index.query(querySpec);
      } else if (indexName == "NameIndex") {
         System.out.println("Compile error");
         querySpec.withKeyConditionExpression("Name = :v_name and begins_with
            (BugID, :v_bug)")
            .withValueMap(new ValueMap()
            .withString(":v_name","Compile error")
            .withString(":v_bug","A-"));
         items = index.query(querySpec);
      } else if (indexName == "DueDateIndex") {
         System.out.println("Items due on 2016-10-15");
         querySpec.withKeyConditionExpression("DueDate = :v_date")
         .withValueMap(new ValueMap()
         .withString(":v_date","2016-10-15"));
         items = index.query(querySpec);
      } else {
         System.out.println("\nInvalid index name");
         return;
      }
      Iterator<Item> iterator = items.iterator();
      System.out.println("Query: getting result...");

      while (iterator.hasNext()) {
         System.out.println(iterator.next().toJSONPretty());
      }
   }
}

DynamoDB - Local Secondary Indexes

一些应用程序仅用主键执行查询,但某些情况可受益于备用排序密钥。通过创建单个或多个本地二级索引让应用程序做出选择。

Some applications only perform queries with the primary key, but some situations benefit from an alternate sort key. Allow your application a choice by creating a single or multiple local secondary indexes.

复杂的数据访问要求,例如梳理数百万个项,需要执行更有效的查询/扫描。本地二级索引为分区密钥值提供备用排序密钥。它们还保存全部或部分表格属性的副本。它们按表格分区密钥组织数据,但使用不同的排序密钥。

Complex data access requirements, such as combing millions of items, make it necessary to perform more efficient queries/scans. Local secondary indices provide an alternate sort key for a partition key value. They also hold copies of all or some table attributes. They organize data by table partition key, but use a different sort key.

使用本地二级索引无需对整个表进行扫描,并允许使用排序键进行简单快捷的查询。

Using a local secondary index removes the need for a whole table scan, and allows a simple and quick query using a sort key.

所有本地二级索引都必须满足特定条件——

All the local secondary indexes must satisfy certain conditions −

  1. Identical partition key and source table partition key.

  2. A sort key of only one scalar attribute.

  3. Projection of the source table sort key acting as a non-key attribute.

所有本地二级索引自动持有父表的 partition key 和 sort key。在查询中,这意味着可以有效收集投射的属性,也可以检索未投射的属性。

All the local secondary indexes automatically hold partition and sort keys from parent tables. In queries, this means efficient gathering of projected attributes, and also retrieval of attributes not projected.

本地二级索引的存储限制仍然是每个 partition key 值 10GB,其中包括所有表项和共享 partition key 值的索引项。

The storage limit for a local secondary index remains 10GB per partition key value, which includes all table items, and index items sharing a partition key value.

Projecting an Attribute

由于复杂性,某些操作需要额外的读取/获取。这些操作可能消耗大量吞吐量。通过隔离这些属性,投射可避免代价高昂的获取并执行丰富查询。记住,投射由复制到二级索引的属性组成。

Some operations require excess reads/fetching due to complexity. These operations can consume substantial throughput. Projection allows you to avoid costly fetching and perform rich queries by isolating these attributes. Remember projections consist of attributes copied into a secondary index.

在创建二级索引时,可以指定投射的属性。回想一下 DynamoDB 提供的三个选项: KEYS_ONLY, INCLUDE, and ALL

When making a secondary index, you specify the attributes projected. Recall the three options provided by DynamoDB: KEYS_ONLY, INCLUDE, and ALL.

在对投射中的某些属性进行选择时,考虑相关的成本权衡——

When opting for certain attributes in projection, consider the associated cost tradeoffs −

  1. If you project only a small set of necessary attributes, you dramatically reduce the storage costs.

  2. If you project frequently accessed non-key attributes, you offset scan costs with storage costs.

  3. If you project most or all non-key attributes, this maximizes flexibility and reduces throughput (no retrievals); however, storage costs rise.

  4. If you project KEYS_ONLY for frequent writes/updates and infrequent queries, it minimizes size, but maintains query preparation.

Local Secondary Index Creation

使用 CreateTable 的 LocalSecondaryIndex 参数可以创建单个或多个本地二级索引。你必须为排序键指定一个非键属性。在创建表时,你创建本地二级索引。在删除时,你删除这些索引。

Use the LocalSecondaryIndex parameter of CreateTable to make a single or multiple local secondary indexes. You must specify one non-key attribute for the sort key. On table creation, you create local secondary indices. On deletion, you delete these indexes.

具有本地二级索引的表的每个分区键值的大小必须遵守 10GB 的限制,但可以存储任意数量的项。

Tables with a local secondary index must obey a limit of 10GB in size per partition key value, but can store any amount of items.

Local Secondary Index Queries and Scans

当索引中的多个项共享排序键值时,在本地二级索引上执行的查询操作会返回所有具有匹配分区键值的项。匹配的项不会按特定顺序返回。对本地二级索引的查询使用最终一致性或强一致性,其中强一致性读取可传递最新的值。

A query operation on local secondary indexes returns all items with a matching partition key value when multiple items in the index share sort key values. Matching items do not return in a certain order. Queries for local secondary indexes use either eventual or strong consistency, with strongly consistent reads delivering the latest values.

一个扫描操作返回全部的二级本地索引数据。扫描要求提供表名和索引名,并允许使用过滤器表达式来丢弃数据。

A scan operation returns all local secondary index data. Scans require you to provide a table and index name, and allow the use of a filter expression to discard data.

Item Writing

在创建本地二级索引时,可以指定排序键属性及其数据类型。如果该项会定义索引键的属性,当写入该项时,其类型必须与键模式的数据类型匹配。

On creation of a local secondary index, you specify a sort key attribute and its data type. When you write an item, its type must match the data type of the key schema if the item defines an attribute of an index key.

DynamoDB 不对表项和本地二级索引项征收一对一关系要求。带有多个本地二级索引的表会比具有较少索引的表产生更高的写入成本。

DynamoDB imposes no one-to-one relationship requirements on table items and local secondary index items. The tables with multiple local secondary indexes carry higher write costs than those with less.

Throughput Considerations in Local Secondary Indexes

查询的读取容量消耗取决于数据访问的本质。查询使用最终或强一致性,强一致性读取使用一个单位,而最终一致性读取使用 0.5 个单位。

Read capacity consumption of a query depends on the nature of data access. Queries use either eventual or strong consistency, with strongly consistent reads using one unit compared to half a unit in eventually consistent reads.

结果限制包括 1MB 的最大值。结果大小来源于匹配索引项大小的总和(向上取整到最接近的 4KB)以及匹配表项大小的总和(同样向上取整到最接近的 4KB)。

Result limitations include a 1MB size maximum. Result sizes come from the sum of matching index item size rounded up to the nearest 4KB, and matching table item size also rounded up to the nearest 4KB.

写入容量消耗保持在预置单位内。通过找到表写入中消耗的单位总和及更新索引中消耗的单位总和来计算总预置成本。

The write capacity consumption remains within provisioned units. Calculate the total provisioned cost by finding the sum of consumed units in table writing and consumed units in updating indices.

你还可以考虑影响成本的关键因素,其中一些因素可以是 −

You can also consider the key factors influencing cost, some of which can be −

  1. When you write an item defining an indexed attribute or update an item to define an undefined indexed attribute, a single write operation occurs.

  2. When a table update changes an indexed key attribute value, two writes occur to delete and then – add an item.

  3. When a write causes the deletion of an indexed attribute, one write occurs to remove the old item projection.

  4. When an item does not exist within the index prior to or after an update, no writes occur.

Local Secondary Index Storage

在表项写入中,DynamoDB 会自动复制正确的属性集到所需的本地二级索引。这会产生账户费用。所用的空间来自于表主密钥字节大小、索引键属性字节大小、任何当前投影属性字节大小以及每个索引项的开销中 100 个字节的总和。

On a table item write, DynamoDB automatically copies the right attribute set to the required local secondary indexes. This charges your account. The space used results from the sum of table primary key byte size, index key attribute byte size, any present projected attribute byte size, and 100 bytes in overhead for each index item.

估算存储可以通过估算平均索引项大小并乘以表项数量来获得。

The estimate storage is got by estimating average index item size and multiplying by table item quantity.

Using Java to Work with Local Secondary Indexes

首先创建一个 DynamoDB 类实例,来创建一个本地二级索引。然后,使用必要的请求信息创建一个 CreateTableRequest 类实例。最后,使用 createTable 方法。

Create a local secondary index by first creating a DynamoDB class instance. Then, create a CreateTableRequest class instance with necessary request information. Finally, use the createTable method.

Example

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
   new ProfileCredentialsProvider()));
String tableName = "Tools";
CreateTableRequest createTableRequest = new
   CreateTableRequest().withTableName(tableName);

//Provisioned Throughput
createTableRequest.setProvisionedThroughput (
   new ProvisionedThroughput()
   .withReadCapacityUnits((long)5)
   .withWriteCapacityUnits(( long)5));

//Attributes
ArrayList<AttributeDefinition> attributeDefinitions =
   new ArrayList<AttributeDefinition>();
   attributeDefinitions.add(new AttributeDefinition()
   .withAttributeName("Make")
   .withAttributeType("S"));

attributeDefinitions.add(new AttributeDefinition()
   .withAttributeName("Model")
   .withAttributeType("S"));

attributeDefinitions.add(new AttributeDefinition()
   .withAttributeName("Line")
   .withAttributeType("S"));

createTableRequest.setAttributeDefinitions(attributeDefinitions);

//Key Schema
ArrayList<KeySchemaElement> tableKeySchema = new
   ArrayList<KeySchemaElement>();

tableKeySchema.add(new KeySchemaElement()
   .withAttributeName("Make")
   .withKeyType(KeyType.HASH));                    //Partition key

tableKeySchema.add(new KeySchemaElement()
   .withAttributeName("Model")
   .withKeyType(KeyType.RANGE));                   //Sort key

createTableRequest.setKeySchema(tableKeySchema);
ArrayList<KeySchemaElement> indexKeySchema = new
   ArrayList<KeySchemaElement>();

indexKeySchema.add(new KeySchemaElement()
   .withAttributeName("Make")
   .withKeyType(KeyType.HASH));                   //Partition key

indexKeySchema.add(new KeySchemaElement()
   .withAttributeName("Line")
   .withKeyType(KeyType.RANGE));                   //Sort key

Projection projection = new Projection()
   .withProjectionType(ProjectionType.INCLUDE);

ArrayList<String> nonKeyAttributes = new ArrayList<String>();
nonKeyAttributes.add("Type");
nonKeyAttributes.add("Year");
projection.setNonKeyAttributes(nonKeyAttributes);

LocalSecondaryIndex localSecondaryIndex = new LocalSecondaryIndex()
   .withIndexName("ModelIndex")
   .withKeySchema(indexKeySchema)
   .withProjection(p rojection);

ArrayList<LocalSecondaryIndex> localSecondaryIndexes = new
   ArrayList<LocalSecondaryIndex>();

localSecondaryIndexes.add(localSecondaryIndex);
createTableRequest.setLocalSecondaryIndexes(localSecondaryIndexes);
Table table = dynamoDB.createTable(createTableRequest);
System.out.println(table.getDescription());

使用 describe 方法来检索有关本地二级索引的信息。只需创建一个 DynamoDB 类实例,创建一个 Table 类实例,并把表传递到 describe 方法。

Retrieve information about a local secondary index with the describe method. Simply create a DynamoDB class instance, create a Table class instance, and pass the table to the describe method.

Example

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
   new ProfileCredentialsProvider()));

String tableName = "Tools";
Table table = dynamoDB.getTable(tableName);
TableDescription tableDescription = table.describe();

List<LocalSecondaryIndexDescription> localSecondaryIndexes =
   tableDescription.getLocalSecondaryIndexes();

Iterator<LocalSecondaryIndexDescription> lsiIter =
   localSecondaryIndexes.iterator();

while (lsiIter.hasNext()) {
   LocalSecondaryIndexDescription lsiDescription = lsiIter.next();
   System.out.println("Index info " + lsiDescription.getIndexName() + ":");
   Iterator<KeySchemaElement> kseIter = lsiDescription.getKeySchema().iterator();

   while (kseIter.hasNext()) {
      KeySchemaElement kse = kseIter.next();
      System.out.printf("\t%s: %s\n", kse.getAttributeName(), kse.getKeyType());
   }

   Projection projection = lsiDescription.getProjection();
   System.out.println("\tProjection type: " + projection.getProjectionType());

   if (projection.getProjectionType().toString().equals("INCLUDE")) {
      System.out.println("\t\tNon-key projected attributes: " +
         projection.getNonKeyAttributes());
   }
}

通过使用与表查询相同的步骤来执行查询。只需创建一个 DynamoDB 类实例、一个 Table 类实例、一个 Index 类实例、一个查询对象,并使用 query 方法。

Perform a query by using the same steps as a table query. Merely create a DynamoDB class instance, a Table class instance, an Index class instance, a query object, and utilize the query method.

Example

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
   new ProfileCredentialsProvider()));

String tableName = "Tools";
Table table = dynamoDB.getTable(tableName);
Index index = table.getIndex("LineIndex");
QuerySpec spec = new QuerySpec()
   .withKeyConditionExpression("Make = :v_make and Line = :v_line")
   .withValueMap(new ValueMap()
   .withString(":v_make", "Depault")
   .withString(":v_line", "SuperSawz"));

ItemCollection<QueryOutcome> items = index.query(spec);
Iterator<Item> itemsIter = items.iterator();

while (itemsIter.hasNext()) {
   Item item = itemsIter.next();
   System.out.println(item.toJSONPretty());
}

你还可以查看以下示例。

You can also review the following example.

Note −以下示例可能假定已经创建了数据源。在尝试执行之前,获取支持库并创建必要的数据源(具有所需特征的表或其他引用源)。

Note − The following example may assume a previously created data source. Before attempting to execute, acquire supporting libraries and create necessary data sources (tables with required characteristics, or other referenced sources).

以下示例还使用了 Eclipse IDE、AWS 凭证文件以及 Eclipse AWS Java 项目中的 AWS Toolkit。

The following example also uses Eclipse IDE, an AWS credentials file, and the AWS Toolkit within an Eclipse AWS Java Project.

Example

import java.util.ArrayList;
import java.util.Iterator;

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;

import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Index;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.ItemCollection;
import com.amazonaws.services.dynamodbv2.document.PutItemOutcome;
import com.amazonaws.services.dynamodbv2.document.QueryOutcome;
import com.amazonaws.services.dynamodbv2.document.Table;
import com.amazonaws.services.dynamodbv2.document.spec.QuerySpec;
import com.amazonaws.services.dynamodbv2.document.utils.ValueMap;

import com.amazonaws.services.dynamodbv2.model.AttributeDefinition;
import com.amazonaws.services.dynamodbv2.model.CreateTableRequest;
import com.amazonaws.services.dynamodbv2.model.KeySchemaElement;
import com.amazonaws.services.dynamodbv2.model.KeyType;
import com.amazonaws.services.dynamodbv2.model.LocalSecondaryIndex;
import com.amazonaws.services.dynamodbv2.model.Projection;
import com.amazonaws.services.dynamodbv2.model.ProjectionType;
import com.amazonaws.services.dynamodbv2.model.ProvisionedThroughput;
import com.amazonaws.services.dynamodbv2.model.ReturnConsumedCapacity;
import com.amazonaws.services.dynamodbv2.model.Select;

public class LocalSecondaryIndexSample {
   static DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
      new ProfileCredentialsProvider()));
   public static String tableName = "ProductOrders";

   public static void main(String[] args) throws Exception {
      createTable();
      query(null);
      query("IsOpenIndex");
      query("OrderCreationDateIndex");
   }
   public static void createTable() {
      CreateTableRequest createTableRequest = new CreateTableRequest()
         .withTableName(tableName)
         .withProvisionedThroughput(new ProvisionedThroughput()
         .withReadCapacityUnits((long) 1)
         .withWriteCapacityUnits((long) 1));

      // Table partition and sort keys attributes
      ArrayList<AttributeDefinition> attributeDefinitions = new
         ArrayList<AttributeDefinition>();

      attributeDefinitions.add(new AttributeDefinition()
         .withAttributeName("CustomerID")
         .withAttributeType("S"));

      attributeDefinitions.add(new AttributeDefinition()
         .withAttributeName("OrderID")
         .withAttributeType("N"));

      // Index primary key attributes
      attributeDefinitions.add(new AttributeDefinition()
         .withAttributeName("OrderDate")
         .withAttributeType("N"));

      attributeDefinitions.add(new AttributeDefinition()
         .withAttributeName("OpenStatus")
         .withAttributeType("N"));
      createTableRequest.setAttributeDefinitions(attributeDefinitions);

      // Table key schema
      ArrayList<KeySchemaElement> tableKeySchema = new
         ArrayList<KeySchemaElement>();
      tableKeySchema.add(new KeySchemaElement()
         .withAttributeName("CustomerID")
         .withKeyType(KeyType.HASH));                    //Partition key

      tableKeySchema.add(new KeySchemaElement()
         .withAttributeName("OrderID")
         .withKeyType(KeyType.RANGE));                   //Sort key

      createTableRequest.setKeySchema(tableKeySchema);
      ArrayList<LocalSecondaryIndex> localSecondaryIndexes = new
         ArrayList<LocalSecondaryIndex>();

      // OrderDateIndex
      LocalSecondaryIndex orderDateIndex = new LocalSecondaryIndex()
         .withIndexName("OrderDateIndex");

      // OrderDateIndex key schema
      ArrayList<KeySchemaElement> indexKeySchema = new
         ArrayList<KeySchemaElement>();
      indexKeySchema.add(new KeySchemaElement()
         .withAttributeName("CustomerID")
         .withKeyType(KeyType.HASH));                   //Partition key

      indexKeySchema.add(new KeySchemaElement()
         .withAttributeName("OrderDate")
         .withKeyType(KeyType.RANGE));                   //Sort key
      orderDateIndex.setKeySchema(indexKeySchema);

      // OrderCreationDateIndex projection w/attributes list
      Projection projection = new Projection()
         .withProjectionType(ProjectionType.INCLUDE);

      ArrayList<String> nonKeyAttributes = new ArrayList<String>();
      nonKeyAttributes.add("ProdCat");
      nonKeyAttributes.add("ProdNomenclature");
      projection.setNonKeyAttributes(nonKeyAttributes);
      orderCreationDateIndex.setProjection(projection);
      localSecondaryIndexes.add(orderDateIndex);

      // IsOpenIndex
      LocalSecondaryIndex isOpenIndex = new LocalSecondaryIndex()
         .withIndexName("IsOpenIndex");

      // OpenStatusIndex key schema
      indexKeySchema = new ArrayList<KeySchemaElement>();
      indexKeySchema.add(new KeySchemaElement()
         .withAttributeName("CustomerID")
         .withKeyType(KeyType.HASH));                   //Partition key

      indexKeySchema.add(new KeySchemaElement()
         .withAttributeName("OpenStatus")
         .withKeyType(KeyType.RANGE));                   //Sort key

      // OpenStatusIndex projection
      projection = new Projection() .withProjectionType(ProjectionType.ALL);
      OpenStatusIndex.setKeySchema(indexKeySchema);
      OpenStatusIndex.setProjection(projection);
      localSecondaryIndexes.add(OpenStatusIndex);

      // Put definitions in CreateTable request
      createTableRequest.setLocalSecondaryIndexes(localSecondaryIndexes);
      System.out.println("Spawning table " + tableName + "...");
      System.out.println(dynamoDB.createTable(createTableRequest));

      // Pause for ACTIVE status
      System.out.println("Waiting for ACTIVE table:" + tableName);
      try {
         Table table = dynamoDB.getTable(tableName);
         table.waitForActive();
      } catch (InterruptedException e) {
         e.printStackTrace();
      }
   }
   public static void query(String indexName) {
      Table table = dynamoDB.getTable(tableName);
      System.out.println("\n*************************************************\n");
      System.out.println("Executing query on" + tableName);
      QuerySpec querySpec = new QuerySpec()
         .withConsistentRead(true)
         .withScanIndexForward(true)
         .withReturnConsumedCapacity(ReturnConsumedCapacity.TOTAL);

      if (indexName == "OpenStatusIndex") {
         System.out.println("\nEmploying index: '" + indexName
            + "' open orders for this customer.");

         System.out.println(
            "Returns only user-specified attribute list\n");
         Index index = table.getIndex(indexName);

         querySpec.withKeyConditionExpression("CustomerID = :v_custmid and
            OpenStatus = :v_openstat")
            .withValueMap(new ValueMap()
            .withString(":v_custmid", "jane@sample.com")
            .withNumber(":v_openstat", 1));

         querySpec.withProjectionExpression(
            "OrderDate, ProdCat, ProdNomenclature, OrderStatus");
            ItemCollection<QueryOutcome> items = index.query(querySpec);
            Iterator<Item> iterator = items.iterator();
            System.out.println("Printing query results...");

         while (iterator.hasNext()) {
            System.out.println(iterator.next().toJSONPretty());
         }
      } else if (indexName == "OrderDateIndex") {
         System.out.println("\nUsing index: '" + indexName
            + "': this customer's orders placed after 05/22/2016.");
         System.out.println("Projected attributes are returned\n");
         Index index = table.getIndex(indexName);

         querySpec.withKeyConditionExpression("CustomerID = :v_custmid and OrderDate
            >= :v_ordrdate")
            .withValueMap(new ValueMap()
            .withString(":v_custmid", "jane@sample.com")
            .withNumber(":v_ordrdate", 20160522));

         querySpec.withSelect(Select.ALL_PROJECTED_ATTRIBUTES);
         ItemCollection<QueryOutcome> items = index.query(querySpec);
         Iterator<Item> iterator = items.iterator();
         System.out.println("Printing query results...");

         while (iterator.hasNext()) {
            System.out.println(iterator.next().toJSONPretty());
         }
      } else {
         System.out.println("\nNo index: All Jane's orders by OrderID:\n");
         querySpec.withKeyConditionExpression("CustomerID = :v_custmid")
            .withValueMap(new ValueMap()
            .withString(":v_custmid", "jane@example.com"));

         ItemCollection<QueryOutcome> items = table.query(querySpec);
         Iterator<Item> iterator = items.iterator();
         System.out.println("Printing query results...");

         while (iterator.hasNext()) {
            System.out.println(iterator.next().toJSONPretty());
         }
      }
   }
}

DynamoDB - Aggregation

DynamoDB 不提供聚合函数。你必须创新地使用查询、扫描、索引和各种工具来执行这些任务。在所有这一切中,这些操作中查询/扫描的吞吐量开销可能会很大。

DynamoDB does not provide aggregation functions. You must make creative use of queries, scans, indices, and assorted tools to perform these tasks. In all this, the throughput expense of queries/scans in these operations can be heavy.

你还可以选择使用适合偏好 DynamoDB 编码语言的库和其它工具。在使用之前确保其与 DynamoDB 的兼容性。

You also have the option to use libraries and other tools for your preferred DynamoDB coding language. Ensure their compatibility with DynamoDB prior to using it.

Calculate Maximum or Minimum

使用结果的升序/降序存储顺序、Limit 参数以及任何设置顺序的参数来查找最高和最低的值。

Utilize the ascending/descending storage order of results, the Limit parameter, and any parameters which set order to find the highest and lowest values.

例如 -

For example −

Map<String, AttributeValue> eaval = new HashMap<>();
eaval.put(":v1", new AttributeValue().withS("hashval"));
queryExpression = new DynamoDBQueryExpression<Table>()
   .withIndexName("yourindexname")
   .withKeyConditionExpression("HK = :v1")
   .withExpressionAttributeValues(values)
   .withScanIndexForward(false);                //descending order

queryExpression.setLimit(1);
QueryResultPage<Lookup> res =
   dynamoDBMapper.queryPage(Table.class, queryExpression);

Calculate Count

使用 DescribeTable 获取表项计数,但请注意它提供的是旧数据。此外,利用 Java getScannedCount method

Use DescribeTable to get a count of the table items, however, note that it provides stale data. Also, utilize the Java getScannedCount method.

使用 LastEvaluatedKey 确保它提供所有结果。

Utilize LastEvaluatedKey to ensure it delivers all results.

例如 -

For example −

ScanRequest scanRequest = new ScanRequest().withTableName(yourtblName);
ScanResult yourresult = client.scan(scanRequest);
System.out.println("#items:" + yourresult.getScannedCount());

Calculating Average and Sum

利用索引和查询/扫描在处理前检索和过滤值。然后通过对象简单地处理这些值。

Utilize indices and a query/scan to retrieve and filter values before processing. Then simply operate on those values through an object.

DynamoDB - Access Control

DynamoDB 使用您提供用于验证请求的凭证。这些凭证是必需的并且必须包括 AWS 资源访问权限。这些权限跨越 DynamoDB 的几乎每个方面,直到操作或功能的次要特征。

DynamoDB uses credentials you provide to authenticate requests. These credentials are required and must include permissions for AWS resource access. These permissions span virtually every aspect of DynamoDB down to the minor features of an operation or functionality.

Types of Permissions

在本节中,我们将讨论 DynamoDB 中的各种权限和资源访问。

In this section, we will discuss regarding the various permissions and resource access in DynamoDB.

Authenticating Users

在注册时,您提供了一个密码和电子邮件,它们作为根凭证。DynamoDB 将此数据与您的 AWS 帐户相关联,并使用它来授予对所有资源的完全访问权限。

On signup, you provided a password and email, which serve as root credentials. DynamoDB associates this data with your AWS account, and uses it to give complete access to all resources.

AWS 建议您仅将根凭证用于创建管理帐户。这允许您创建具有较少权限的 IAM 帐户/用户。IAM 用户是使用 IAM 服务衍生的其他帐户。他们的访问权限/特权包括访问安全页面和某些自定义权限,如表修改。

AWS recommends you use your root credentials only for the creation of an administration account. This allows you to create IAM accounts/users with less privileges. IAM users are other accounts spawned with the IAM service. Their access permissions/privileges include access to secure pages and certain custom permissions like table modification.

访问密钥为其他帐户和访问提供了另一个选项。使用它们来授予访问权限,并且在某些情况下还可以避免手动授予访问权限。联合用户通过允许通过身份提供程序进行访问,提供了另一种选择。

The access keys provide another option for additional accounts and access. Use them to grant access, and also to avoid manual granting of access in certain situations. Federated users provide yet another option by allowing access through an identity provider.

Administration

AWS 资源仍归帐户所有。权限策略管理授予生成或访问资源的权限。管理员将权限策略与 IAM 身份相关联,这意味着角色、组、用户和服务。他们还将权限附加到资源。

AWS resources remain under ownership of an account. Permissions policies govern the permissions granted to spawn or access resources. Administrators associate permissions policies with IAM identities, meaning roles, groups, users, and services. They also attach permissions to resources.

权限指定用户、资源和操作。注意,管理员只是具有管理员权限的帐户。

Permissions specify users, resources, and actions. Note administrators are merely accounts with administrator privileges.

Operation and Resources

表仍然是 DynamoDB 中的主要资源。子资源作为附加资源,例如流和索引。这些资源使用唯一名称,其中一些名称在以下表中提及:

Tables remain the main resources in DynamoDB. Subresources serve as additional resources, e.g., streams and indices. These resources use unique names, some of which are mentioned in the following table −

Type

ARN (Amazon Resource Name)

Stream

arn:aws:dynamodb:region:account-id:table/table-name/stream/stream-label

Index

arn:aws:dynamodb:region:account-id:table/table-name/index/index-name

Table

arn:aws:dynamodb:region:account-id:table/table-name

Ownership

资源所有者被定义为生成资源的 AWS 帐户,或在资源创建中负责请求验证的负责人实体帐户。考虑它如何在 DynamoDB 环境中起作用:

A resource owner is defined as an AWS account which spawned the resource, or principal entity account responsible for request authentication in resource creation. Consider how this functions within the DynamoDB environment −

  1. In using root credentials to create a table, your account remains resource owner.

  2. In creating an IAM user and granting the user permission to create a table, your account remains the resource owner.

  3. In creating an IAM user and granting the user, and anyone capable of assuming the role, permission to create a table, your account remains the resource owner.

Manage Resource Access

访问管理主要需要关注描述用户和资源访问的权限策略。您将策略与 IAM 身份或资源相关联。但是,DynamoDB 仅支持 IAM/身份策略。

Management of access mainly requires attention to a permissions policy describing users and resource access. You associate policies with IAM identities or resources. However, DynamoDB only supports IAM/identity policies.

基于身份(IAM)的策略允许您以以下方式授予特权:

Identity-based (IAM) policies allow you to grant privileges in the following ways −

  1. Attach permissions to users or groups.

  2. Attach permissions to roles for cross-account permissions.

其他 AWS 允许基于资源的策略。这些策略允许访问诸如 S3 存储桶之类的资源。

Other AWS allow resource-based policies. These policies permit access to things like an S3 bucket.

Policy Elements

策略定义动作、效果、资源和主体;并授予执行这些操作的权限。

Policies define actions, effects, resources, and principals; and grant permission to perform these operations.

Note − API 操作可能需要针对多个动作的权限。

Note − The API operations may require permissions for multiple actions.

仔细查看以下策略元素−

Take a closer look at the following policy elements −

  1. Resource − An ARN identifies this.

  2. Action − Keywords identify these resource operations, and whether to allow or deny.

  3. Effect − It specifies the effect for a user request for an action, meaning allow or deny with denial as the default.

  4. Principal − This identifies the user attached to the policy.

Conditions

在授予权限时,您可以指定策略何时变为活动,例如在特定日期。使用条件键来表示条件,其中包括 AWS 系统范围键和 DynamoDB 键。这些键将在本教程的后面部分详细讨论。

In granting permissions, you can specify conditions for when policies become active such as on a particular date. Express conditions with condition keys, which include AWS systemwide keys and DynamoDB keys. These keys are discussed in detail later in the tutorial.

Console Permissions

用户需要某些基本权限才能使用控制台。他们还需要针对其他标准服务中的控制台的权限−

A user requires certain basic permissions to use the console. They also require permissions for the console in other standard services −

  1. CloudWatch

  2. Data Pipeline

  3. Identity and Access Management

  4. Notification Service

  5. Lambda

如果 IAM 策略证明过于有限,则用户将无法有效地使用控制台。另外,您不必担心仅调用 CLI 或 API 的那些用户的权限。

If the IAM policy proves too limited, the user cannot use the console effectively. Also, you do not need to worry about user permissions for those only calling the CLI or API.

Common Use Iam Policies

AWS 在具有独立的 IAM 管理策略的权限中涵盖了常见操作。它们提供了关键权限,使您无需深入研究自己必须授予的内容。

AWS covers common operations in permissions with standalone IAM managed policies. They provide key permissions allowing you to avoid deep investigations into what you must grant.

其中一些如下所示−

Some of them are as follows −

  1. AmazonDynamoDBReadOnlyAccess − It gives read-only access via the console.

  2. AmazonDynamoDBFullAccess − It gives full access via the console.

  3. AmazonDynamoDBFullAccesswithDataPipeline − It gives full access via the console and permits export/import with Data Pipeline.

您当然还可以制定自定义策略。

You can also ofcourse make custom policies.

Granting Privileges: Using The Shell

您可以使用 Javascript shell 授予权限。以下程序显示了典型的权限策略−

You can grant permissions with the Javascript shell. The following program shows a typical permissions policy −

{
   "Version": "2016-05-22",
   "Statement": [
      {
         "Sid": "DescribeQueryScanToolsTable",
         "Effect": "Deny",

         "Action": [
            "dynamodb:DescribeTable",
            "dynamodb:Query",
            "dynamodb:Scan"
         ],
         "Resource": "arn:aws:dynamodb:us-west-2:account-id:table/Tools"
      }
   ]
}

您可以查看以下三个示例−

You can review the three examples which are as follows −

Block the user from executing any table action.

Block the user from executing any table action.

{
   "Version": "2016-05-23",
   "Statement": [
      {
         "Sid": "AllAPIActionsOnTools",
         "Effect": "Deny",
         "Action": "dynamodb:*",
         "Resource": "arn:aws:dynamodb:us-west-2:155556789012:table/Tools"
      }
   ]
}

Block access to a table and its indices.

Block access to a table and its indices.

{
   "Version": "2016-05-23",
   "Statement": [
      {
         "Sid": "AccessAllIndexesOnTools",
         "Effect": "Deny",
         "Action": [
            "dynamodb:*"
         ],
         "Resource": [
            "arn:aws:dynamodb:us-west-2:155556789012:table/Tools",
            "arn:aws:dynamodb:us-west-2:155556789012:table/Tools/index/*"
         ]
      }
   ]
}

Block a user from making a reserved capacity offering purchase.

Block a user from making a reserved capacity offering purchase.

{
   "Version": "2016-05-23",
   "Statement": [
      {
         "Sid": "BlockReservedCapacityPurchases",
         "Effect": "Deny",
         "Action": "dynamodb:PurchaseReservedCapacityOfferings",
         "Resource": "arn:aws:dynamodb:us-west-2:155556789012:*"
      }
   ]
}

Granting Privileges: Using the GUI Console

您还可以使用 GUI 控制台来创建 IAM 政策。首先,从导航窗格中选择 Tables 。在数据表列表中,选择目标数据表并按照下列步骤进行操作。

You can also use the GUI console to create IAM policies. To begin with, choose Tables from the navigation pane. In the table list, choose the target table and follow these steps.

Step 1 − 选择 Access control 选项卡。

Step 1 − Select the Access control tab.

Step 2 − 选择身份验证提供程序、操作和策略属性。输入所有设置后,选择 Create policy

Step 2 − Select the identity provider, actions, and policy attributes. Select Create policy after entering all settings.

Step 3 − 选择 Attach policy instructions ,并完成每个必需步骤,以将策略与相应的 IAM 角色关联起来。

Step 3 − Choose Attach policy instructions, and complete each required step to associate the policy with the appropriate IAM role.

DynamoDB - Permissions API

DynamoDB API 提供了一组大型操作,这些操作需要权限。在设置权限时,您必须建立允许的操作、允许的资源以及每个操作的条件。

DynamoDB API offers a large set of actions, which require permissions. In setting permissions, you must establish the actions permitted, resources permitted, and conditions of each.

您可以在策略的操作字段内指定操作。在策略的资源字段内指定资源值。但请确保您使用正确的语法,该语法在 API 操作前包含 Dynamodb: 前缀。

You can specify actions within the Action field of the policy. Specify resource value within the Resource field of the policy. But do ensure that you use the correct syntax containing the Dynamodb: prefix with the API operation.

示例 − dynamodb:CreateTable

For example − dynamodb:CreateTable

您还可以使用条件键来筛选权限。

You can also employ condition keys to filter permissions.

Permissions and API Actions

仔细查看下表中给出的 API 操作和关联权限 −

Take a good look at the API actions and associated permissions given in the following table −

API Operation

Necessary Permission

BatchGetItem

dynamodb:BatchGetItem

BatchWriteItem

dynamodb:BatchWriteItem

CreateTable

dynamodb:CreateTable

DeleteItem

dynamodb:DeleteItem

DeleteTable

dynamodb:DeleteTable

DescribeLimits

dynamodb:DescribeLimits

DescribeReservedCapacity

dynamodb:DescribeReservedCapacity

DescribeReservedCapacityOfferings

dynamodb:DescribeReservedCapacityOfferings

DescribeStream

dynamodb:DescribeStream

DescribeTable

dynamodb:DescribeTable

GetItem

dynamodb:GetItem

GetRecords

dynamodb:GetRecords

GetShardIterator

dynamodb:GetShardIterator

ListStreams

dynamodb:ListStreams

ListTables

dynamodb:ListTables

PurchaseReservedCapacityOfferings

dynamodb:PurchaseReservedCapacityOfferings

PutItem

dynamodb:PutItem

Query

dynamodb:Query

Scan

dynamodb:Scan

UpdateItem

dynamodb:UpdateItem

UpdateTable

dynamodb:UpdateTable

Resources

在下表中,您可以查看与每个允许的 API 操作相关的资源 −

In the following table, you can review the resources associated with each permitted API action −

API Operation

Resource

BatchGetItem

arn:aws:dynamodb:region:account-id:table/table-name

BatchWriteItem

arn:aws:dynamodb:region:account-id:table/table-name

CreateTable

arn:aws:dynamodb:region:account-id:table/table-name

DeleteItem

arn:aws:dynamodb:region:account-id:table/table-name

DeleteTable

arn:aws:dynamodb:region:account-id:table/table-name

DescribeLimits

arn:aws:dynamodb:region:account-id:*

DescribeReservedCapacity

arn:aws:dynamodb:region:account-id:*

DescribeReservedCapacityOfferings

arn:aws:dynamodb:region:account-id:*

DescribeStream

arn:aws:dynamodb:region:account-id:table/table-name/stream/stream-label

DescribeTable

arn:aws:dynamodb:region:account-id:table/table-name

GetItem

arn:aws:dynamodb:region:account-id:table/table-name

GetRecords

arn:aws:dynamodb:region:account-id:table/table-name/stream/stream-label

GetShardIterator

arn:aws:dynamodb:region:account-id:table/table-name/stream/stream-label

ListStreams

arn:aws:dynamodb:region:account-id:table/table-name/stream/*

ListTables

*

PurchaseReservedCapacityOfferings

arn:aws:dynamodb:region:account-id:*

PutItem

arn:aws:dynamodb:region:account-id:table/table-name

Query

arn:aws:dynamodb:region:account-id:table/table-name or arn:aws:dynamodb:region:account-id:table/table-name/index/index-name

Scan

arn:aws:dynamodb:region:account-id:table/table-name or arn:aws:dynamodb:region:account-id:table/table-name/index/index-name

UpdateItem

arn:aws:dynamodb:region:account-id:table/table-name

UpdateTable

arn:aws:dynamodb:region:account-id:table/table-name

DynamoDB - Conditions

在授予权限时,DynamoDB 允许通过详细的 IAM 策略(带条件键)为这些权限指定条件。这支持诸如对特定项目和属性的访问之类的设置。

In granting permissions, DynamoDB allows specifying conditions for them through a detailed IAM policy with condition keys. This supports settings like access to specific items and attributes.

Note − DynamoDB 不支持任何标记。

Note − The DynamoDB does not support any tags.

Detailed Control

几个条件允许进行到项目和属性的特定性,例如基于用户账户授予对特定项目的只读访问权限。实施此级别控制的条件化 IAM 策略管理安全凭证。然后再将策略应用到所需的使用者、组和角色上。Web Identity Federation(将在后面讨论该主题)也提供了一种方法,可以通过 Amazon、Facebook 和 Google 登录来控制用户访问。

Several conditions allow specificity down to items and attributes like granting read-only access to specific items based on user account. Implement this level of control with conditioned IAM policies, which manages the security credentials. Then simply apply the policy to the desired users, groups, and roles. Web Identity Federation, a topic discussed later, also provides a way to control user access through Amazon, Facebook, and Google logins.

IAM 策略的条件元素实施访问控制。您只需将其添加到一项策略中即可。它一个使用示例包括拒绝或允许对数据表项目和属性的访问。条件元素还可以使用条件键来限制权限。

The condition element of IAM policy implements access control. You simply add it to a policy. An example of its use consists of denying or permitting access to table items and attributes. The condition element can also employ condition keys to limit permissions.

您可以查看条件键的以下两个示例 −

You can review the following two examples of the condition keys −

  1. dynamodb:LeadingKeys − It prevents the item access by users without an ID matching the partition key value.

  2. dynamodb:Attributes − It prevents users from accessing or operating on attributes outside of those listed.

在评估时,IAM 策略会导致值为真或假。如果任何部分评估为假,则整个策略评估为假,从而导致拒绝访问。务必在条件键中指定所有所需信息,以确保用户具有适当的访问权限。

On evaluation, IAM policies result in a true or false value. If any part evaluates to false, the whole policy evaluates to false, which results in denial of access. Be sure to specify all required information in condition keys to ensure users have appropriate access.

Predefined Condition Keys

AWS 提供了一组适用于所有服务的预定义条件键。它们支持广泛的用途,并能详细检查用户和访问权限。

AWS offers a collection of predefined condition keys, which apply to all services. They support a broad range of uses and fine detail in examining users and access.

Note − 条件键区分大小写。

Note − There is case sensitivity in condition keys.

你可以查看以下特定于服务的部分键的选项 −

You can review a selection of the following service-specific keys −

  1. dynamodb:LeadingKey − It represents a table’s first key attribute; the partition key. Use the ForAllValues modifier in conditions.

  2. dynamodb:Select − It represents a query/scan request Select parameter. It must be of the value ALL_ATTRIBUTES, ALL_PROJECTED_ATTRIBUTES, SPECIFIC_ATTRIBUTES, or COUNT.

  3. dynamodb:Attributes − It represents an attribute name list within a request, or attributes returned from a request. Its values and their functions resemble API action parameters, e.g., BatchGetItem uses AttributesToGet.

  4. dynamodb:ReturnValues − It represents a requests’ ReturnValues parameter, and can use these values: ALL_OLD, UPDATED_OLD, ALL_NEW, UPDATED_NEW, and NONE.

  5. dynamodb:ReturnConsumedCapacity − It represents a request’s ReturnConsumedCapacity parameter, and can use these values: TOTAL and NONE.

DynamoDB - Web Identity Federation

Web Identity Federation 允许你简化对大型用户组进行的身份验证和授权。你可以跳过创建单独帐户,并要求用户登录身份提供程序以获取临时证书或令牌。它使用 AWS Security Token Service (STS) 来管理证书。应用程序使用这些令牌与服务进行交互。

Web Identity Federation allows you to simplify authentication and authorization for large user groups. You can skip the creation of individual accounts, and require users to login to an identity provider to get temporary credentials or tokens. It uses AWS Security Token Service (STS) to manage credentials. Applications use these tokens to interact with services.

Web Identity Federation 还支持其他身份提供程序,如亚马逊、Google 和 Facebook。

Web Identity Federation also supports other identity providers such as – Amazon, Google, and Facebook.

Function − 在使用中,Web Identity Federation 首先调用身份提供程序进行用户和应用程序身份验证,然后提供程序返回令牌。这会导致应用程序调用 AWS STS 并传递令牌作为输入。STS 授权该应用程序并授予其临时访问证书,这允许应用程序使用 IAM 身份并基于策略访问资源。

Function − In use, Web Identity Federation first calls an identity provider for user and app authentication, and the provider returns a token. This results in the app calling AWS STS and passing the token for input. STS authorizes the app and grants it temporary access credentials, which allow the app to use an IAM role and access resources based on policy.

Implementing Web Identity Federation

在使用前,你必须执行以下三个步骤 −

You must perform the following three steps prior to use −

  1. Use a supported third party identity provider to register as a developer.

  2. Register your application with the provider to obtain an app ID.

  3. Create a single or multiple IAM roles, including policy attachment. You must use a role per provider per app.

预先设置一个 IAM 身份以使用 Web Identity Federation。然后你的应用程序必须执行三步流程 −

Assume one of your IAM roles to use Web Identity Federation. Your app must then perform a three-step process −

  1. Authentication

  2. Credential acquisition

  3. Resource Access

在第一步中,你的应用程序使用它自己接口调用提供程序,然后管理令牌处理过程。

In the first step, your app uses its own interface to call the provider and then manages the token process.

然后,步骤2管理令牌,要求您的应用程序向AWS STS发送请求。该请求包含第一个令牌、提供应用ID和IAM角色的ARN。STS提供在一定时间后到期的凭据设置。

Then step two manages tokens and requires your app to send an AssumeRoleWithWebIdentity request to AWS STS. The request holds the first token, the provider app ID, and the ARN of the IAM role. The STS the provides credentials set to expire after a certain period.

在最后一步中,您的应用程序收到来自STS的答复,其中包含DynamoDB资源的访问信息。它由访问凭证、到期时间、角色和角色ID组成。

In the final step, your app receives a response from STS containing access information for DynamoDB resources. It consists of access credentials, expiration time, role, and role ID.

DynamoDB - Data Pipeline

数据管道允许将数据导出和导入到表、文件或S3存储桶中。当然,这在备份、测试和类似需求或场景中被证明是有用的。

Data Pipeline allows for exporting and importing data to/from a table, file, or S3 bucket. This of course proves useful in backups, testing, and for similar needs or scenarios.

在导出中,您可以使用Data Pipeline控制台创建一个新的管道并启动Amazon EMR(Elastic MapReduce)群集以执行导出操作。EMR从DynamoDB读取数据并写入目标。我们将在本教程的后面详细讨论EMR。

In an export, you use the Data Pipeline console, which makes a new pipeline and launches an Amazon EMR (Elastic MapReduce) cluster to perform the export. An EMR reads data from DynamoDB and writes to the target. We discuss EMR in detail later in this tutorial.

在导入操作中,您可以使用Data Pipeline控制台,它会创建一个管道并启动EMR来执行导入操作。它从源读取数据并写入目标。

In an import operation, you use the Data Pipeline console, which makes a pipeline and launches EMR to perform the import. It reads data from the source and writes to the destination.

由于使用了S3和EMR等服务,因此导出/导入操作会产生一定的成本。

Note − Export/import operations carry a cost given the services used, specifically, EMR and S3.

Using Data Pipeline

使用Data Pipeline时,必须指定操作和资源权限。您可以利用IAM角色或策略来定义它们。执行导入/导出操作的用户应注意,他们需要一个有效的访问密钥ID和密钥。

You must specify action and resource permissions when using Data Pipeline. You can utilize an IAM role or policy to define them. The users who are performing imports/exports should make a note that they would require an active access key ID and secret key.

IAM Roles for Data Pipeline

您需要两个IAM角色才能使用Data Pipeline -

You need two IAM roles to use Data Pipeline −

  1. DataPipelineDefaultRole − This has all the actions you permit the pipeline to perform for you.

  2. DataPipelineDefaultResourceRole − This has resources you permit the pipeline to provision for you.

如果您是Data Pipeline的新手,则必须生成每个角色。由于现有的角色,所有先前用户都拥有这些角色。

If you are new to Data Pipeline, you must spawn each role. All the previous users possess these roles due to the existing roles.

使用IAM控制台为Data Pipeline创建IAM角色,并执行以下四个步骤 -

Use the IAM console to create IAM roles for Data Pipeline, and perform the following four steps −

  • 登录位于以下位置的IAM控制台:

Step 1 − Log in to the IAM console located at https://console.aws.amazon.com/iam/

  • 从仪表板中选择角色。

Step 2 − Select Roles from the dashboard.

  • 选择“创建角色”。然后在“角色名称”字段中输入“DataPipelineDefaultRole”,并选择“继续:权限”。在“权限策略”面板中的“分配权限”列表中,导航到“AWS服务”,并选择“Elastic MapReduce”。在“选择操作”面板中选择“全部操作”。

Step 3 − Select Create New Role. Then enter DataPipelineDefaultRole in the Role Name field, and select Next Step. In the AWS Service Roles list in the Role Type panel, navigate to Data Pipeline, and choose Select. Select Create Role in the Review panel.

  • 选择“查看策略”。

Step 4 − Select Create New Role.

DynamoDB - Data Backup

利用Data Pipeline的导入/导出功能执行备份。您执行备份的方式取决于您使用的是GUI控制台还是直接使用Data Pipeline(API)。使用控制台时,为每个表创建单独的管道,或者在使用直接选项时,在一个管道中导入/导出多个表。

Utilize Data Pipeline’s import/export functionality to perform backups. How you execute a backup depends on whether you use the GUI console, or use Data Pipeline directly (API). Either create separate pipelines for each table when using the console, or import/export multiple tables in a single pipeline if using a direct option.

Exporting and Importing Data

在执行导出操作之前,必须创建一个Amazon S3存储桶。您可以从一个或多个表导出数据。

You must create an Amazon S3 bucket prior to performing an export. You can export from one or more tables.

执行以下四个步骤来执行导出 -

Perform the following four step process to execute an export −

Step 1 - 登录 AWS 管理控制台,然后打开位于 https://console.aws.amazon.com/datapipeline/ 的数据管道控制台。

Step 1 − Log in to the AWS Management Console and open the Data Pipeline console located at https://console.aws.amazon.com/datapipeline/

Step 2 - 如果在所用 AWS 区域中没有管道,请选 Get started now 。如果有,请选 Create new pipeline

Step 2 − If you have no pipelines in the AWS region used, select Get started now. If you have one or more, select Create new pipeline.

Step 3 - 在创建页面上,为你的管道输入一个名称。为源参数选 Build using a template 。从列表中选 Export DynamoDB table to S3 。在 Source DynamoDB table name 字段中输入源表。

Step 3 − On the creation page, enter a name for your pipeline. Choose Build using a template for the Source parameter. Select Export DynamoDB table to S3 from the list. Enter the source table in the Source DynamoDB table name field.

Output S3 Folder 文本框中以以下格式输入目标 S3 存储桶:s3://nameOfBucket/region/nameOfFolder。在 S3 location for logs 文本框中为日志文件输入一个 S3 目标。

Enter the destination S3 bucket in the Output S3 Folder text box using the following format: s3://nameOfBucket/region/nameOfFolder. Enter an S3 destination for the log file in S3 location for logs text box.

Step 4 - 在输入所有设置后,选 Activate

Step 4 − Select Activate after entering all settings.

管道完成其创建过程可能需要几分钟时间。使用控制台监控其状态。通过查看导出文件,使用 S3 控制台确认处理成功。

The pipeline may take several minutes to finish its creation process. Use the console to monitor its status. Confirm successful processing with the S3 console by viewing the exported file.

Importing Data

只有在满足以下条件时,才能成功导入:你创建了一个目标表,目标和源使用相同的名称,并且目标和源使用相同的键架构。

Successful imports can only happen if the following conditions are true: you created a destination table, the destination and source use identical names, and the destination and source use identical key schema.

你可以使用一个已填充的目标表,但是,导入会替换与源项目共享键的数据项目,并且还会将附加项目添加到表中。该目标还可以使用不同的区域。

You can use a populated destination table, however, imports replace data items sharing a key with source items, and also add excess items to the table. The destination can also use a different region.

尽管你可以导出多个源,但每次操作只能导入一个。你可以按照以下步骤执行导入操作 −

Though you can export multiple sources, you can only import one per operation. You can perform an import by adhering to the following steps −

Step 1 - 登录 AWS 管理控制台,然后打开数据管道控制台。

Step 1 − Log in to the AWS Management Console, and then open the Data Pipeline console.

Step 2 - 如果打算执行跨区域导入,则应选择目标区域。

Step 2 − If you are intending to execute a cross region import, then you should select the destination region.

Step 3 - 选择 Create new pipeline

Step 3 − Select Create new pipeline.

Step 4 - 在 Name 字段中输入管道名称,为源参数选择 Build using a template ,并在模板列表中选择 Import DynamoDB backup data from S3

Step 4 − Enter the pipeline name in the Name field. Choose Build using a template for the Source parameter, and in the template list, select Import DynamoDB backup data from S3.

Input S3 Folder 文本框中输入源文件的位置。在 Target DynamoDB table name 字段中输入目标表名。然后在 S3 location for logs 文本框中输入日志文件的位置。

Enter the location of the source file in the Input S3 Folder text box. Enter the destination table name in the Target DynamoDB table name field. Then enter the location for the log file in the S3 location for logs text box.

Step 5 - 在输入所有设置后,选 Activate

Step 5 − Select Activate after entering all settings.

在创建管道后,导入会立即开始。管道完成创建过程可能需要几分钟时间。

The import starts immediately after the pipeline creation. It may take several minutes for the pipeline to complete the creation process.

Errors

当出现错误时,数据管道控制台会显示 ERROR 作为管道状态。点击出错的管道会转到其详细信息页面,该页面显示该过程的每一步以及失败发生的位置。其中的日志文件也会提供一些见解。

When errors occur, the Data Pipeline console displays ERROR as the pipeline status. Clicking the pipeline with an error takes you to its detail page, which reveals every step of the process and the point at which the failure occurred. Log files within also provide some insight.

你可以按如下方式查看常见错误原因 −

You can review the common causes of the errors as follows −

  1. The destination table for an import does not exist, or does not use identical key schema to the source.

  2. The S3 bucket does not exist, or you do not have read/write permissions for it.

  3. The pipeline timed out.

  4. You do not have the necessary export/import permissions.

  5. Your AWS account reached its resource limit.

DynamoDB - Monitoring

Amazon 通过 CloudWatch 控制台、命令行或 CloudWatch API 提供 CloudWatch,以聚合并分析性能。您还可以使用它设置警报并执行任务。它对某些事件执行特定的操作。

Amazon offers CloudWatch for aggregating and analyzing performance through the CloudWatch console, command line, or CloudWatch API. You can also use it to set alarms and perform tasks. It performs specified actions on certain events.

Cloudwatch Console

通过访问管理控制台,然后在 https://console.aws.amazon.com/cloudwatch/ 打开 CloudWatch 控制台,利用 CloudWatch。

Utilize CloudWatch by accessing the Management Console, and then opening the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

然后,您可以执行以下步骤 −

You can then perform the following steps −

  1. Select Metrics from the navigation pane.

  2. Under DynamoDB metrics within the CloudWatch Metrics by Category pane, choose Table Metrics.

  3. Use the upper pane to scroll below and examine the entire list of table metrics. The Viewing list provides metrics options.

在结果界面中,您可以通过选择资源名称和指标旁边的复选框来选择/取消选择每个指标。然后,您将能够查看每个项目的图表。

In the results interface, you can select/deselect each metric by selecting the checkbox beside the resource name and metric. Then you would be able to view graphs for each item.

API Integration

您可以使用查询访问 CloudWatch。使用指标值执行 CloudWatch 操作。注意 DynamoDB 不会发送值为零的指标。它只是跳过这些指标在该值期间内保持的时间段。

You can access CloudWatch with queries. Use metric values to perform CloudWatch actions. Note DynamoDB does not send metrics with a value of zero. It simply skips metrics for time periods where those metrics remain at that value.

以下是其中一些最常用的指标 −

The following are some of the most commonly used metrics −

  1. ConditionalCheckFailedRequests − It tracks the quantity of failed attempts at conditional writes such as conditional PutItem writes. The failed writes increment this metric by one on evaluation to false. It also throws an HTTP 400 error.

  2. ConsumedReadCapacityUnits − It quantifies the capacity units used over a certain time period. You can use this to examine individual table and index consumption.

  3. ConsumedWriteCapacityUnits − It quantifies the capacity units used over a certain time period. You can use this to examine individual table and index consumption.

  4. ReadThrottleEvents − It quantifies requests exceeding provisioned capacity units in table/index reads. It increments on each throttle including batch operations with multiple throttles.

  5. ReturnedBytes − It quantifies the bytes returned in retrieval operations within a certain time period.

  6. ReturnedItemCount − It quantifies the items returned in Query and Scan operations over a certain time period. It addresses only items returned, not those evaluated, which are typically totally different figures.

Note - 存在更多指标,大多数指标允许您计算平均值、和、最大值、最小值和计数。

Note − There are many more metrics that exist, and most of these allow you to calculate averages, sums, maximum, minimum, and count.

DynamoDB - CloudTrail

DynamoDB 包括 CloudTrail 集成。它从或为帐户中的 DynamoDB 捕获低级别 API 请求,并将日志文件发送到指定的 S3 存储桶。它针对来自控制台或 API 的调用。您可以使用这些数据来确定请求的来源、用户、时间戳等信息。

DynamoDB includes CloudTrail integration. It captures low-level API requests from or for DynamoDB in an account, and sends log files to a specified S3 bucket. It targets calls from the console or API. You can use this data to determine requests made and their source, user, timestamp, and more.

启用后,它会跟踪包含其他服务记录的日志文件中的动作。它支持八个动作和两个流 -

When enabled, it tracks actions in log files, which include other service records. It supports eight actions and two streams −

八个动作如下 -

The eight actions are as follows −

  1. CreateTable

  2. DeleteTable

  3. DescribeTable

  4. ListTables

  5. UpdateTable

  6. DescribeReservedCapacity

  7. DescribeReservedCapacityOfferings

  8. PurchaseReservedCapacityOfferings

而两个流如下 -

While, the two streams are −

  1. DescribeStream

  2. ListStreams

所有日志都包含有关提出请求的帐户的信息。您可以确定详细信息,例如根或 IAM 用户是否提出了请求,或者是否使用了临时凭证或联合凭证。

All the logs contain information about accounts making requests. You can determine detailed information like whether root or IAM users made the request, or whether with temporary credentials or federated.

日志文件会根据您指定的时间长度保留在存储中,并进行归档和删除设置。默认情况下,会创建加密的日志。您可以为新日志设置警报。您还可以在一个存储桶中整理跨区域和帐户的多个日志。

The log files remain in storage for however long you specify, with settings for archiving and deletion. The default creates encrypted logs. You can set alerts for new logs. You can also organize multiple logs, across regions and accounts, into a single bucket.

Interpreting Log Files

每个文件包含一个或多个条目。每个条目由多个 JSON 格式的事件组成。条目表示一个请求,并包含关联信息;不对顺序提供保证。

Each file contains a single or multiple entries. Each entry consists of multiple JSON format events. An entry represents a request, and includes associated information; with no guarantee of order.

您可以查看以下示例日志文件 -

You can review the following sample log file −

{"Records": [
   {
      "eventVersion": "5.05",
      "userIdentity": {
         "type": "AssumedRole",
         "principalId": "AKTTIOSZODNN8SAMPLE:jane",
         "arn": "arn:aws:sts::155522255533:assumed-role/users/jane",
         "accountId": "155522255533",
         "accessKeyId": "AKTTIOSZODNN8SAMPLE",

         "sessionContext": {
            "attributes": {
               "mfaAuthenticated": "false",
               "creationDate": "2016-05-11T19:01:01Z"
            },

            "sessionIssuer": {
               "type": "Role",
               "principalId": "AKTTI44ZZ6DHBSAMPLE",
               "arn": "arn:aws:iam::499955777666:role/admin-role",
               "accountId": "499955777666",
               "userName": "jill"
            }
         }
      },

      "eventTime": "2016-05-11T14:33:20Z",
      "eventSource": "dynamodb.amazonaws.com",
      "eventName": "DeleteTable",
      "awsRegion": "us-west-2",
      "sourceIPAddress": "192.0.2.0",
      "userAgent": "console.aws.amazon.com",
      "requestParameters": {"tableName": "Tools"},

      "responseElements": {"tableDescription": {
         "tableName": "Tools",
         "itemCount": 0,

         "provisionedThroughput": {
            "writeCapacityUnits": 25,
            "numberOfDecreasesToday": 0,
            "readCapacityUnits": 25
         },
         "tableStatus": "DELETING",
         "tableSizeBytes": 0
      }},
      "requestID": "4D89G7D98GF7G8A7DF78FG89AS7GFSO5AEMVJF66Q9ASUAAJG",
      "eventID": "a954451c-c2fc-4561-8aea-7a30ba1fdf52",
      "eventType": "AwsApiCall",
      "apiVersion": "2013-04-22",
      "recipientAccountId": "155522255533"
   }
]}

DynamoDB - MapReduce

亚马逊的弹性 MapReduce (EMR) 允许您快速高效地处理大数据。EMR 在 EC2 实例上运行 Apache Hadoop,但简化了流程。您可以利用 Apache Hive 通过 HiveQL 来查询映射归约作业流,该查询语言类似于 SQL。Apache Hive 是一种优化查询和应用程序的方法。

Amazon’s Elastic MapReduce (EMR) allows you to quickly and efficiently process big data. EMR runs Apache Hadoop on EC2 instances, but simplifies the process. You utilize Apache Hive to query map reduce job flows through HiveQL, a query language resembling SQL. Apache Hive serves as a way to optimize queries and your applications.

您可以使用管理控制台的 EMR 选项卡、EMR CLI、API 或 SDK 来启动作业流。您还可以选择交互式运行 Hive 或利用脚本。

You can use the EMR tab of the management console, the EMR CLI, an API, or an SDK to launch a job flow. You also have the option to run Hive interactively or utilize a script.

然而,EMR 读/写操作会影响吞吐量消耗,在大型请求中,它会使用退避算法的保护执行重试。此外,与其他操作和任务同时运行 EMR 可能会导致限制。

The EMR read/write operations impact throughput consumption, however, in large requests, it performs retries with the protection of a backoff algorithm. Also, running EMR concurrently with other operations and tasks may result in throttling.

DynamoDB/EMR 集成不支持二进制和二进制集属性。

The DynamoDB/EMR integration does not support binary and binary set attributes.

DynamoDB/EMR Integration Prerequisites

在使用 EMR 之前查看此必要项目清单 -

Review this checklist of necessary items before using EMR −

  1. An AWS account

  2. A populated table under the same account employed in EMR operations

  3. A custom Hive version with DynamoDB connectivity

  4. DynamoDB connectivity support

  5. An S3 bucket (optional)

  6. An SSH client (optional)

  7. An EC2 key pair (optional)

Hive Setup

在使用 EMR 之前,创建一个密钥对以交互方式运行 Hive。密钥对允许连接到 EC2 实例和作业流的主节点。

Before using EMR, create a key pair to run Hive in interactive mode. The key pair allows connection to EC2 instances and master nodes of job flows.

您可以通过以下步骤执行此操作 -

You can perform this by following the subsequent steps −

  1. Log in to the management console, and open the EC2 console located at https://console.aws.amazon.com/ec2/

  2. Select a region in the upper, right-hand portion of the console. Ensure the region matches the DynamoDB region.

  3. In the Navigation pane, select Key Pairs.

  4. Select Create Key Pair.

  5. In the Key Pair Name field, enter a name and select Create.

  6. Download the resulting private key file which uses the following format: filename.pem.

Note − 无法在没有密钥对的情况下连接到 EC2 实例。

Note − You cannot connect to EC2 instances without the key pair.

Hive Cluster

创建一个支持 Hive 的群集以运行 Hive。它将构建一个适用于 Hive 到 DynamoDB 连接的应用程序和基础设施必需的环境。

Create a hive-enabled cluster to run Hive. It builds the required environment of applications and infrastructure for a Hive-to-DynamoDB connection.

您可以通过执行以下步骤来执行此任务 −

You can perform this task by using the following steps −

  1. Access the EMR console.

  2. Select Create Cluster.

  3. In the creation screen, set the cluster configuration with a descriptive name for the cluster, select Yes for termination protection and check on Enabled for logging, an S3 destination for log folder S3 location, and Enabled for debugging.

  4. In the Software Configuration screen, ensure the fields hold Amazon for Hadoop distribution, the latest version for AMI version, a default Hive version for Applications to be Installed-Hive, and a default Pig version for Applications to be Installed-Pig.

  5. In the Hardware Configuration screen, ensure the fields hold Launch into EC2-Classic for Network, No Preference for EC2 Availability Zone, the default for Master-Amazon EC2 Instance Type, no check for Request Spot Instances, the default for Core-Amazon EC2 Instance Type, 2 for Count, no check for Request Spot Instances, the default for Task-Amazon EC2 Instance Type, 0 for Count, and no check for Request Spot Instances.

务必设置一个限制以提供足够的容量来防止群集故障。

Be sure to set a limit providing sufficient capacity to prevent cluster failure.

  1. In the Security and Access screen, ensure fields hold your key pair in EC2 key pair, No other IAM users in IAM user access, and Proceed without roles in IAM role.

  2. Review the Bootstrap Actions screen, but do not modify it.

  3. Review settings, and select Create Cluster when finished.

Summary 窗格出现在群集开始时。

A Summary pane appears on the start of the cluster.

Activate SSH Session

您需要一个活动 SSH 会话来连接到主节点并执行 CLI 操作。通过在 EMR 控制台中选择群集来找到主节点。它将主节点列为 Master Public DNS Name

You need an active the SSH session to connect to the master node and execute CLI operations. Locate the master node by selecting the cluster in the EMR console. It lists the master node as Master Public DNS Name.

如果没有 PuTTY,请安装它。然后启动 PuTTYgen 并选择 Load 。选择您的 PEM 文件并将其打开。PuTTYgen 将通知您导入成功。选择 Save private key 以采用 PuTTY 私钥格式 (PPK) 保存,选择 Yes 以在不使用口令的情况下保存。然后为 PuTTY 密钥输入一个名称,按 Save 并关闭 PuTTYgen。

Install PuTTY if you do not have it. Then launch PuTTYgen and select Load. Choose your PEM file, and open it. PuTTYgen will inform you of successful import. Select Save private key to save in PuTTY private key format (PPK), and choose Yes for saving without a pass phrase. Then enter a name for the PuTTY key, hit Save, and close PuTTYgen.

先启动 PuTTY,然后使用 PuTTY 连接到主节点。从“类别”列表中选择 Session 。在“主机名”字段中输入 hadoop@DNS。展开“SSH”下的“类别”列表,选择 Auth 。在控制选项屏幕中,选择 Browse 以进行用于验证的私钥文件。然后选择您的私钥文件并将其打开。选择 Yes 以响应弹出安全警报。

Use PuTTY to make a connection with the master node by first starting PuTTY. Choose Session from the Category list. Enter hadoop@DNS within the Host Name field. Expand Connection > SSH in the Category list, and choose Auth. In the controlling options screen, select Browse for Private key file for authentication. Then select your private key file and open it. Select Yes for the security alert pop-up.

连接到主节点后,会出现 Hadoop 命令提示符,这意味着您可以开始一个交互式 Hive 会话。

When connected to the master node, a Hadoop command prompt appears, which means you can begin an interactive Hive session.

Hive Table

Hive 作为一款数据仓库工具,可使用 HiveQL 查询 EMR 集群。之前的设置可提供一个有用的提示。通过输入“hive”并输入您希望执行的任何命令,互动式运行 Hive 命令。请参阅我们的 Hive 教程,以了解 Hive 的详细信息。

Hive serves as a data warehouse tool allowing queries on EMR clusters using HiveQL. The previous setups give you a working prompt. Run Hive commands interactively by simply entering “hive,” and then any commands you wish. See our Hive tutorial for more information on Hive.

DynamoDB - Table Activity

Amazon DynamoDB 流可让您跟踪和响应表内项目的更改。使用此功能创建一个应用程序,该应用程序可通过更新不同来源上的信息来响应这些更改。同步大型多用户系统中成千上万用户的相关信息。使用此功能向用户发送更新通知。其应用程序具有多样且重要的功能。Amazon DynamoDB 流作为用于实现此功能的主要工具。

DynamoDB streams enable you to track and respond to table item changes. Employ this functionality to create an application which responds to changes by updating information across sources. Synchronize data for thousands of users of a large, multi-user system. Use it to send notifications to users on updates. Its applications prove diverse and substantial. DynamoDB streams serve as the main tool used to achieve this functionality.

流记录按时间序列捕获表内项目的修改信息。它们最多保存这些数据 24 小时。应用程序使用这些数据几乎实时地查看原始项目和修改后的项目。

The streams capture time-ordered sequences containing item modifications within a table. They hold this data for a maximum of 24 hours. Applications use them to view the original and modified items, almost in real-time.

启用后的流可捕获所有修改。在任何 CRUD 操作中,Amazon DynamoDB 都会创建一个包含修改后的项目主键属性的流记录。您可以为流配置其它信息,例如修改前和修改后的镜像。

Streams enabled on a table capture all modifications. On any CRUD operation, DynamoDB creates a stream record with the primary key attributes of the modified items. You can configure streams for additional information such as before and after images.

流具有两项保障,即:

The Streams carry two guarantees −

  1. Each record appears one time in the stream and

  2. Each item modification results in the stream records of the same order as that of the modifications.

所有流都进行实时处理,这让您可将它们应用于应用程序中的相关功能。

All streams process in real-time to allow you to employ them for related functionality in applications.

Managing Streams

您可以在创建表时启用流。对于现有表,您可以禁用或更改流设置。流提供异步操作的功能,这意味着不会影响表的性能。

On table creation, you can enable a stream. Existing tables allow stream disabling or settings changes. Streams offer the feature of asynchronous operation, which means no table performance impact.

利用 AWS 管理控制台简化流管理。首先,导航到该控制台,然后选择 Tables 。在“概述”选项卡中,选择 Manage Stream 。在窗口中,选择在表数据修改时添加到流中的信息。输入所有设置后,选择 Enable

Utilize the AWS Management console for simple stream management. First, navigate to the console, and choose Tables. In the Overview tab, choose Manage Stream. Inside the window, select the information added to a stream on table data modifications. After entering all settings, select Enable.

如果您想禁用任何现有流,请选择 Manage Stream ,然后再选择 Disable

If you want to disable any existing streams, select Manage Stream, and then Disable.

您还可以使用 CreateTable 和 UpdateTable API 启用或更改流。使用 StreamSpecification 参数配置流。StreamEnabled 指定状态,对于启用为 true,对于禁用为 false。

You can also utilize the APIs CreateTable and UpdateTable to enable or change a stream. Use the parameter StreamSpecification to configure the stream. StreamEnabled specifies status, meaning true for enabled and false for disabled.

StreamViewType 指定添加到流中的信息:KEYS_ONLY、NEW_IMAGE、OLD_IMAGE 和 NEW_AND_OLD_IMAGES。

StreamViewType specifies information added to the stream: KEYS_ONLY, NEW_IMAGE, OLD_IMAGE, and NEW_AND_OLD_IMAGES.

Stream Reading

通过连接到端点并发出 API 请求来读取和处理流。每个流都包含多个流记录,而每个记录都作为单个修改存在,而此修改拥有流。流记录包括一个序号,用于显示发布顺序。记录属于称为分片的组。分片用作多个记录的容器,并且还包含用于访问和遍历记录所需的信息。24 小时后,记录会自动删除。

Read and process streams by connecting to an endpoint and making API requests. Each stream consists of stream records, and every record exists as a single modification which owns the stream. Stream records include a sequence number revealing publishing order. Records belong to groups also known as shards. Shards function as containers for several records, and also hold information needed for accessing and traversing records. After 24 hours, records automatically delete.

这些分片根据需要生成和删除,并且不会长时间存在。它们还会自动分成多个新分片,通常是为了响应写入活动激增。禁用流后,打开的分片会关闭。分片之间的层次关系意味着应用程序必须优先处理父分片,以保证正确的处理顺序。您可以使用 Kinesis Adapter 来自动执行此操作。

These Shards are generated and deleted as needed, and do not last long. They also divide into multiple new shards automatically, typically in response to write activity spikes. On stream disabling, open shards close. The hierarchical relationship between shards means applications must prioritize the parent shards for correct processing order. You can use Kinesis Adapter to automatically do this.

Note - 未造成任何更改的操作不会写入流记录。

Note − The operations resulting in no change do not write stream records.

访问和处理记录需要执行以下任务:

Accessing and processing records requires performing the following tasks −

  1. Determine the ARN of the target stream.

  2. Determine the shard(s) of the stream holding the target records.

  3. Access the shard(s) to retrieve the desired records.

Note −最多应同时有2个进程读取一个分片。如果超过2个进程,它可能会抑制来源。

Note − There should be a maximum of 2 processes reading a shard at once. If it exceeds 2 processes, then it can throttle the source.

可用的流API操作包括

The stream API actions available include

  1. ListStreams

  2. DescribeStream

  3. GetShardIterator

  4. GetRecords

您可以查看流读操作的以下示例:−

You can review the following example of the stream reading −

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBStreamsClient;

import com.amazonaws.services.dynamodbv2.model.AttributeAction;
import com.amazonaws.services.dynamodbv2.model.AttributeDefinition;
import com.amazonaws.services.dynamodbv2.model.AttributeValue;
import com.amazonaws.services.dynamodbv2.model.AttributeValueUpdate;

import com.amazonaws.services.dynamodbv2.model.CreateTableRequest;
import com.amazonaws.services.dynamodbv2.model.DescribeStreamRequest;
import com.amazonaws.services.dynamodbv2.model.DescribeStreamResult;
import com.amazonaws.services.dynamodbv2.model.DescribeTableResult;

import com.amazonaws.services.dynamodbv2.model.GetRecordsRequest;
import com.amazonaws.services.dynamodbv2.model.GetRecordsResult;
import com.amazonaws.services.dynamodbv2.model.GetShardIteratorRequest;
import com.amazonaws.services.dynamodbv2.model.GetShardIteratorResult;

import com.amazonaws.services.dynamodbv2.model.KeySchemaElement;
import com.amazonaws.services.dynamodbv2.model.KeyType;
import com.amazonaws.services.dynamodbv2.model.ProvisionedThroughput;
import com.amazonaws.services.dynamodbv2.model.Record;

import com.amazonaws.services.dynamodbv2.model.Shard;
import com.amazonaws.services.dynamodbv2.model.ShardIteratorType;
import com.amazonaws.services.dynamodbv2.model.StreamSpecification;
import com.amazonaws.services.dynamodbv2.model.StreamViewType;
import com.amazonaws.services.dynamodbv2.util.Tables;

public class StreamsExample {
   private static AmazonDynamoDBClient dynamoDBClient =
      new AmazonDynamoDBClient(new ProfileCredentialsProvider());
   private static AmazonDynamoDBStreamsClient streamsClient =
      new AmazonDynamoDBStreamsClient(new ProfileCredentialsProvider());

   public static void main(String args[]) {
      dynamoDBClient.setEndpoint("InsertDbEndpointHere");
      streamsClient.setEndpoint("InsertStreamEndpointHere");

      // table creation
      String tableName = "MyTestingTable";
      ArrayList<AttributeDefinition> attributeDefinitions =
         new ArrayList<AttributeDefinition>();

      attributeDefinitions.add(new AttributeDefinition()
         .withAttributeName("ID")
         .withAttributeType("N"));

      ArrayList<KeySchemaElement> keySchema = new
         ArrayList<KeySchemaElement>();

      keySchema.add(new KeySchemaElement()
         .withAttributeName("ID")
         .withKeyType(KeyType.HASH));                       //Partition key

      StreamSpecification streamSpecification = new StreamSpecification();
      streamSpecification.setStreamEnabled(true);
      streamSpecification.setStreamViewType(StreamViewType.NEW_AND_OLD_IMAGES);
      CreateTableRequest createTableRequest = new CreateTableRequest()
         .withTableName(tableName)
         .withKeySchema(keySchema)
         .withAttributeDefinitions(attributeDefinitions)
         .withProvisionedThroughput(new ProvisionedThroughput()
         .withReadCapacityUnits(1L)
         .withWriteCapacityUnits(1L))
         .withStreamSpecification(streamSpecification);

      System.out.println("Executing CreateTable for " + tableName);
      dynamoDBClient.createTable(createTableRequest);
      System.out.println("Creating " + tableName);

      try {
         Tables.awaitTableToBecomeActive(dynamoDBClient, tableName);
      } catch (InterruptedException e) {
         e.printStackTrace();
      }

      // Get the table's stream settings
      DescribeTableResult describeTableResult =
         dynamoDBClient.describeTable(tableName);

      String myStreamArn = describeTableResult.getTable().getLatestStreamArn();
      StreamSpecification myStreamSpec =
         describeTableResult.getTable().getStreamSpecification();

      System.out.println("Current stream ARN for " + tableName + ": "+ myStreamArn);
      System.out.println("Stream enabled: "+ myStreamSpec.getStreamEnabled());
      System.out.println("Update view type: "+ myStreamSpec.getStreamViewType());

      // Add an item
      int numChanges = 0;
      System.out.println("Making some changes to table data");
      Map<String, AttributeValue> item = new HashMap<String, AttributeValue>();
      item.put("ID", new AttributeValue().withN("222"));
      item.put("Alert", new AttributeValue().withS("item!"));
      dynamoDBClient.putItem(tableName, item);
      numChanges++;

      // Update the item
      Map<String, AttributeValue> key = new HashMap<String, AttributeValue>();
      key.put("ID", new AttributeValue().withN("222"));
      Map<String, AttributeValueUpdate> attributeUpdates =
      new HashMap<String, AttributeValueUpdate>();

      attributeUpdates.put("Alert", new AttributeValueUpdate()
         .withAction(AttributeAction.PUT)
         .withValue(new AttributeValue().withS("modified item")));

      dynamoDBClient.updateItem(tableName, key, attributeUpdates);
      numChanges++;

      // Delete the item
      dynamoDBClient.deleteItem(tableName, key);
      numChanges++;

      // Get stream shards
      DescribeStreamResult describeStreamResult =
      streamsClient.describeStream(new DescribeStreamRequest()
         .withStreamArn(myStreamArn));
      String streamArn =
         describeStreamResult.getStreamDescription().getStreamArn();
      List<Shard> shards =
         describeStreamResult.getStreamDescription().getShards();

      // Process shards
      for (Shard shard : shards) {
         String shardId = shard.getShardId();
         System.out.println("Processing " + shardId + " in "+ streamArn);

         // Get shard iterator
         GetShardIteratorRequest getShardIteratorRequest = new
            GetShardIteratorRequest()
            .withStreamArn(myStreamArn)
            .withShardId(shardId)
            .withShardIteratorType(ShardIteratorType.TRIM_HORIZON);

         GetShardIteratorResult getShardIteratorResult =
            streamsClient.getShardIterator(getShardIteratorRequest);
         String nextItr = getShardIteratorResult.getShardIterator();

         while (nextItr != null && numChanges > 0) {
            // Read data records with iterator
            GetRecordsResult getRecordsResult =
               streamsClient.getRecords(new GetRecordsRequest().
               withShardIterator(nextItr));

            List<Record> records = getRecordsResult.getRecords();
            System.out.println("Pulling records...");

            for (Record record : records) {
               System.out.println(record);
               numChanges--;
            }
            nextItr = getRecordsResult.getNextShardIterator();
         }
      }
   }
}

DynamoDB - Error Handling

在请求处理失败时,DynamoDB 会引发错误。每个错误都包含以下组件:HTTP 状态码、异常名称和消息。错误管理依赖于传播错误的SDK或您的代码。

On unsuccessful processing of a request, DynamoDB throws an error. Each error consists of the following components: HTTP status code, exception name, and message. Error management rests on your SDK, which propagates errors, or your own code.

Codes and Messages

异常会被划分为不同的HTTP首部状态码。4xx和5xx持有与请求问题和AWS相关的错误。

Exceptions fall into different HTTP header status codes. The 4xx and 5xx hold errors related to request issues and AWS.

HTTP 4xx类别的以下异常包括:−

A selection of exceptions in the HTTP 4xx category are as follows −

  1. AccessDeniedException − The client failed to sign the request correctly.

  2. ConditionalCheckFailedException − A condition evaluated to false.

  3. IncompleteSignatureException − The request included an incomplete signature.

HTTP 5xx 类别的异常如下所示:−

Exceptions in the HTTP 5xx category are as follows −

  1. Internal Server Error

  2. Service Unavailable

Retries and Backoff Algorithms

错误来自各种来源,例如服务器、交换机、负载平衡器以及其他结构和系统。通用解决方案包括支持可靠性的简单重试。所有 SDK 都自动包含此逻辑,并且您可以设置重试参数以满足您的应用程序需求。

Errors come from a variety of sources such as servers, switches, load balancers, and other pieces of structures and systems. Common solutions consist of simple retries, which supports reliability. All SDKs include this logic automatically, and you can set retry parameters to suit your application needs.

For example −Java 提供了一个 maxErrorRetry 值来停止重试。

For example − Java offers a maxErrorRetry value to stop retries.

Amazon 建议除了重试之外,还使用退避解决方案,以控制流。这包括逐渐增加重试之间的等待时间,并最终在相当短的时间后停止重试。请注意,SDK 会自动执行重试,但不会执行指数退避。

Amazon recommends using a backoff solution in addition to retries in order to control flow. This consists of progressively increasing wait periods between retries and eventually stopping after a fairly short period. Note SDKs perform automatic retries, but not exponential backoff.

以下程序是重试退避的一个示例:−

The following program is an example of the retry backoff −

public enum Results {
   SUCCESS,
   NOT_READY,
   THROTTLED,
   SERVER_ERROR
}
public static void DoAndWaitExample() {
   try {
      // asynchronous operation.
      long token = asyncOperation();
      int retries = 0;
      boolean retry = false;

      do {
         long waitTime = Math.min(getWaitTime(retries), MAX_WAIT_INTERVAL);
         System.out.print(waitTime + "\n");

         // Pause for result
         Thread.sleep(waitTime);

         // Get result
         Results result = getAsyncOperationResult(token);

         if (Results.SUCCESS == result) {
            retry = false;
         } else if (Results.NOT_READY == result) {
            retry = true;
         } else if (Results.THROTTLED == result) {
            retry = true;
         } else if (Results.SERVER_ERROR == result) {
            retry = true;
         } else {

            // stop on other error
            retry = false;
         }
      } while (retry && (retries++ < MAX_RETRIES));
   }
   catch (Exception ex) {
   }
}
public static long getWaitTime(int retryCount) {
   long waitTime = ((long) Math.pow(3, retryCount) * 100L);
   return waitTime;
}

DynamoDB - Best Practices

在使用各种来源和元素时,某些做法可以优化代码,防止错误并最大限度地降低吞吐量的成本。

Certain practices optimize code, prevent errors, and minimize throughput cost when working with various sources and elements.

以下是 DynamoDB 中一些最重要且使用最广泛的最佳实践。

The following are some of the most important and commonly used best practices in DynamoDB.

Tables

表的分布意味着最佳方法是在所有表项目中均匀地分摊读写活动。

The distribution of tables means the best approaches spread read/write activity evenly across all table items.

目标是均匀地访问表项目上的数据。最佳吞吐量使用取决于主键选择和项目工作负载模式。在分区键值之间均匀地分摊工作负载。避免使用少量大量使用的分区键值等。选择像大量不同的分区键值这样的更优选择。

Aim for uniform data access on table items. Optimal throughput usage rests on primary key selection and item workload patterns. Spread the workload evenly across partition key values. Avoid things like a small amount of heavily used partition key values. Opt for better choices like large quantities of distinct partition key values.

了解分区行为。评估通过 DynamoDB 自动分配的分区。

Gain an understanding of partition behavior. Estimate partitions automatically allocated by DynamoDB.

DynamoDB 提供突发吞吐量使用,它为“突发”功率保留未使用的吞吐量。避免频繁使用此选项,因为突发会快速消耗大量吞吐量;此外,它并不是一个可靠的资源。

DynamoDB offers burst throughput usage, which reserves unused throughput for “bursts” of power. Avoid heavy use of this option because bursts consume large amounts of throughput quickly; furthermore, it does not prove a reliable resource.

在上传时,分布数据以获得更好的性能。通过并发上传到所有已分配的服务器来实施此操作。

On uploads, distribute data in order to achieve better performance. Implement this by uploading to all allocated servers concurrently.

缓存常用项,将读取活动卸载到缓存而不是数据库。

Cache frequently used items to offload read activity to the cache rather than the database.

Items

限制、性能、大小和访问成本仍然是项目中最受关注的问题。选择一对多表。删除属性并划分表以匹配访问模式。通过这种简单的方法,您可以显着提高效率。

Throttling, performance, size, and access costs remain the biggest concerns with items. Opt for one-to-many tables. Remove attributes and divide tables to match access patterns. You can improve efficiency dramatically through this simple approach.

将大值压缩后再存储。利用标准压缩工具。为 S3 等大属性值使用备用存储。您可以将对象存储在 S3 中,并在项目中存储一个标识符。

Compress large values prior to storing them. Utilize standard compression tools. Use alternate storage for large attribute values such as S3. You can store the object in S3, and an identifier in the item.

通过虚拟项目块在多个项目中分布大属性。这提供了解决项目大小限制的方法。

Distribute large attributes across several items through virtual item pieces. This provides a workaround for the limitations of item size.

Queries and Scans

查询和扫描主要面临吞吐量消耗方面的挑战。避免突发,这通常是由切换到强一致性读取等操作导致的。以低资源消耗的方式使用并行扫描(即没有限制的后台函数)。此外,仅将它们与大型表一起使用,并且在您未充分利用吞吐量或扫描操作性能不佳的情况下使用它们。

Queries and scans mainly suffer from throughput consumption challenges. Avoid bursts, which typically result from things like switching to a strongly consistent read. Use parallel scans in a low-resource way (i.e., background function with no throttling). Furthermore, only employ them with large tables, and situations where you do not fully utilize throughput or scan operations offer poor performance.

Local Secondary Indices

索引在吞吐量和存储成本以及查询效率方面存在问题。除非您经常查询属性,否则请避免建立索引。在投影中,做出明智选择,因为它们会使索引变得臃肿。仅选择那些使用频繁的。

Indexes present issues in the areas of throughput and storage costs, and the efficiency of queries. Avoid indexing unless you query the attributes often. In projections, choose wisely because they bloat indexes. Select only those heavily used.

使用稀疏索引,即排序键在所有表项目中均未出现的索引。它们有利于针对大多数表项目中不存在的属性进行的查询。

Utilize sparse indexes, meaning indexes in which sort keys do not appear in all table items. They benefit queries on attributes not present in most table items.

注意项目集合(所有表项目及其索引)的扩展。添加/更新操作会导致表和索引增长,并且集合的限制仍然为 10GB。

Pay attention to the item collection (all table items and their indices) expansion. Add/update operations cause both tables and indexes to grow, and 10GB remains the limit for collections.

Global Secondary Indices

索引在吞吐量和存储成本以及查询效率方面存在问题。选择键属性传播,它就像表中的读/写传播一样,提供工作负载的均匀性。选择可以均匀分布数据的属性。另外,使用稀疏索引。

Indexes present issues in the areas of throughput and storage costs, and the efficiency of queries. Opt for key attributes spreading, which like read/write spreading in tables provides workload uniformity. Choose attributes which evenly spread data. Also, utilize sparse indexes.

利用全局二级索引对请求少量数据的查询进行快速搜索。

Exploit global secondary indices for fast searches in queries requesting a modest amount of data.