Dynamodb 简明教程
DynamoDB - Local Secondary Indexes
一些应用程序仅用主键执行查询,但某些情况可受益于备用排序密钥。通过创建单个或多个本地二级索引让应用程序做出选择。
Some applications only perform queries with the primary key, but some situations benefit from an alternate sort key. Allow your application a choice by creating a single or multiple local secondary indexes.
复杂的数据访问要求,例如梳理数百万个项,需要执行更有效的查询/扫描。本地二级索引为分区密钥值提供备用排序密钥。它们还保存全部或部分表格属性的副本。它们按表格分区密钥组织数据,但使用不同的排序密钥。
Complex data access requirements, such as combing millions of items, make it necessary to perform more efficient queries/scans. Local secondary indices provide an alternate sort key for a partition key value. They also hold copies of all or some table attributes. They organize data by table partition key, but use a different sort key.
使用本地二级索引无需对整个表进行扫描,并允许使用排序键进行简单快捷的查询。
Using a local secondary index removes the need for a whole table scan, and allows a simple and quick query using a sort key.
所有本地二级索引都必须满足特定条件——
All the local secondary indexes must satisfy certain conditions −
-
Identical partition key and source table partition key.
-
A sort key of only one scalar attribute.
-
Projection of the source table sort key acting as a non-key attribute.
所有本地二级索引自动持有父表的 partition key 和 sort key。在查询中,这意味着可以有效收集投射的属性,也可以检索未投射的属性。
All the local secondary indexes automatically hold partition and sort keys from parent tables. In queries, this means efficient gathering of projected attributes, and also retrieval of attributes not projected.
本地二级索引的存储限制仍然是每个 partition key 值 10GB,其中包括所有表项和共享 partition key 值的索引项。
The storage limit for a local secondary index remains 10GB per partition key value, which includes all table items, and index items sharing a partition key value.
Projecting an Attribute
由于复杂性,某些操作需要额外的读取/获取。这些操作可能消耗大量吞吐量。通过隔离这些属性,投射可避免代价高昂的获取并执行丰富查询。记住,投射由复制到二级索引的属性组成。
Some operations require excess reads/fetching due to complexity. These operations can consume substantial throughput. Projection allows you to avoid costly fetching and perform rich queries by isolating these attributes. Remember projections consist of attributes copied into a secondary index.
在创建二级索引时,可以指定投射的属性。回想一下 DynamoDB 提供的三个选项: KEYS_ONLY, INCLUDE, and ALL 。
When making a secondary index, you specify the attributes projected. Recall the three options provided by DynamoDB: KEYS_ONLY, INCLUDE, and ALL.
在对投射中的某些属性进行选择时,考虑相关的成本权衡——
When opting for certain attributes in projection, consider the associated cost tradeoffs −
-
If you project only a small set of necessary attributes, you dramatically reduce the storage costs.
-
If you project frequently accessed non-key attributes, you offset scan costs with storage costs.
-
If you project most or all non-key attributes, this maximizes flexibility and reduces throughput (no retrievals); however, storage costs rise.
-
If you project KEYS_ONLY for frequent writes/updates and infrequent queries, it minimizes size, but maintains query preparation.
Local Secondary Index Creation
使用 CreateTable 的 LocalSecondaryIndex 参数可以创建单个或多个本地二级索引。你必须为排序键指定一个非键属性。在创建表时,你创建本地二级索引。在删除时,你删除这些索引。
Use the LocalSecondaryIndex parameter of CreateTable to make a single or multiple local secondary indexes. You must specify one non-key attribute for the sort key. On table creation, you create local secondary indices. On deletion, you delete these indexes.
具有本地二级索引的表的每个分区键值的大小必须遵守 10GB 的限制,但可以存储任意数量的项。
Tables with a local secondary index must obey a limit of 10GB in size per partition key value, but can store any amount of items.
Local Secondary Index Queries and Scans
当索引中的多个项共享排序键值时,在本地二级索引上执行的查询操作会返回所有具有匹配分区键值的项。匹配的项不会按特定顺序返回。对本地二级索引的查询使用最终一致性或强一致性,其中强一致性读取可传递最新的值。
A query operation on local secondary indexes returns all items with a matching partition key value when multiple items in the index share sort key values. Matching items do not return in a certain order. Queries for local secondary indexes use either eventual or strong consistency, with strongly consistent reads delivering the latest values.
一个扫描操作返回全部的二级本地索引数据。扫描要求提供表名和索引名,并允许使用过滤器表达式来丢弃数据。
A scan operation returns all local secondary index data. Scans require you to provide a table and index name, and allow the use of a filter expression to discard data.
Item Writing
在创建本地二级索引时,可以指定排序键属性及其数据类型。如果该项会定义索引键的属性,当写入该项时,其类型必须与键模式的数据类型匹配。
On creation of a local secondary index, you specify a sort key attribute and its data type. When you write an item, its type must match the data type of the key schema if the item defines an attribute of an index key.
DynamoDB 不对表项和本地二级索引项征收一对一关系要求。带有多个本地二级索引的表会比具有较少索引的表产生更高的写入成本。
DynamoDB imposes no one-to-one relationship requirements on table items and local secondary index items. The tables with multiple local secondary indexes carry higher write costs than those with less.
Throughput Considerations in Local Secondary Indexes
查询的读取容量消耗取决于数据访问的本质。查询使用最终或强一致性,强一致性读取使用一个单位,而最终一致性读取使用 0.5 个单位。
Read capacity consumption of a query depends on the nature of data access. Queries use either eventual or strong consistency, with strongly consistent reads using one unit compared to half a unit in eventually consistent reads.
结果限制包括 1MB 的最大值。结果大小来源于匹配索引项大小的总和(向上取整到最接近的 4KB)以及匹配表项大小的总和(同样向上取整到最接近的 4KB)。
Result limitations include a 1MB size maximum. Result sizes come from the sum of matching index item size rounded up to the nearest 4KB, and matching table item size also rounded up to the nearest 4KB.
写入容量消耗保持在预置单位内。通过找到表写入中消耗的单位总和及更新索引中消耗的单位总和来计算总预置成本。
The write capacity consumption remains within provisioned units. Calculate the total provisioned cost by finding the sum of consumed units in table writing and consumed units in updating indices.
你还可以考虑影响成本的关键因素,其中一些因素可以是 −
You can also consider the key factors influencing cost, some of which can be −
-
When you write an item defining an indexed attribute or update an item to define an undefined indexed attribute, a single write operation occurs.
-
When a table update changes an indexed key attribute value, two writes occur to delete and then – add an item.
-
When a write causes the deletion of an indexed attribute, one write occurs to remove the old item projection.
-
When an item does not exist within the index prior to or after an update, no writes occur.
Local Secondary Index Storage
在表项写入中,DynamoDB 会自动复制正确的属性集到所需的本地二级索引。这会产生账户费用。所用的空间来自于表主密钥字节大小、索引键属性字节大小、任何当前投影属性字节大小以及每个索引项的开销中 100 个字节的总和。
On a table item write, DynamoDB automatically copies the right attribute set to the required local secondary indexes. This charges your account. The space used results from the sum of table primary key byte size, index key attribute byte size, any present projected attribute byte size, and 100 bytes in overhead for each index item.
估算存储可以通过估算平均索引项大小并乘以表项数量来获得。
The estimate storage is got by estimating average index item size and multiplying by table item quantity.
Using Java to Work with Local Secondary Indexes
首先创建一个 DynamoDB 类实例,来创建一个本地二级索引。然后,使用必要的请求信息创建一个 CreateTableRequest 类实例。最后,使用 createTable 方法。
Create a local secondary index by first creating a DynamoDB class instance. Then, create a CreateTableRequest class instance with necessary request information. Finally, use the createTable method.
Example
DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
new ProfileCredentialsProvider()));
String tableName = "Tools";
CreateTableRequest createTableRequest = new
CreateTableRequest().withTableName(tableName);
//Provisioned Throughput
createTableRequest.setProvisionedThroughput (
new ProvisionedThroughput()
.withReadCapacityUnits((long)5)
.withWriteCapacityUnits(( long)5));
//Attributes
ArrayList<AttributeDefinition> attributeDefinitions =
new ArrayList<AttributeDefinition>();
attributeDefinitions.add(new AttributeDefinition()
.withAttributeName("Make")
.withAttributeType("S"));
attributeDefinitions.add(new AttributeDefinition()
.withAttributeName("Model")
.withAttributeType("S"));
attributeDefinitions.add(new AttributeDefinition()
.withAttributeName("Line")
.withAttributeType("S"));
createTableRequest.setAttributeDefinitions(attributeDefinitions);
//Key Schema
ArrayList<KeySchemaElement> tableKeySchema = new
ArrayList<KeySchemaElement>();
tableKeySchema.add(new KeySchemaElement()
.withAttributeName("Make")
.withKeyType(KeyType.HASH)); //Partition key
tableKeySchema.add(new KeySchemaElement()
.withAttributeName("Model")
.withKeyType(KeyType.RANGE)); //Sort key
createTableRequest.setKeySchema(tableKeySchema);
ArrayList<KeySchemaElement> indexKeySchema = new
ArrayList<KeySchemaElement>();
indexKeySchema.add(new KeySchemaElement()
.withAttributeName("Make")
.withKeyType(KeyType.HASH)); //Partition key
indexKeySchema.add(new KeySchemaElement()
.withAttributeName("Line")
.withKeyType(KeyType.RANGE)); //Sort key
Projection projection = new Projection()
.withProjectionType(ProjectionType.INCLUDE);
ArrayList<String> nonKeyAttributes = new ArrayList<String>();
nonKeyAttributes.add("Type");
nonKeyAttributes.add("Year");
projection.setNonKeyAttributes(nonKeyAttributes);
LocalSecondaryIndex localSecondaryIndex = new LocalSecondaryIndex()
.withIndexName("ModelIndex")
.withKeySchema(indexKeySchema)
.withProjection(p rojection);
ArrayList<LocalSecondaryIndex> localSecondaryIndexes = new
ArrayList<LocalSecondaryIndex>();
localSecondaryIndexes.add(localSecondaryIndex);
createTableRequest.setLocalSecondaryIndexes(localSecondaryIndexes);
Table table = dynamoDB.createTable(createTableRequest);
System.out.println(table.getDescription());
使用 describe 方法来检索有关本地二级索引的信息。只需创建一个 DynamoDB 类实例,创建一个 Table 类实例,并把表传递到 describe 方法。
Retrieve information about a local secondary index with the describe method. Simply create a DynamoDB class instance, create a Table class instance, and pass the table to the describe method.
Example
DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
new ProfileCredentialsProvider()));
String tableName = "Tools";
Table table = dynamoDB.getTable(tableName);
TableDescription tableDescription = table.describe();
List<LocalSecondaryIndexDescription> localSecondaryIndexes =
tableDescription.getLocalSecondaryIndexes();
Iterator<LocalSecondaryIndexDescription> lsiIter =
localSecondaryIndexes.iterator();
while (lsiIter.hasNext()) {
LocalSecondaryIndexDescription lsiDescription = lsiIter.next();
System.out.println("Index info " + lsiDescription.getIndexName() + ":");
Iterator<KeySchemaElement> kseIter = lsiDescription.getKeySchema().iterator();
while (kseIter.hasNext()) {
KeySchemaElement kse = kseIter.next();
System.out.printf("\t%s: %s\n", kse.getAttributeName(), kse.getKeyType());
}
Projection projection = lsiDescription.getProjection();
System.out.println("\tProjection type: " + projection.getProjectionType());
if (projection.getProjectionType().toString().equals("INCLUDE")) {
System.out.println("\t\tNon-key projected attributes: " +
projection.getNonKeyAttributes());
}
}
通过使用与表查询相同的步骤来执行查询。只需创建一个 DynamoDB 类实例、一个 Table 类实例、一个 Index 类实例、一个查询对象,并使用 query 方法。
Perform a query by using the same steps as a table query. Merely create a DynamoDB class instance, a Table class instance, an Index class instance, a query object, and utilize the query method.
Example
DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
new ProfileCredentialsProvider()));
String tableName = "Tools";
Table table = dynamoDB.getTable(tableName);
Index index = table.getIndex("LineIndex");
QuerySpec spec = new QuerySpec()
.withKeyConditionExpression("Make = :v_make and Line = :v_line")
.withValueMap(new ValueMap()
.withString(":v_make", "Depault")
.withString(":v_line", "SuperSawz"));
ItemCollection<QueryOutcome> items = index.query(spec);
Iterator<Item> itemsIter = items.iterator();
while (itemsIter.hasNext()) {
Item item = itemsIter.next();
System.out.println(item.toJSONPretty());
}
你还可以查看以下示例。
You can also review the following example.
Note −以下示例可能假定已经创建了数据源。在尝试执行之前,获取支持库并创建必要的数据源(具有所需特征的表或其他引用源)。
Note − The following example may assume a previously created data source. Before attempting to execute, acquire supporting libraries and create necessary data sources (tables with required characteristics, or other referenced sources).
以下示例还使用了 Eclipse IDE、AWS 凭证文件以及 Eclipse AWS Java 项目中的 AWS Toolkit。
The following example also uses Eclipse IDE, an AWS credentials file, and the AWS Toolkit within an Eclipse AWS Java Project.
Example
import java.util.ArrayList;
import java.util.Iterator;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Index;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.ItemCollection;
import com.amazonaws.services.dynamodbv2.document.PutItemOutcome;
import com.amazonaws.services.dynamodbv2.document.QueryOutcome;
import com.amazonaws.services.dynamodbv2.document.Table;
import com.amazonaws.services.dynamodbv2.document.spec.QuerySpec;
import com.amazonaws.services.dynamodbv2.document.utils.ValueMap;
import com.amazonaws.services.dynamodbv2.model.AttributeDefinition;
import com.amazonaws.services.dynamodbv2.model.CreateTableRequest;
import com.amazonaws.services.dynamodbv2.model.KeySchemaElement;
import com.amazonaws.services.dynamodbv2.model.KeyType;
import com.amazonaws.services.dynamodbv2.model.LocalSecondaryIndex;
import com.amazonaws.services.dynamodbv2.model.Projection;
import com.amazonaws.services.dynamodbv2.model.ProjectionType;
import com.amazonaws.services.dynamodbv2.model.ProvisionedThroughput;
import com.amazonaws.services.dynamodbv2.model.ReturnConsumedCapacity;
import com.amazonaws.services.dynamodbv2.model.Select;
public class LocalSecondaryIndexSample {
static DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
new ProfileCredentialsProvider()));
public static String tableName = "ProductOrders";
public static void main(String[] args) throws Exception {
createTable();
query(null);
query("IsOpenIndex");
query("OrderCreationDateIndex");
}
public static void createTable() {
CreateTableRequest createTableRequest = new CreateTableRequest()
.withTableName(tableName)
.withProvisionedThroughput(new ProvisionedThroughput()
.withReadCapacityUnits((long) 1)
.withWriteCapacityUnits((long) 1));
// Table partition and sort keys attributes
ArrayList<AttributeDefinition> attributeDefinitions = new
ArrayList<AttributeDefinition>();
attributeDefinitions.add(new AttributeDefinition()
.withAttributeName("CustomerID")
.withAttributeType("S"));
attributeDefinitions.add(new AttributeDefinition()
.withAttributeName("OrderID")
.withAttributeType("N"));
// Index primary key attributes
attributeDefinitions.add(new AttributeDefinition()
.withAttributeName("OrderDate")
.withAttributeType("N"));
attributeDefinitions.add(new AttributeDefinition()
.withAttributeName("OpenStatus")
.withAttributeType("N"));
createTableRequest.setAttributeDefinitions(attributeDefinitions);
// Table key schema
ArrayList<KeySchemaElement> tableKeySchema = new
ArrayList<KeySchemaElement>();
tableKeySchema.add(new KeySchemaElement()
.withAttributeName("CustomerID")
.withKeyType(KeyType.HASH)); //Partition key
tableKeySchema.add(new KeySchemaElement()
.withAttributeName("OrderID")
.withKeyType(KeyType.RANGE)); //Sort key
createTableRequest.setKeySchema(tableKeySchema);
ArrayList<LocalSecondaryIndex> localSecondaryIndexes = new
ArrayList<LocalSecondaryIndex>();
// OrderDateIndex
LocalSecondaryIndex orderDateIndex = new LocalSecondaryIndex()
.withIndexName("OrderDateIndex");
// OrderDateIndex key schema
ArrayList<KeySchemaElement> indexKeySchema = new
ArrayList<KeySchemaElement>();
indexKeySchema.add(new KeySchemaElement()
.withAttributeName("CustomerID")
.withKeyType(KeyType.HASH)); //Partition key
indexKeySchema.add(new KeySchemaElement()
.withAttributeName("OrderDate")
.withKeyType(KeyType.RANGE)); //Sort key
orderDateIndex.setKeySchema(indexKeySchema);
// OrderCreationDateIndex projection w/attributes list
Projection projection = new Projection()
.withProjectionType(ProjectionType.INCLUDE);
ArrayList<String> nonKeyAttributes = new ArrayList<String>();
nonKeyAttributes.add("ProdCat");
nonKeyAttributes.add("ProdNomenclature");
projection.setNonKeyAttributes(nonKeyAttributes);
orderCreationDateIndex.setProjection(projection);
localSecondaryIndexes.add(orderDateIndex);
// IsOpenIndex
LocalSecondaryIndex isOpenIndex = new LocalSecondaryIndex()
.withIndexName("IsOpenIndex");
// OpenStatusIndex key schema
indexKeySchema = new ArrayList<KeySchemaElement>();
indexKeySchema.add(new KeySchemaElement()
.withAttributeName("CustomerID")
.withKeyType(KeyType.HASH)); //Partition key
indexKeySchema.add(new KeySchemaElement()
.withAttributeName("OpenStatus")
.withKeyType(KeyType.RANGE)); //Sort key
// OpenStatusIndex projection
projection = new Projection() .withProjectionType(ProjectionType.ALL);
OpenStatusIndex.setKeySchema(indexKeySchema);
OpenStatusIndex.setProjection(projection);
localSecondaryIndexes.add(OpenStatusIndex);
// Put definitions in CreateTable request
createTableRequest.setLocalSecondaryIndexes(localSecondaryIndexes);
System.out.println("Spawning table " + tableName + "...");
System.out.println(dynamoDB.createTable(createTableRequest));
// Pause for ACTIVE status
System.out.println("Waiting for ACTIVE table:" + tableName);
try {
Table table = dynamoDB.getTable(tableName);
table.waitForActive();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
public static void query(String indexName) {
Table table = dynamoDB.getTable(tableName);
System.out.println("\n*************************************************\n");
System.out.println("Executing query on" + tableName);
QuerySpec querySpec = new QuerySpec()
.withConsistentRead(true)
.withScanIndexForward(true)
.withReturnConsumedCapacity(ReturnConsumedCapacity.TOTAL);
if (indexName == "OpenStatusIndex") {
System.out.println("\nEmploying index: '" + indexName
+ "' open orders for this customer.");
System.out.println(
"Returns only user-specified attribute list\n");
Index index = table.getIndex(indexName);
querySpec.withKeyConditionExpression("CustomerID = :v_custmid and
OpenStatus = :v_openstat")
.withValueMap(new ValueMap()
.withString(":v_custmid", "jane@sample.com")
.withNumber(":v_openstat", 1));
querySpec.withProjectionExpression(
"OrderDate, ProdCat, ProdNomenclature, OrderStatus");
ItemCollection<QueryOutcome> items = index.query(querySpec);
Iterator<Item> iterator = items.iterator();
System.out.println("Printing query results...");
while (iterator.hasNext()) {
System.out.println(iterator.next().toJSONPretty());
}
} else if (indexName == "OrderDateIndex") {
System.out.println("\nUsing index: '" + indexName
+ "': this customer's orders placed after 05/22/2016.");
System.out.println("Projected attributes are returned\n");
Index index = table.getIndex(indexName);
querySpec.withKeyConditionExpression("CustomerID = :v_custmid and OrderDate
>= :v_ordrdate")
.withValueMap(new ValueMap()
.withString(":v_custmid", "jane@sample.com")
.withNumber(":v_ordrdate", 20160522));
querySpec.withSelect(Select.ALL_PROJECTED_ATTRIBUTES);
ItemCollection<QueryOutcome> items = index.query(querySpec);
Iterator<Item> iterator = items.iterator();
System.out.println("Printing query results...");
while (iterator.hasNext()) {
System.out.println(iterator.next().toJSONPretty());
}
} else {
System.out.println("\nNo index: All Jane's orders by OrderID:\n");
querySpec.withKeyConditionExpression("CustomerID = :v_custmid")
.withValueMap(new ValueMap()
.withString(":v_custmid", "jane@example.com"));
ItemCollection<QueryOutcome> items = table.query(querySpec);
Iterator<Item> iterator = items.iterator();
System.out.println("Printing query results...");
while (iterator.hasNext()) {
System.out.println(iterator.next().toJSONPretty());
}
}
}
}