我们正在寻找添加运动型流和运动式消防方,以将数据从我们的DynamoDB操作数据存储迁移到S3。
我创建了Kinesis流和Kinesis Firehose交付流,以将数据发送到S3存储桶。所有插入,修改和删除事件都将被捕获,转换并添加到S3存储桶中,并使用数据前缀/[Year]/[montr]/[Day]捕获。
我的问题是在启用运动动力学之前围绕DynamoDB中的数据。将数据迁移到S3的最佳方法是什么?我知道您可以从DynamoDB表中进行S3导出,但这将数据放入预定义的文件夹中。
这里对最佳方法有任何想法吗?
We are looking to add Kinesis Streams and Kinesis Firehose to migrate data from our DynamoDB operational data store to S3.
I have created the Kinesis Stream and Kinesis Firehose Delivery Stream to send the data to an S3 bucket. All Insert, Modified and Remove events are being captured, transformed and added to the S3 bucket with a prefix of data/[YEAR]/[MONTH]/[DAY].
The question I have is around the data in the DynamoDB prior to enabling Kinesis. What is the best way to migrate the data to S3. I understand that you can do an Export to S3 from the DynamoDB table but that puts data into a predefined folder.
Any idea on the best approach here?
发布评论
评论(1)
DynamoDB流和DynamoDB导出的格式不同,因为它们的用例略有不同。但是,可以从两者中创建一个单一视图。如果您想对从DynamoDB导出到S3的数据上运行分析查询,则可能要使用athena作为SQL引擎。
The formats of the DynamoDB Stream and the DynamoDB Export are different, as they are serving slightly different use cases. Nevertheless, it is possible to create a single view from both. If you want to run analytical queries on the data that you exported from DynamoDB into S3, you probably want to use Athena as your SQL engine.