AWS DAX性能问题与表扫描
嗨,我正在研究一个项目,该项目需要将所有Dyanamo DB文档带入内存中。我将使用table.scan()
boto3方法,该方法对于所有10k记录几乎需要33秒。
我已经配置了DAX并将其用于表扫描,该扫描需要将近42秒,具有相同的10K记录,具有相同的Lambda配置。我尝试了多次结果相同。
我尝试了以下代码:
daxclient = amazondax.AmazonDaxClient.resource(endpoint_url="...")
table = daxclient.Table('table_name')
start_time = time.perf_counter()
retry = True
while retry:
try:
response = table.scan(TableName ="table_name")
retry = 'LastEvaluatedKey' in response
scan_args['ExclusiveStartKey'] = response.get('LastEvaluatedKey')
except Exception as e:
print(e)
print(time.perf_counter()-start_time)
我尝试了boto3 getItem()
方法,这变得更快,就像第一次需要0.4秒,然后需要0.01秒。
不知道为什么它不使用表扫描方法。
请建议。
Hi I am working on an project that requires to bring all dyanamo db document in memory. I will be using table.scan()
boto3 method which nearly takes 33 seconds for all 10k records.
I have configured the DAX and using it for table scan, which takes nearly the 42 seconds with same 10k records with same lambda configuration. I tried multiple times results are same.
I tried below code :
daxclient = amazondax.AmazonDaxClient.resource(endpoint_url="...")
table = daxclient.Table('table_name')
start_time = time.perf_counter()
retry = True
while retry:
try:
response = table.scan(TableName ="table_name")
retry = 'LastEvaluatedKey' in response
scan_args['ExclusiveStartKey'] = response.get('LastEvaluatedKey')
except Exception as e:
print(e)
print(time.perf_counter()-start_time)
I tried boto3 getItem()
method this becomes faster like first time it takes 0.4seconds and after that it takes 0.01 seconds.
Not sure why it is not working with table scan method.
Please suggest.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
DAX不会缓存扫描结果。因此,您不应该期望提高性能,并且由于您在通往数据库的途中通过额外的服务器弹跳,因此可以期待性能惩罚。
您必须有很大的项目才能查看这些性能号码。而且您正在进行扫描吗?您可能需要仔细检查DynamoDB是正确的。
DAX doesn’t cache scan results. You therefore shouldn’t expect a performance boost and, since you’re bouncing through an extra server on the way to the database, can expect a performance penalty.
You must have very large items to see these performance numbers. And you’re doing a scan a lot? You might want to double check DynamoDB is the right fit.