AWS DAX性能问题与表扫描

发布于 2025-02-10 08:24:20 字数 764 浏览 1 评论 0原文

嗨,我正在研究一个项目,该项目需要将所有Dyanamo DB文档带入内存中。我将使用table.scan() boto3方法,该方法对于所有10k记录几乎需要33秒。

我已经配置了DAX并将其用于表扫描,该扫描需要将近42秒,具有相同的10K记录,具有相同的Lambda配置。我尝试了多次结果相同。

我尝试了以下代码:

daxclient = amazondax.AmazonDaxClient.resource(endpoint_url="...")
table = daxclient.Table('table_name')
start_time = time.perf_counter()
retry = True
while retry:
     try:
         response = table.scan(TableName ="table_name")
         retry = 'LastEvaluatedKey' in response
         scan_args['ExclusiveStartKey'] = response.get('LastEvaluatedKey')
     except Exception as e:
         print(e)
print(time.perf_counter()-start_time)

我尝试了boto3 getItem()方法,这变得更快,就像第一次需要0.4秒,然后需要0.01秒。

不知道为什么它不使用表扫描方法。

请建议。

Hi I am working on an project that requires to bring all dyanamo db document in memory. I will be using table.scan() boto3 method which nearly takes 33 seconds for all 10k records.

I have configured the DAX and using it for table scan, which takes nearly the 42 seconds with same 10k records with same lambda configuration. I tried multiple times results are same.

I tried below code :

daxclient = amazondax.AmazonDaxClient.resource(endpoint_url="...")
table = daxclient.Table('table_name')
start_time = time.perf_counter()
retry = True
while retry:
     try:
         response = table.scan(TableName ="table_name")
         retry = 'LastEvaluatedKey' in response
         scan_args['ExclusiveStartKey'] = response.get('LastEvaluatedKey')
     except Exception as e:
         print(e)
print(time.perf_counter()-start_time)

I tried boto3 getItem() method this becomes faster like first time it takes 0.4seconds and after that it takes 0.01 seconds.

Not sure why it is not working with table scan method.

Please suggest.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

一百个冬季 2025-02-17 08:24:20

DAX不会缓存扫描结果。因此,您不应该期望提高性能,并且由于您在通往数据库的途中通过额外的服务器弹跳,因此可以期待性能惩罚。

您必须有很大的项目才能查看这些性能号码。而且您正在进行扫描吗?您可能需要仔细检查DynamoDB是正确的。

DAX doesn’t cache scan results. You therefore shouldn’t expect a performance boost and, since you’re bouncing through an extra server on the way to the database, can expect a performance penalty.

You must have very large items to see these performance numbers. And you’re doing a scan a lot? You might want to double check DynamoDB is the right fit.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文