Dynamo DB put_item失败静音

发布于 2025-01-22 11:23:21 字数 1591 浏览 3 评论 0原文

我正在使用lambda用表格写入Dynamo(实际上是两个Dyngos),并带有表格资源,如指定的在这里

代码流基本上是:

Step 1) Reads data from external source
Step 2) Do some cleaning and filtering
Step 3) Write data from Step 2 to first dynamo using comprehension
Step 4) Write data from Step 2 to second dynamo using comprehension

两个编写步骤均使用下面的功能完成

def parse_response_from_dynamo(response:dict, entry:str):
    if response['ResponseMetadata']['HTTPStatusCode']!=200:
      print(f"Failed to write {entry}")
      return

def create_entry_in_dynamo(
    data:List[Dict], table_name:str)->None:
    """
    Write each dict in data to table_name in dynamo
    Input:
        data(List[Dict]): list of dicts, where each entry is the data
            to be sent to dynamo
        table_name(str): name of the table to which data will be written
    """

    dynamo = boto3.resource('dynamodb')
    table = dynamo.Table(table_name)

    [parse_response_from_dynamo(table.put_item(Item=item), item['id'])
           for item in data
    ]
    

,直到步骤3正常工作,所有项目都写入第一个发电机,我确实在AWS控制台中看到了一些活动。问题是第4步,我也可以在控制台中看到写入活动,但是这些项目没有出现在表中。 我发现这个

到目前为止,Debug完成了:

  • 我认为理解是问题,因此我使用常规来检查响应,然后上升到100个顺序写作同一张桌子为止。正常工作的
  • 两个表都用5个RCU/WCU提供,并检查了第一个调试步骤,我消耗了几乎2个WCU,所以我不认为问题在这里。
  • 在本地运行代码,步骤4成功运行(我最尴尬的是)
  • AWS文档以及SO问题,提到了每个分区的1KB限制。从项目摘要中读取手册成功写入后,AVG项目大小为42个字节。

我在

请问有人可以在这里熄灯吗?

I'm using Lambda to write to dynamo (actually two dynamos) with the Table resource as specified here.

code flow is basically:

Step 1) Reads data from external source
Step 2) Do some cleaning and filtering
Step 3) Write data from Step 2 to first dynamo using comprehension
Step 4) Write data from Step 2 to second dynamo using comprehension

Both writing steps are done using the function below

def parse_response_from_dynamo(response:dict, entry:str):
    if response['ResponseMetadata']['HTTPStatusCode']!=200:
      print(f"Failed to write {entry}")
      return

def create_entry_in_dynamo(
    data:List[Dict], table_name:str)->None:
    """
    Write each dict in data to table_name in dynamo
    Input:
        data(List[Dict]): list of dicts, where each entry is the data
            to be sent to dynamo
        table_name(str): name of the table to which data will be written
    """

    dynamo = boto3.resource('dynamodb')
    table = dynamo.Table(table_name)

    [parse_response_from_dynamo(table.put_item(Item=item), item['id'])
           for item in data
    ]
    

Up until Step 3 works just fine, with all items being written to the first dynamo and I do see some activity in the AWS console. The problem is the Step 4 where I also am able to see write activity in the console, but the items don't appear in the table.
I found this similar SO question, confirming the consoles' graph I was supposed to inspect and I'm also way below 1k writes, 200 writes tops and very much rare.

Debug done so far:

  • I believe comprehension to be the problem, so I used a regular for to inspect the responses and went up until 100 sequential writes for the same table. Worked normally
  • Both tables are provisioned with 5 RCU/WCU and inspecting the first debug step, I consumed almost 2 WCU, so I don't believe the problem is here.
  • Ran locally the code and Step 4 ran successfully (what I most found awkward)
  • The AWS docs as well as the SO questions mentions the 1kb limit per partition. Reading from Items Summary after the manual successful write, the avg item size is of 42 bytes.

I read upon error handling for dynamo, but I didn't received any error in any of the tests.

Can anybody shed a light here, please ?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文