Dynamo DB put_item失败静音
我正在使用lambda用表格写入Dynamo(实际上是两个Dyngos),并带有表格资源,如指定的在这里。
代码流基本上是:
Step 1) Reads data from external source
Step 2) Do some cleaning and filtering
Step 3) Write data from Step 2 to first dynamo using comprehension
Step 4) Write data from Step 2 to second dynamo using comprehension
两个编写步骤均使用下面的功能完成
def parse_response_from_dynamo(response:dict, entry:str):
if response['ResponseMetadata']['HTTPStatusCode']!=200:
print(f"Failed to write {entry}")
return
def create_entry_in_dynamo(
data:List[Dict], table_name:str)->None:
"""
Write each dict in data to table_name in dynamo
Input:
data(List[Dict]): list of dicts, where each entry is the data
to be sent to dynamo
table_name(str): name of the table to which data will be written
"""
dynamo = boto3.resource('dynamodb')
table = dynamo.Table(table_name)
[parse_response_from_dynamo(table.put_item(Item=item), item['id'])
for item in data
]
,直到步骤3正常工作,所有项目都写入第一个发电机,我确实在AWS控制台中看到了一些活动。问题是第4步,我也可以在控制台中看到写入活动,但是这些项目没有出现在表中。 我发现这个
到目前为止,Debug完成了:
- 我认为理解是问题,因此我使用常规来检查响应,然后上升到100个顺序写作同一张桌子为止。正常工作的
- 两个表都用5个RCU/WCU提供,并检查了第一个调试步骤,我消耗了几乎2个WCU,所以我不认为问题在这里。
- 在本地运行代码,步骤4成功运行(我最尴尬的是)
- AWS文档以及SO问题,提到了每个分区的1KB限制。从项目摘要中读取手册成功写入后,AVG项目大小为42个字节。
我在
请问有人可以在这里熄灯吗?
I'm using Lambda to write to dynamo (actually two dynamos) with the Table resource as specified here.
code flow is basically:
Step 1) Reads data from external source
Step 2) Do some cleaning and filtering
Step 3) Write data from Step 2 to first dynamo using comprehension
Step 4) Write data from Step 2 to second dynamo using comprehension
Both writing steps are done using the function below
def parse_response_from_dynamo(response:dict, entry:str):
if response['ResponseMetadata']['HTTPStatusCode']!=200:
print(f"Failed to write {entry}")
return
def create_entry_in_dynamo(
data:List[Dict], table_name:str)->None:
"""
Write each dict in data to table_name in dynamo
Input:
data(List[Dict]): list of dicts, where each entry is the data
to be sent to dynamo
table_name(str): name of the table to which data will be written
"""
dynamo = boto3.resource('dynamodb')
table = dynamo.Table(table_name)
[parse_response_from_dynamo(table.put_item(Item=item), item['id'])
for item in data
]
Up until Step 3 works just fine, with all items being written to the first dynamo and I do see some activity in the AWS console. The problem is the Step 4 where I also am able to see write activity in the console, but the items don't appear in the table.
I found this similar SO question, confirming the consoles' graph I was supposed to inspect and I'm also way below 1k writes, 200 writes tops and very much rare.
Debug done so far:
- I believe comprehension to be the problem, so I used a regular for to inspect the responses and went up until 100 sequential writes for the same table. Worked normally
- Both tables are provisioned with 5 RCU/WCU and inspecting the first debug step, I consumed almost 2 WCU, so I don't believe the problem is here.
- Ran locally the code and Step 4 ran successfully (what I most found awkward)
- The AWS docs as well as the SO questions mentions the 1kb limit per partition. Reading from Items Summary after the manual successful write, the avg item size is of 42 bytes.
I read upon error handling for dynamo, but I didn't received any error in any of the tests.
Can anybody shed a light here, please ?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论