S3 对象创建触发事件到 lambda

发布于 2025-01-11 10:59:09 字数 132 浏览 0 评论 0原文

我已经从 S3 创建了一个 AWS lambda 触发器 - 创建对象通知。当 lambda 收到对象创建通知时,它还会启动 ec2 实例。对于下一步,我希望 ec2 访问 s3 中的对象/数据并对这些数据进行一些处理。我如何从 ec2 访问该数据?

I have created an AWS lambda trigger from S3 - create object notification. When lambda gets object creation notification it also launches the ec2 instance. For the next step, I want ec2 to access the object/data form s3 and do some processing on that data. How can I access that data from ec2?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

谎言月老 2025-01-18 10:59:09

当 Lambda 函数启动 EC2 实例时,我希望它传递 S3 对象密钥作为 EC2 用户数据

When the Lambda function launches the EC2 instance, I would have it pass the S3 object key as part of the EC2 user-data.

晨光如昨 2025-01-18 10:59:09

您似乎想要使用现有的 Amazon EC2 实例来处理上传到 Amazon S3 存储桶的对象。

执行此操作的标准方法是:

  • 创建 Amazon SQS 队列
  • 配置 Amazon S3 存储桶,以便在创建新对象时向 SQS 队列发送消息
  • 在该队列上编写代码Amazon EC2 实例持续轮询 Amazon SQS 队列,请求消息(使用 WaitTimeSeconds=20 减少调用次数,如果消息可用,它将立即返回)
  • 一旦收到消息,代码可以下载S3对象(消息中提供了Bucket和Key),然后处理文件。文件处理完毕后,您的代码应该删除 SQS 消息

如果对象上传之间的时间间隔很长并且您希望省钱,您可以在代码中添加一些逻辑来执行 >在给定时间段内未收到消息时关闭实例。然后,您可以根据队列大小创建一个 Amazon CloudWatch 警报,该警报可以触发 AWS Lambda 函数来启动实例(如果实例当前未运行)。

要在每次 Amazon EC2 实例启动时运行脚本,您可以将其放置在 /var/lib/cloud/scripts/per-boot/ 中。

另请参阅:自动停止 EC2 实例当他们完成任务时 - DEV 社区

It appears that you are wanting to use an existing Amazon EC2 instance to process objects that are uploaded to an Amazon S3 bucket.

The standard method for doing this is:

  • Create an Amazon SQS queue
  • Configure the Amazon S3 bucket to send a message to the SQS queue when a new object is created
  • Write code on the Amazon EC2 instance that continuously polls the Amazon SQS queue, asking for a message (use WaitTimeSeconds=20 to reduce the number of calls, it will return immediately if a message is available)
  • Once a message is received, the code can download the S3 object (the Bucket and Key are provided in the message) and then process the file. Once the file is processed, your code should delete the SQS message

If there are long time-periods between object uploads and you wish to save money, you could add some logic to your code that does a Shutdown of the instance when no messages have been received for a given period of time. You could then create an Amazon CloudWatch alarm based on queue size that can trigger an AWS Lambda function to Start the instance if it is not currently Running.

To run a script every time that an Amazon EC2 instance starts, you can place it in /var/lib/cloud/scripts/per-boot/.

See also: Auto-Stop EC2 instances when they finish a task - DEV Community

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文