我如何在express App中实现队列
我正在使用部署在 EC2 容器中的 express 应用程序。 此应用程序从 AWSLambda 获取请求，其中包含一些数据来处理 Web Scrapping 服务（部署在 EC2 中，因为在 AWSLambda 中部署它很困难）。 问题是我需要在 express 应用程序中实现队列服务，以避免根据实例大小同时打开超过 X 个浏览器数量。 如何实现队列服务以等待网络抓取器请求终止，然后再启动另一个请求？ 时间不是问题，因为是在凌晨执行的计划任务。
I'm working with an express app which is deployed in a EC2 container.
This app gets the request from anAWSLambda with some data to handle a Web Scrapping service (is deployed in EC2 because deploying it in AWSLambda is difficult).
The problem is that I need to implement a queue service in the express app to avoid opening more than X quantity of browsers at the same time depending the instance size.
How can I implement a queue service to await for a web scrapper request to be terminated before launching another one?
The time is not a problem because is a scheduled task that executes in early morning.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
简单的内存队列是不够的，因为您不希望在发生崩溃时丢失请求。
如果您对应用程序崩溃感到满意或认为发生这种情况的可能性非常低，那么像下面这样的节点模块可能会很方便。
https://github.com/mcollina/fastq
如果可靠性很重要，那么亚马逊 SQS 应该很好去。
https： //docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/sqs-examples-send-receive-messages.html
将处理请求处理程序。有一个简单的超时基本处理程序，可以侦听队列并执行任务。
A simple in memory queue would not be enough, as you would not want request to be lost incase there is a crash.
If you are ok with app crash or think there there is very low probability then node modules like below could be handy.
https://github.com/mcollina/fastq
If reliability is important then amazon SQS should be good to go.
https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/sqs-examples-send-receive-messages.html
Queue the work on request handler. Have a simple timeout base handler which can listen to queue and perform task.