处理排队的长时间运行进程的服务 - 从哪里开始？
构建服务来处理排队的长时间运行进程的最佳方法是什么？例如，这就是我们正在尝试做的事情
- 用户将 401k 数据上传到网络
- 数据进入处理队列（数据库表）
- 401k 被处理（每个客户端可能需要几分钟）
- 用户通过电子邮件收到 401k 已处理的通知接受
我们当然可以编写一个服务来处理这个问题，但是如果服务失败怎么办？如何通知某人？如果出现异常怎么办？此外，此服务是否应该处理其他非用户启动的进程（例如发送电子邮件等）。
我们拥有一支熟练的 .NET 开发人员团队，但我们所有的经验都是在任务计划程序启动的 Web/客户端应用程序或控制台应用程序中，因此我们将非常感谢您提供的任何指导。
What's the best way to build a service to handle queued long-running processes? For example, this is what we're trying to do
- User uploads 401k data to web
- Data goes into processing queue (a database table)
- 401k is processed (could take a couple minutes per client)
- User is informed via e-mail that 401k was accepted
We could certainly write a service to handle this, but what if the service fails? How would someone be notified? What if there is an exception? Also, should this service handle other non-user-initiated processes (like sending e-mails and such).
We have a team of skilled .NET developers, but all of our experience is in web/client apps or console apps launched by the task scheduler, so any direction you can provide would be much appreciated.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
考虑使用 MSMQ (
System.Messaging) 用于此任务。也许是这样的流程：您对潜在故障的其他担忧可能是另一项服务，或者可能是您网站中的轮询机制。读取数据库以获取最后处理的消息的日期时间。读取“等待”的项目数。如果数字不令人满意，请向管理员/客户/等发送电子邮件。如有必要。
使用 MSMQ 意味着您可以使用拉式系统，并且不会给您的数据库带来负担。通过针对数据库轮询每个n 来进行网络。消息发送都是事务性的，因此您根本不必担心丢失/未确认的消息。
@Jess：了解你正在试图掩护基地。适当的队列会有所帮助，因为它不会用请求来破坏您的数据库。您的应用程序/项目/客户的规模将决定这是否是一个问题。事实上，您可以将所有这些都放入 Page_Load 中的单个 .aspx 中，但您肯定知道得更多。
回复：矫枉过正。我“读”到了这个问题，即人们对停机时间以及这样的应用程序如何处理这个问题存在一些担忧。开箱即用的 Web 服务不能：
很好地处理数据库中断，因为它依赖数据库来保存其工作的输出。它是“捕手”、“工人”和“结果者”三者合一的一击。这真的属于DMZ吗？注意第 1 项：“Web”，而不是 Intranet。
无需添加更多 web+worker 节点即可扩展。
此建议的解决方案涉及：
MSMQ 有自己的存储实现，因此它不依赖于您的应用程序数据库来进行排队和交付顺序。它是 Windows 的一部分，并作为服务运行。如果 MSMQ 已关闭，则 Windows 已关闭，或者安全性配置不正确。
异步处理。您的网络层可以简单地“捕获”请求，并将其“放入”队列中。就是这样。无需启动线程，无需依赖应用程序数据库。它可以重新开始接收其他用户的请求。用户不必“感受到”繁忙工作日的滞后。
可扩展性。您可以将 Windows 服务部署到 1 台以上的计算机上来工作。
工作分离：401k 处理、电子邮件发送、请求处理 + 身份验证都在不同的模块中。
安全性 - 是否 100% 清楚这是在 Intranet 还是 Internet 解决方案中。如果暴露在互联网上，那么请考虑如何将消息从 DMZ 发送到内部应用程序。您能否证明从开放互联网对您的应用程序数据库进行读写访问是合理的？考虑某种立面。队列提供了这一点。
Consider using MSMQ (
System.Messaging) for this task. Perhaps a flow like this:Your other concern around potential failings could be yet another service, or perhaps a polling mechanism in your website. Read the DB for the datetime of the last processed message. Read the number of items 'waiting'. If the numbers aren't satisfactory, send email to admins/customers/etc. as necessary.
Using MSMQ means you get to use a pull-system, and won't be having to burden your database & network by polling every n against the database. The message send are all transactional, so you won't have to worry at all about lost/unAck'd mssages.
@Jess: Understood you're trying to cover bases. A proper queue would help in that it's not hammering your database with requests. The scale of your app/project/customers will dictate on whether this is an issue. Indeed, you could have thrown this all into a single .aspx in Page_Load, but you definitely know better.
Re: overkill. I 'read' into the question that there were some concerns about downtimes and how such an app could handle this. A web service, out of the box, doesn't:
handle DB outages well, as it's reliant on the DB to save the output of its work. It's the 'catcher', the 'worker', and the 'resulter' all in one shot. Does this really belong in the DMZ? Note Item 1: "Web", not intranet.
scale without adding more web+worker nodes.
This suggested solution involves:
MSMQ has its own storage implementation, so it's not reliant on your app database for queueing and order-of-delivery. It's part of Windows, and runs as a service.If MSMQ was down, then Windows is down, or security is not configured correctly.
asynchronous processing. Your web tier can simply 'catch' the request, and 'put' to the queue. That's it. No spinning up threads, no reliance on application databases. It can get back to the job of receiving requests from other users. Users won't have to 'feel' the lag of a busy workday.
scalability. You can deploy your Windows Service to 1+ machines to work.
separation of work: 401k processing, emailing, request-handling + authentication all in distinct modules.
security - it's not 100% clear whether this in an intranet or internet solution. If internet-exposed, then consider how you want to send messages from DMZ to your internal application. Can you justify having read-write access to your application DB from the open internet? Consider a facade of some kind. The queue provides this.
如果您的经验是在网络应用程序中，请在网络应用程序中进行。您可以在 Web 服务的上下文中而不是在其自己的上下文中完成处理。
编辑
显然我还不够清楚。 IIS 提供 ASP.NET 和静态内容，但它也是托管处理数据库中作业的轮询线程的理想位置。我称其为理想的，因为它可以更新可以在内部网页中显示的静态变量，并且可以从 IIS 提供的所有日志记录、重置和其他基础设施中受益。这避免了创建单独的服务只是为了运行一个线程的所有复杂性，然后必须处理该解决方案的成本。
我已经这样做过几次了，效果很好。
If your experience is in web apps, do it in the web app. You can have your processing done within the context of the web service instead of in its own.
edit
Apparently I wasn't clear enough. IIS serves up ASP.NET and static content, but it's also an ideal place to host a polling thread that processes the jobs in the database. I call it ideal because it can update static variables that can be displayed in an internal web page, and it can benefit from all of the logging, reseting and other infrastructure that IIS provides. This avoids all of the complexity of creating a separate service just to run one thread, and then have to deal with cost of that solution.
I've done this a few times, with good results.