有什么非常有效的方法可以让批量邮件应用程序准确跟踪其进度？
我们有一个控制台应用程序（当前为.NET），可以向订阅邮件列表的人批量发送邮件。由于该应用程序当前存在的限制，我们正在重新实现该应用程序。
它必须具备的功能之一是，它可以在操作意外中断后恢复。这意味着每次成功发送电子邮件时，它都必须以一种可以从上次中断的地方重新开始的方式进行跟踪。它将从另一台服务器获取所需的信息（基本上是使用数字 ID 标识的收件人列表），该服务器具有包含此信息的数据库。
我们的设置很简单：我们有一台包含收件人的基于 Windows 的 Web/数据库服务器，并且我们有运行 Debian 的 SMTP 服务器。
我们提出了几个可以解决此问题的选项：
- 每次发送操作后将信号发送回数据库
- 通过仅将收件人的最后一个 id 写入此文件（每次写入都会覆盖其内容）来跟踪小文件每次发送操作。
- 跟踪在主机上运行的数据库（mysql、postgresql、sqlite 等）。
限制是应用程序应该快速发送邮件。至于要发送的邮件数量，每批几百到几万封不等，也可能一天好几批。总体而言，每天通常会收到 1000 到 50,000 封邮件，但这一数字还会增长。另外，它必须能够准确地恢复，所以我不能等到发送 50 封邮件，然后将进度写入文件或数据库等。
到目前为止，我针对上述解决方案提出了以下建议：
- 我们目前的应用程序使用此解决方案。但应用程序将在与数据库服务器不同的服务器上运行（它们也不在同一网络中，但应用程序将在邮件服务器上运行，与当前情况相反），所以我无法想象这是最有效的解决方案。
- 这可能非常快，但它不会对硬盘造成压力，导致其使用寿命严重缩短吗？ （这台服务器是较旧的 Opteron，我相信，它可能早于 SATA，但如果是这样，也不会太早。）
- 这可能非常快且高效，但是否有必要设置一个数据库来仅用于目的存储 2 个数字（批次的 ID，以及该批次中最后一个收件人的 ID）？开销可能会减慢速度吗？
除了上述解决方案之外，是否还有其他我尚未考虑的选项，可以在不真正减慢应用程序速度的情况下进行跟踪？我的假设准确吗？
We have a console application (currently .NET) that sends out mail in bulk to people that have subscribed to a mailing list. We are re-implementing this application because of limitations that it currently has.
One of the features it must have, is that it can resume after an unexpected interrupt of its operation. This means that every time it succesfully sends an e-mail, it has to keep track in a way that it can pick back up right where it left off. It'll get the information it needs (basically the list of recipients which are identified using a numeric id) from a different server, which has the database containing this information.
Our setup is simple: we have one Windows-based web/database server that contains the recipients, and we have the SMTP-server running Debian.
We have come up with several options that would solve this:
- Send a signal back to the database after every send operation
- Keep track in a small file by writing only the last id of the recipient to this file (overwriting its contents with each write) after every send operation.
- Keep track in a database that runs on the host machine (mysql, postgresql, sqlite, etc)
The constraints are that the application is supposed to send mails fast. As for amounts of mails it has to send, it'll vary between several hundreds to several tens of thousands per batch, and it could be several batches per day, too. Overall it's usally between 1000 and 50.000 mails on a day, but this will grow. Also, it must be able to resume accurately so I can't wait until, say, 50 mails are sent, and then write the progress to a file or database or so.
This what I came up with so far with regards to the above solutions:
- We currently have our application use this solution. But the application will run on a different server than the database server (they aren't in the same network either, but the application will run on the mail server, as opposed to the current situation) so I can't imagine that being the most efficient solution.
- This could be very fast, but wouldn't it strain the hard drive to the point where its lifespan could be severely shortened? (This server is an older Opteron, I believe, it may pre-date SATA, but if so, not by much.)
- This may be very fast, and efficient, but would it be necessary to setup a database for the purpose of only storing 2 numbers (id of the batch, and id of the last recipient within that batch)? Would overhead maybe slow this down?
Apart from the above solutions, are there other options I haven't yet considered, to keep track without really slowing the application down? Are my assumptions accurate?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
每天 1000-50000 封电子邮件对我来说似乎并不算多，所以我认为您目前不必太担心容量。在我工作的地方，我们有一个 Windows 服务实例，它一次从数据库（存储我们的电子邮件数据的位置）读取 100 行，连续处理每一行并更新数据库以将电子邮件标记为已发送。我并不是说这是一个好的设计（事实并非如此），但我们经常使用此设置每天发送超过 50,000 封电子邮件。
如果您确实需要扩展 - 即您可以在未来 3、6、12 个月的增长方面进行量化，并且显示显着增长 - 那么我现在会在可扩展性方面投入真正的精力。如果你不这样做，我会专注于保持简单和轻量级。
为什么不在批量电子邮件应用程序处理每封电子邮件时将其标记为“处理中”，然后在工作完成后将其标记为“已发送”（均在数据库中）？这种方法还可以让您根据规模需求对应用程序进行多线程处理（当然，如果您为此进行设计）。
1000-50000 emails per day doesn't seem like an awful lot to me, so I don't think you will have to worry too much about capacity at the moment. Where I work we have a single instance of a Windows service which reads 100 rows from a database (where our email data is stored) at a time, processes each row in succession and updates the database to mark the email as sent. I'm not saying this is a good design (it isn't) but we regularly send more than 50k emails per day using this setup.
If you have a real need to scale - i.e. one you can quantify in terms of growth over the next 3, 6, 12 months and which shows significant growth - then I'd put real effort into scalability now. If you don't, I'd focus on keeping it simple and lightweight.
Why not mark each email message as "in process" while it's being processed by your bulk email application, and then mark it as "sent" (both in the db) when the work is done? This approach could allow you to multi-thread your application as scale demands as well (if you design for that, of course).