如何大规模地为所有用户定期处理某个逻辑或作业？
我的项目中有大量用户，例如 50m。
我应该每天为每个用户创建一个播放列表，为此，我当前正在使用以下方法：
我的用户表中有一个列，其中保存为该用户创建播放列表的最新时间，我将其命名为 <代码>last_playlist_created_at。
我在用户表上运行查询并获取前 1000 条信息，该查询选择 last_playlist_created_at 过去一天的用户列表，并按 last_playlist_created_at 对结果进行升序排序>
之后，我对结果运行 foreach 并在消息代理中为每个结果发布一条消息。
在消息代理后面，我启动大约 64 个工作进程来处理消息（为用户创建播放列表）并更新用户表中的 last_playlist_created_at。
如果我的消息代理消息列表为空，我将重复这些步骤（While - Do-While）
我认为处理方法足够好并且也可以扩展， 但是我们用来为每个用户创建消息的方法是不可扩展的！
我应该如何为每个用户发送大量消息？
I have a large set of users in my project like 50m.
I should create a playlist for each user every day, for doing this, I'm currently using this method:
I have a column in my users' table that holds the latest time of creating a playlist for that user, and I name it last_playlist_created_at.
I run a query on the users' table and get the top 1000s, that selects the list of users which their last_playlist_created_at is past one day and sort the result in ascending order by last_playlist_created_at
After that, I run a foreach on the result and publish a message for each in my message-broker.
Behind the message-broker, I start around 64 workers to process the messages (create a playlist for the user) and update last_playlist_created_at in the users' table.
If my message-broker messages list was empty, I will repeat these steps (While - Do-While)
I think the processing method is good enough and can be scalable as well,
but the method we use to create the message for each user is not scalable!
How should I do to dispatch a large set of messages for each of my users?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
好的，所以我的答案完全基于您的评论，您提到您使用 while(true) 来检查播放列表是否需要更新，这看起来并不那么微不足道。
虽然这是一个设计问题并且有多种解决方案，但我将如何解决它。
首先，将更新用户的播放列表视为一项工作。
现在，就您而言，这是一项预定的作业。 IE。每天一次。
Ok, so my answer is completely based on your comment where you mentioned that you use
while(true)to check if the playlist needs to be updated which does not seem so trivial.Although this is a design question and there are multiple solutions, here's how I would solve it.
First up, think of updating the playlist for a user as a job.
Now, in your case this is a scheduled Job. ie. once a day.