Your title states that you have a Rails performance issue, but do you know this for certain? From the rest of your question it sounds like you're trying to anticipate a possible future performance issue. The only way to deal sensibly with performance issues is to get your application into the wild and profile it. Doing so will give you empirical data as to what the real performance issues are.
Given that Amazon SQS isn't free and the fact that using it will almost certainly add complexity to your application, I would migrate to it if and when database load becomes a problem. Don't try to second guess problems before they arise, because you'll find that you'll likely face different problems when your app goes live, some of which you probably haven't considered.
The main point is that you've already decided to use background processing, which is the correct decision, given that any sort of processing that isn't instantaneous doesn't belong within the Rails' request/response cycle, as it blocks that Rails process. You can always scale with Amazon later if you need to.
Is your app hosted on Amazon EC2 already? I probably wouldn't move an existing app over to AWS just so I can use SQS, but if you're already using Amazon's infrastructure, SQS ia a great choice. You could certainly set up your own messaging system (such as RabbitMQ), but by going with SQS that's one less thing you have to worry about.
There are a lot of options to add background processing to Rails apps, such as delayed_job or background_job, but my personal favorite is Workling. It gives you a nice abstraction layer that allows you to plug in different background runners without having to change the actual implementation of your jobs.
I've also used SQS for a separate Ruby (non-Rails) project and generally found it reliable and fast enough. Like James pointed out above, you can read up to 10 messages at once, so you'll definitely want to do that (my Workling SQS client does this and buffers the messages locally).
I agree with John Topley that you don't want to over-complicate your application if you don't need to. That being said there are times when it is good to make this kind of decision early, do you anticipate high load from the beginning? Are you rolling this out to an existing user base or is it a public site that may or may not take off?
If you know you will need to handle a large amount of traffic from the beginning then this might be a good step. If you don't want to spend the money to use SQS take a look at some of the free queue solutions out there like RabbitMQ.
I currently push a couple million messages a month through SQS and it works pretty well. Make sure you plan for it being down or slow from time to time, so you would need to work in some retry facilities and exponential backoff. One of the nice things is that you can get 10 messages at a time which speeds up being able to work through the queue, you can use one request to get the 10 messages and process them 1 by 1.
发布评论
评论(5)
您的标题指出您遇到了 Rails 性能问题,但您确定知道这一点吗? 从您问题的其余部分来看,您似乎正在尝试预测未来可能出现的性能问题。 明智地处理性能问题的唯一方法是将您的应用程序投入使用并对其进行分析。 这样做将为您提供有关真正性能问题的经验数据。
鉴于 Amazon SQS 不是免费的,而且使用它几乎肯定会增加应用程序的复杂性,如果和当数据库负载成为问题时我会迁移到它。 不要试图在问题出现之前再次猜测问题,因为您会发现当应用程序上线时您可能会遇到不同的问题,其中一些问题您可能没有考虑过。
主要的一点是,您已经决定使用后台处理,这是正确的决定,因为任何类型的非瞬时处理都不属于 Rails 的请求/响应周期,因为它会阻止 Rails过程。 如果需要,您可以随时通过亚马逊进行扩展。
Your title states that you have a Rails performance issue, but do you know this for certain? From the rest of your question it sounds like you're trying to anticipate a possible future performance issue. The only way to deal sensibly with performance issues is to get your application into the wild and profile it. Doing so will give you empirical data as to what the real performance issues are.
Given that Amazon SQS isn't free and the fact that using it will almost certainly add complexity to your application, I would migrate to it if and when database load becomes a problem. Don't try to second guess problems before they arise, because you'll find that you'll likely face different problems when your app goes live, some of which you probably haven't considered.
The main point is that you've already decided to use background processing, which is the correct decision, given that any sort of processing that isn't instantaneous doesn't belong within the Rails' request/response cycle, as it blocks that Rails process. You can always scale with Amazon later if you need to.
您的应用程序已托管在 Amazon EC2 上吗? 我可能不会为了使用 SQS 而将现有应用程序迁移到 AWS,但如果您已经在使用 Amazon 的基础设施,那么 SQS 是一个不错的选择。 您当然可以设置自己的消息传递系统(例如 RabbitMQ),但是通过使用 SQS,您就少了一件需要担心的事情。
有很多选项可以向 Rails 应用程序添加后台处理,例如 delayed_job 或 background_job,但我个人最喜欢的是 工作。 它为您提供了一个很好的抽象层,允许您插入不同的后台运行程序,而无需更改作业的实际实现。
我维护一个添加 SQS 客户端的工作分支。 有一些缺点(阅读评论或我的博客文章< /a> 了解更多详细信息),但总的来说,它在我上次启动时对我们来说效果很好。
我还在一个单独的 Ruby(非 Rails)项目中使用了 SQS,并且通常发现它足够可靠且足够快。 就像 James 上面指出的那样,您一次最多可以读取 10 条消息,因此您肯定会想要这样做(我的 Workling SQS 客户端会执行此操作并在本地缓冲消息)。
Is your app hosted on Amazon EC2 already? I probably wouldn't move an existing app over to AWS just so I can use SQS, but if you're already using Amazon's infrastructure, SQS ia a great choice. You could certainly set up your own messaging system (such as RabbitMQ), but by going with SQS that's one less thing you have to worry about.
There are a lot of options to add background processing to Rails apps, such as delayed_job or background_job, but my personal favorite is Workling. It gives you a nice abstraction layer that allows you to plug in different background runners without having to change the actual implementation of your jobs.
I maintain a Workling fork that adds an SQS client. There are some shortcomings (read the comments or my blog post for more details), but overall it worked well for us at my last startup.
I've also used SQS for a separate Ruby (non-Rails) project and generally found it reliable and fast enough. Like James pointed out above, you can read up to 10 messages at once, so you'll definitely want to do that (my Workling SQS client does this and buffers the messages locally).
我同意 John Topley 的观点,即如果不需要,您不想使您的应用程序过于复杂。 话虽这么说,有时尽早做出这种决定是件好事,但您是否从一开始就预计会有高负载? 您是否将其推广到现有用户群,或者它是一个可能会或可能不会成功的公共网站?
如果您知道从一开始就需要处理大量流量,那么这可能是一个很好的步骤。 如果您不想花钱使用 SQS,请查看一些免费队列解决方案,例如 RabbitMQ。
我目前每月通过 SQS 推送几百万条消息,效果非常好。 确保您计划好它会不时出现故障或变慢,因此您需要使用一些重试设施和指数退避。 好处之一是您可以一次获取 10 条消息,这加快了队列的工作速度,您可以使用一个请求来获取 10 条消息并一一处理它们。
I agree with John Topley that you don't want to over-complicate your application if you don't need to. That being said there are times when it is good to make this kind of decision early, do you anticipate high load from the beginning? Are you rolling this out to an existing user base or is it a public site that may or may not take off?
If you know you will need to handle a large amount of traffic from the beginning then this might be a good step. If you don't want to spend the money to use SQS take a look at some of the free queue solutions out there like RabbitMQ.
I currently push a couple million messages a month through SQS and it works pretty well. Make sure you plan for it being down or slow from time to time, so you would need to work in some retry facilities and exponential backoff. One of the nice things is that you can get 10 messages at a time which speeds up being able to work through the queue, you can use one request to get the 10 messages and process them 1 by 1.
Amazon SQS 是一项优秀的服务,除非以下几点变得很重要:
如果其中任何一项很重要,您需要考虑真正的企业 MQ 服务,例如 StormMQ 、RabbitMQ,甚至 onlinemq.com。
我发现这个博客系列很有趣,因为它毫不犹豫地将 Amazon SQS 与 StormMQ 进行了比较:
http://blog.stormmq.com/2011/01/ 06/苹果和橘子表演/
Amazon SQS is a fine service, except where the following things become important:
If any of these things are important you need to look at a real enterprise MQ service such as StormMQ, RabbitMQ, or even onlinemq.com.
I found this blog series interesting as it compares Amazon SQS to StormMQ without holding any punches back:
http://blog.stormmq.com/2011/01/06/apples-and-oranges-performance/
如果您在迁移到 EC2 时遇到问题,您可以使用 onlinemq.com 等其他服务。
If you have issues with moving to EC2, You can use other services like onlinemq.com.