端到端测试 RESTful Web 服务 (Rails)
我正在尝试筛选无数的测试解决方案，但我什至不确定我的方向是否正确。故事是这样的：我们正在运行一个 RESTful Web 服务，作为 Rails 应用程序实现，为我们的移动客户端提供支持。我们正在对 Web 服务进行单元测试（当然），但这涉及模拟应用程序的许多部分，例如搜索堆栈 (Apache SOLR)。
此外，我们的测试不（即不能！）涵盖关键路线，例如移动登录/登录过程，因为这涉及 API 应用程序和移动网站之间的通信，用户可以在其中输入凭据，例如SSO（Janrain Engage）。因此，标准的 Rails 集成测试是行不通的。
我意识到，理论上，如果测试套件设计得很好，模拟仅严格发生在下一层测试开始的那些连接点，然后通过分别对服务 API 和移动网站进行单元或功能测试，一个可以获得相同的测试覆盖率。但我发现，在实践中，如果有多个开发人员独立开发测试套件，这就是一种幻想；我只是承认我们的单元测试设计得不够好。特别是在练习 TDD 时，我发现虽然测试可以驱动应用程序代码，但测试代码设计仅针对被测单元进行定制，从而导致测试套件规模相当大。
我发现的另一件事是，有时我们并没有纯粹使用单元测试来检测回归，例如，由于连锁效应，错误的查询被发送到 SOLR 服务器。这就是为什么我认为确保整个堆栈沿着关键路线工作的唯一真正方法是在每次部署之前在临时服务器上自动进行端到端测试，即将实际的 HTTP 请求发送到应用程序。
我的问题是：
- 您认为这是合理的做法吗？我在网络上发现了很少关于端到端测试实时 API 的信息，这让我想知道我是否有任何意义，
- 您会建议哪些工具/设置？我们使用 Watir 为我们的网站运行验收测试，但它对于 Web 服务来说似乎有点过分了（不需要浏览器环境，不需要 JS 或任何 UI 风格的东西）。甚至像 Ruby 脚本一样简单？
- 您可以给我设计此类测试的一般最佳实践或建议吗？
I'm trying to sift through the myriad of test solutions out there, and I'm not even sure if I'm headed the right direction. The story is: we're running a RESTful Web service, implemented as a Rails app, which backs our mobile clients. We're unit testing the Web service (of course), but that involves mocking out many parts of the application, for instance, the search stack (Apache SOLR).
Moreover, our tests don't (i.e. cannot!) cover critical routes such as the mobile sign-in/sign-on process, since that involves communication between the API application and the mobile website, where the user can enter credentials, e.g. for SSO (Janrain Engage). Hence, a standard Rails integration test won't do.
I realize that in theory, if the test suite is very well designed, where mocking only happens strictly at the those join points where the tests for the next layer start, then by unit or functional testing the service API and the mobile website separately, one could get the same test coverage. I find that in practice though, this is an illusion if you have several developers working on the test suite independently; I just admit that our unit tests are simply not that well designed. Particularly when exercising TDD I found that while the tests can drive the application code, the test code design is only tailored to the unit under test, resulting in a rather wildly grown test suite.
Another thing I found is that sometimes we didn't detect regressions purely using unit tests, where e.g. bad queries were sent to the SOLR server due to knock-on effects. That's why I thought the only true way to ensure that the entire stack works along the critical routes is to automatically end-to-end test it on a staging server before every deployment, i.e. having actual HTTP requests sent to the app.
My questions would be:
- do you think this is a reasonable thing to do at all? I found very little information about end-to-end testing a live API on the Web, leaving me wondering whether I'm making any sense at all
- which tools/setup would you suggest? We use Watir to run acceptance tests for our website, but it seems to be overkill for a Web service (no browser environment needed, no JS or anything UI-ish). Something as simple as a Ruby script even?
- any general best practices or advice you can give me w.r.t. to designing such tests?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
您可能对此感兴趣：http://groups.google.com /group/ruby-capybara/browse_thread/thread/5c27bf866eb7fad3
您可能想要尝试的是结合 Cucumber （或类似）使用上面链接中提到的工具之一，因此您可以执行类似
这样的操作，您可以测试 API 的完整堆栈并检查结果。
This might be of interest to you: http://groups.google.com/group/ruby-capybara/browse_thread/thread/5c27bf866eb7fad3
What you might want to to try is combining Cucumber (or similar) with one of the tools mentioned in the link above, so you can do something like
This way you can test the API's full stack and check the result.
我们最近遇到了类似的问题，例如仅在单元级别进行测试速度很快，但不足以发现与集成相关的异常。
我真的很喜欢 Cucumber 的 gherkin 语法，它对我的价值在于我们可以运行一次练习，然后使用单独的标题进行多次期望，就像它们是单独的 RSpec
it块一样。有一个名为 Turnip 的 gem，它可以使用 gherkin 语法编写 RSpec 测试。我们已经使用它有一段时间了，我对它非常满意，我们甚至开始减少集成级别的缺陷。
我在一篇博客文章中收集了我的经验：https://blog.kalina.tech/2019/10/end-to-end-testing-for-api-only-rails-apps-with-cucumber-syntax.html
We've faced similar problems lately, like testing only on the unit level was fast but not enough to spot integration related anomalies.
I really like Cucumber's gherkin syntax, the value for me in it is that we can run the exercise once and then make several expectations after with separate titles, like if they would be separate RSpec
itblocks.There is a gem called Turnip, which makes possible to write RSpec tests with the gherkin syntax. We have been using it for a while and I'm quite happy with it, we even started having less integration level defects.
I collected my experience regarding in a blog post: https://blog.kalina.tech/2019/10/end-to-end-testing-for-api-only-rails-apps-with-cucumber-syntax.html