如何在结果质量很大程度上是主观的情况下实现自动化测试
我一直在通过一系列快速原型开发自动化和统计生成程序，以了解 API 的许可成本是否能产生良好的投资回报。 有限的时间框架和我自己有限的经验导致了一个没有自动化质量或正确性测试和指标的有用工具。
该程序处理 PDF 文档的本地化过程。 具体来说，它生成有关文件某些内容的报告（大约字数、图像数等），并具有一些内容提取和处理功能。 它主要用于减少评估PDF本地化项目成本的时间和成本。
该应用程序现已批准进行更正式的开发流程，包括错误跟踪系统的请求以及初步测试、发布和反馈周期。
那么问题是，在这种应用程序中，您将如何进行质量保证和测试，其中数字通常是基于某些启发式的最佳猜测，并且由于源文档的可怕结构，处理后的输出并不总是有用？ 当数字明显疯狂时，我计划在报告中添加警告，但还能做些什么来保证质量呢？
到目前为止，我拥有的最复杂的解决方案是通过在构建环境中进行断言测试并编写一堆传统的用户测试用例（我希望避免）来保证某些辅助方法的结果。
您如何测试主观质量指标？
我正在使用 C# 工作，但我更喜欢通用的最佳实践答案，而不是任何过于特定于框架的答案。
I have been evolving an automation and statistics generation program through a series of rapid prototypes to see if the license cost of an API generates a good return on investment. The limited time frame, and my own limited experience has led to a useful tool with no automated quality or correctness tests and metrics.
The program deals with the localization process for PDF documents. Specifically, it generates a report on some of the contents of the files (approx. word count, image count etc.) and has some content extraction and processing features. It is mainly used to reduce the time and cost of evaluating the cost of a PDF localization project.
The application has been now approved for a more formal development process, including a request for a bug tracking system and a preliminary test, release and feedback cycle.
The question then, is how would you go about QA and testing in this kind of application, where the numbers are often a best guess based on some heuristic and the processed output is not always useful due to the horrific construction of the source documents? I plan to add warnings to the report when the numbers are obviously crazy, but what else can be done to guarantee quality?
So far the most sophisticated solution I have is to guarantee the results of some helper methods through assertion testing in the build environment and writing a bunch of traditional user test cases (which I'd prefer to avoid).
How do you test for subjective quality measures?
I am working in C#, but I favour a general best practices answer over anything too framework specific.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
我不确定您的应用程序到底在做什么，但回答一般性问题：
构建代表您的输入范围的测试用例集合，并查看它是否可以正确判断这些输入。 您无法真正绕过测试实际的输入文档。
然后，到了某个时刻，您必须接受自动化测试所能完成的工作是有限的。 例如，当事物变得非常主观的美学或可用性时，您将需要一个真实的人来获得有用的判断。
我希望我能给出更有帮助的答案。
I'm not sure exactly what you're application is doing, but to answer the general question:
Build a collection of test cases which represent your range of inputs and see if it can correctly judge those. You can't really get around testing actual input documents.
And then, there's a point where you'll have to accept that there's a limit to what can be accomplished with automated testing. When things get really subjective aesthetics or usability, for instance) you're going to need an actual human to get a useful judgment.
I wish I could give a more helpful answer.
尝试审批测试。
Try Approval Tests.