自动测试网页(并通过 DSL 从用例生成)
我的目标是:
- 我们的客户可以生成新的网络测试。
- 我们的持续集成服务器进行测试环境部署;它应该针对它执行测试。
- 测试也可以针对其他一些环境运行。
(最终的验收测试应该由客户进行,以测试字体等,但这对于我们的测试环境来说将是一个很好的预验收检查。客户可以专注于现在以外的其他事情。)
通常是一些属性(例如文本字段 id) )发生了变化或者其他什么,测试将在几周内中断。似乎记录的测试经常会失败,因此最好轻松地记录一个新的测试,而不是尝试维护和修改旧的测试。
现在,我找到了一种全新的方法。也许录音不是正确的方式。 怎么样,如果我们的客户可以用人类可读的自己的语言制作用例,机器可以理解这些用例并将其编译为网络记录(使用领域特定语言,DSL)。 这不是科幻小说,它已经被制作出来了,所以请继续阅读。 :-)
我尝试过使用这些自动 Web 测试框架:
- Visual Studio Web 测试(客户无法执行)
- Selenium(仅适用于 Firefox,我们的客户有 IE)
- WatiN(.NET 版本的 Watir,记录器似乎是一个有点问题)
- HP Quick Test Pro(不够容易进行新测试)
这些都没有提供我真正需要的东西......但 Selenium 是最接近的一个。
我们的客户说芬兰语,因此在软件项目开始时,在规范阶段,用户编写一个用例,例如:
- Avaa "OmaLomake"
- Syötä "Tuomas" kohtaan "nimi"
- Paina "Seuraava"
翻译:
- 打开“MyForm”
- 插入“Tuomas” “进入字段“名称”,
- 现在按“下一步”
...这是一个人类可读的用例,但也可以编译为自动网络验收测试。 Open、Insert、into field 和 Press 是关键字,其他是值。
什么样的 DSL 工具适合这个?
微软正在他们的奥斯陆项目中制作一个新的 DSL 制作工具,名为 MGrammar。这意味着您可以创建一种自定义语言,使非技术人员可以轻松地使用机器。 (与 Cobol 和 Visual Basic 相同的基本想法(但失败了)。)
我发现有人已经用 MGrammar 制作了这种 DSL,但它是针对 Watin,而不是 Selenium: http://www.codinginstinct.com/2008/ 11/creating-watin-dsl-using-mgrammar.html
因此,持续集成服务器进程将是:
- 从源代码控制中获取新版本(像往常一样)。
- 构建、运行单元测试并分析代码(像往常一样)。
- 制作安装包并在版本控制中标记版本(像往常一样)。
- 将用例编译为 Web 测试
- 运行 Web 测试
- 接受/拒绝软件 :-)
在持续集成服务器中运行 Web 测试通常意味着大量的配置工作。所以,在我尝试这个之前,我很好奇,你觉得怎么样? 您是否使用过相同类型的设置?您的经验是什么? (具体的环境是什么?)
DSL 怎么样,它是否有足够的能力来满足用例,还是会成为另一个无休止的开发任务?客户会生成测试吗?
My goal is:
- Our customers could generate new web-tests.
- Our continuous integration server makes a test-environment deployment; it should execute the tests against it
- The test could also be run against some other environment.
(Final acceptance tests should be made by the customer, to test fonts etc, but this would be a great pre-acceptance check for our test-environment. Customers could focus on other things than now.)
Usually some property (like text field id) has changed or something and the tests will break in a few weeks. It seems that recorded tests broke often, so it's better to easily record a new one than trying to maintenance and modify an old test.
Now, I found a whole new approach. Maybe recording is not the right way.
How about, if our customers could make use cases in a human readable own language which the machine would understand and compile to web-recording (with Domain Specific Language, DSL).
This is not sci-fi, it has been already made, so read on. :-)
I have tried to use these automatic web testing frameworks:
- Visual Studio web test (Customers can't execute)
- Selenium (Works only with Firefox, our customers have IE)
- WatiN (.NET version of Watir, recorder seems to be a bit buggy)
- HP Quick Test Pro (Not easy enough to make new tests)
None of these have provided actually what I need... but Selenium is the closest one.
Our customers speak Finnish, so in the beginning of a software project, in specification phase, user writes a use case like:
- Avaa "OmaLomake"
- Syötä "Tuomas" kohtaan "nimi"
- Paina "Seuraava"
Translation:
- Open "MyForm"
- Insert "Tuomas" into field "name"
- Press "Next"
Now... This is a human-readable use case, but also it can be compiled to automatic web acceptance test. Open, Insert, into field and Press are keywords, others are values.
What kind of DSL tool would be good for this?
Microsoft is making a new DSL-making-tool in their Oslo-project called MGrammar. It means that you can make a custom language to make it easy for non-technical people to work with machines. (The same basic idea that was (and failed) with Cobol and Visual Basic.)
I found that someone has already made this kind of DSL with MGrammar, but it is for Watin, not Selenium:
http://www.codinginstinct.com/2008/11/creating-watin-dsl-using-mgrammar.html
So the continuous integration server process will be:
- Fetch a new version from source control (as usual).
- Build, run unit tests and analyze the code (as usual).
- Make an installation package and tag version in version control (as usual).
- Compile use cases to web tests
- Run web tests
- Accept/Reject the software :-)
Running a web-test in the continuous integration server usually means a lot of configuration work. So, before I try this, I'm curious, what do you think?
Have you used same kind of setup, and what are your experiences? (What exact environment?)
How about DSL, will it have enough power for use cases or will it be another endless development task? Will the customers ever generate the tests?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
发布评论
评论(3)
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
TestPlan 软件使用专门的语言来编写测试。它具有高度特定领域性,并且在 Web 环境中运行良好。它支持 Selenium 后端,因此您可以获得兼容性,而且它可以在没有浏览器的情况下运行,从而实现更快的测试。我已经在一些相当大的网络项目中使用了它,这些项目正是您正在寻找的设置类型。
您的示例脚本可能如下所示:
就是这样。它很好地描述了用户想要做什么,并且是一个功能测试。您可以将脚本组合成单元并具有自定义功能。因此,如果您确实需要,可以将 Finish 等同于名称。
The TestPlan software uses a specialize language for writing tests. It is highly domain specific and works very well in web environments. It supports the Selenium backend so you gain that compatibility, plus it can run without a browser, for even faster tests. I have used it on some fairly large web projects in the type of setup you are looking for.
Your example script might look like this:
That's it. It nicely describes what the user wants to do and is a functioning test. You can combine scripts into units and have custom function as well. So if you really wanted you could write the Finish equivalents to the names.