That is the problem with detailed test plan up front. You trying to guess what kind of errors, how many, and in what areas you will get. This may be tricky.
Maybe you should have overall Master Test Plan specifying only test strategy, main tool set, risks, relative amount of how much testing you want to put in given areas (based on risk).
Then when you starting to work on given functionality or iteration (I hope you are doing this in iterations not waterfall), you prepare detailed test plan for this set of work. You adjust your tools/estimates/test coverage based on experiences from previous parts.
This way you can say at the beginning what is your general approach and priorities, but you let yourself adapt later as project progresses.
Question about how much testing you need to put into testing COTS is the same as with any software: you need to evaluate the risk.
If your software need to be Validated because of external
regulations (FDA,DoD..)
you will need to go deep with your
tests, almost test entire app. One
problem here may be ensuring
external regulator, that tools you
used for validation are validated
(and that is a troublesome).
If your application is
mission-critical for your company,
than you still need to do a lot of
testing based on extensive risk
analysis.
If your application is not concerned
with all above, you can go with
lighter testing. Probably you can
skip functionality that was tested
by platform manufacturer, and focus
on your customisations. On the other
hand I would still write tests (at
least happy paths) for
workflows you will be using in your
business processes.
When we started learning Selenium testing in 2008 we created Recruiting application from SalesForce handbook and created a suite of tests and described our path step by step in our blog. It may help you get started if you decide to write Selenium code to test your app.
I believe the problem with SalesForce is you have Unit and UI testing, but no Service-level testing. The SpecFlow I've seen which drives Selenium UI is brittle and doesn't encapsulate what I'm after in engineering a service-level test solution:
( When I navigate to "/Selenium-Testing-Cookbook-Gundecha-Unmesh/dp/1849515743"
And I click the 'buy now' button
And then I click the 'proceed to checkout' button)
That is not the spirit or intent of Specflow.
Given I have not selected a product
When I select Proceed to Checkout
Then ensure I am presented with a message
In order to test that with selenium, you essentially have to translate that to clicks and typing, whereas in the .NET realm, you can instantiate objects, etc., in the middle-tier, and perform hundreds of instances and derivations against the same BACKGROUND (mock setup).
I'm told that you can expose SF through an API at some security risk. I'd love to find more about THAT.
发布评论
评论（3）
这就是预先制定详细测试计划的问题。你试图猜测你会在哪些领域出现什么样的错误、多少错误。这可能很棘手。
也许您应该制定总体主测试计划，仅指定测试策略、主要工具集、风险、您想要在给定区域进行的测试的相对数量（基于风险）。
然后，当您开始处理给定的功能或迭代时（我希望您在迭代而不是瀑布中进行此操作），您将为这组工作准备详细的测试计划。您可以根据之前部分的经验调整您的工具/估计/测试覆盖范围。
这样，您可以在一开始就说出您的总体方法和优先事项是什么，但您可以随着项目的进展让自己适应。
关于需要进行多少测试的问题 COTS 与任何软件：您需要评估风险。
如果您的软件需要
已验证，因为外部
法规（FDA，DoD..)
你需要深入了解你的
测试，几乎测试整个应用程序。一
这里的问题可能是确保
外部调节器，为您提供工具
用于验证的已验证
（这很麻烦）。
如果您的应用程序是
对您的公司至关重要的任务，
比你还需要做很多
基于广泛风险的测试
分析。
如果您的申请不受影响
有了以上所有，你可以去
打火机测试。也许你可以
跳过已测试的功能
按平台制造商和重点
关于您的定制。另一方面
我仍然会编写测试（在
至少快乐路径）
您将在您的项目中使用的工作流程
业务流程。
That is the problem with detailed test plan up front. You trying to guess what kind of errors, how many, and in what areas you will get. This may be tricky.
Maybe you should have overall Master Test Plan specifying only test strategy, main tool set, risks, relative amount of how much testing you want to put in given areas (based on risk).
Then when you starting to work on given functionality or iteration (I hope you are doing this in iterations not waterfall), you prepare detailed test plan for this set of work. You adjust your tools/estimates/test coverage based on experiences from previous parts.
This way you can say at the beginning what is your general approach and priorities, but you let yourself adapt later as project progresses.
Question about how much testing you need to put into testing COTS is the same as with any software: you need to evaluate the risk.
If your software need to be
Validated because of external
regulations (FDA,DoD..)
you will need to go deep with your
tests, almost test entire app. One
problem here may be ensuring
external regulator, that tools you
used for validation are validated
(and that is a troublesome).
If your application is
mission-critical for your company,
than you still need to do a lot of
testing based on extensive risk
analysis.
If your application is not concerned
with all above, you can go with
lighter testing. Probably you can
skip functionality that was tested
by platform manufacturer, and focus
on your customisations. On the other
hand I would still write tests (at
least happy paths) for
workflows you will be using in your
business processes.
当我们在 2008 年开始学习 Selenium 测试时，我们根据 SalesForce 手册创建了招聘应用程序，并创建了一套测试，并在 博客。如果您决定编写 Selenium 代码来测试您的应用程序，它可能会帮助您入门。
When we started learning Selenium testing in 2008 we created Recruiting application from SalesForce handbook and created a suite of tests and described our path step by step in our blog. It may help you get started if you decide to write Selenium code to test your app.
我认为 SalesForce 的问题在于您有单元和 UI 测试，但没有服务级别测试。我见过的驱动 Selenium UI 的 SpecFlow 很脆弱，并且没有封装我在设计服务级测试解决方案时所追求的内容：（
当我导航到“/Selenium-Testing-Cookbook-Gundecha-Unmesh/dp/ 1849515743”
我点击“立即购买”按钮
然后我单击“继续结帐”按钮）
这不是 Specflow 的精神或意图。
鉴于我还没有选择产品
当我选择继续结帐时
然后确保我看到一条消息。
为了使用 selenium 进行测试，您基本上必须将其转换为单击和键入，而在 .NET 领域中，您可以在中间层实例化对象等，然后执行针对相同背景的数百个实例和派生（模拟设置）。
我听说您可以通过 API 公开 SF，但存在一定的安全风险。我很想找到更多相关信息。
I believe the problem with SalesForce is you have Unit and UI testing, but no Service-level testing. The SpecFlow I've seen which drives Selenium UI is brittle and doesn't encapsulate what I'm after in engineering a service-level test solution:
( When I navigate to "/Selenium-Testing-Cookbook-Gundecha-Unmesh/dp/1849515743"
And I click the 'buy now' button
And then I click the 'proceed to checkout' button)
That is not the spirit or intent of Specflow.
Given I have not selected a product
When I select Proceed to Checkout
Then ensure I am presented with a message
In order to test that with selenium, you essentially have to translate that to clicks and typing, whereas in the .NET realm, you can instantiate objects, etc., in the middle-tier, and perform hundreds of instances and derivations against the same BACKGROUND (mock setup).
I'm told that you can expose SF through an API at some security risk. I'd love to find more about THAT.