SimpleTest WebTestCases 相互影响，而不是确定性输出
我正在使用 CodeIgniter 开发一些项目，并在 SimpleTest 中编写单元测试和 Web 测试。我注意到我的测试不是确定性的，即它们及时产生不同的输出。我的意思是测试用例应该是严格确定性的，而不是依赖于随机变量等。
测试看起来像是互相影响的。很多时候，当一切顺利时，我已经通过了 100 个测试，但是当我编写一个失败的新测试方法时，其他几个测试也会失败。但通常在纠正失败的测试用例中的问题并重新运行整个测试套件 2-3 次后，整个套件再次通过。
WebTestCases 通常会发生这种情况。
您知道可能出现什么问题吗？
我不修改任何共享的类变量等。
我浏览了 SimpleTest 的代码（或多或少，快速分析整个流程很大），看起来浏览器的实例是在启动不同的测试之前重新创建的。
最奇怪的是，重新运行后，一些错误消失了，最后全部消失了。这涉及到一些缓存吗？
我将不胜感激您的提示，因为除了网站上的 API 之外，网络上确实没有太多关于 SimpleTest 的文档/博客条目/论坛帖子。
I am developing some project with CodeIgniter and write unit tests and web tests in SimpleTest. I've noticed that my tests are not deterministic, i.e. they produce different outputs in time. I mean the test cases that should be strictly deterministic, not relying on random variables etc.
The tests look like affecting each other. Quite often, when everything goes okay, I have let's say 100 passed tests, but when I write a new test method that fails, then several other tests also do fail. But often after correcting the problem in my failing test case and re-running whole test suite 2-3 times whole suite gives a pass again.
This happens with WebTestCases generally.
Do you have any idea what could be the problem?
I do not modify any class variables that are shared etc.
I've glance at the code of SimpleTest (more or less, it's big to analyze whole flow quickly) and it looks like the instance of browser is re-created before launching different tests.
The thing that is the strangest is that after re-running, some errors disappear, and finally, all of them. Is there some caching involved in this?
I'll be grateful for hints as there is really not much documentation / blog entries / forum posts about SimpleTest in the web, except its API on the website.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
可能的事情：
在链条中的某个地方？
正在测试正确的东西吗？
数据库顶部，以及失败
损坏了数据库中的数据，
您可能会看到像您一样的结果
提到。
Things it might be:
somewhere in the chain?
are testing the right things?
top of a database, and the failure
corrupted the data in the database,
you might see results like you
mention.
（编辑：将答案移至单独的帖子）
呵呵，我进行了相当彻底的调查，似乎 SimpleTest 库中存在错误。
他们使用
fsockopen打开连接，然后通过fwrite发送请求，然后错误地从套接字获取响应。我的意思是：我们可能会从套接字读取 0 个字节，但我们还没有完成，正如我们错误地假设的那样，因为服务器可能很忙，并稍后发送数据，而我们提前结束了读取。这样，我们就没有读取整个响应，而是仅针对部分响应进行测试，导致其失败。(edit: moved the answer as a separate post)
Huh, I made quite thorough investigation and it seems that there is a bug in SimpleTest library.
They use
fsockopenfor opening connection, then send request viafwrite, and then incorrectly fetch response from socket. What I mean: it can happen that we read 0 bytes from socket, but we're not done, as we falsely assume, cause the server can be busy, and send data later, while we prematurely ended reading. That way, we haven't read whole response and we do tests against only partial response, causing it to fail.