如何使用unittest从测试或设置中停止所有测试？

发布于 2024-09-25 06:45:48 字数 266 浏览 2 评论 0原文

我正在扩展 python 2.7 unittest 框架来进行一些功能测试。我想做的一件事是停止所有测试在测试内部和 setUpClass() 方法内部运行。有时，如果测试失败，则程序已损坏，不再有任何继续测试的用处，因此我想停止测试运行。

我注意到 TestResult 有一个 shouldStop 属性和一个 stop() 方法，但我不确定如何在测试内部访问它。

有人有什么想法吗？有更好的办法吗？

原文

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

非要怀念 2024-10-02 06:45:48

如果您有兴趣，这里是一个简单的示例，您可以自己决定如何使用 py.test：

# content of test_module.py
import pytest
counter = 0
def setup_function(func):
    global counter
    counter += 1
    if counter >=3:
        pytest.exit("decided to stop the test run")

def test_one():
    pass
def test_two():
    pass
def test_three():
    pass

如果运行此命令，您会得到：

$ pytest test_module.py 
============== test session starts =================
platform linux2 -- Python 2.6.5 -- pytest-1.4.0a1
test path 1: test_module.py

test_module.py ..

!!!! Exit: decided to stop the test run !!!!!!!!!!!!
============= 2 passed in 0.08 seconds =============

您还可以将 py.test.exit() 调用放入测试中或项目特定的插件中。

旁注：py.test原生支持py.test --maxfail=NUM实现NUM次失败后停止。

Sidenote2：py.test 对以传统 unittest.TestCase 样式运行测试的支持有限。

In case you are interested, here is a simple example how you could make a decision yourself about exiting a test suite cleanly with py.test:

# content of test_module.py
import pytest
counter = 0
def setup_function(func):
    global counter
    counter += 1
    if counter >=3:
        pytest.exit("decided to stop the test run")

def test_one():
    pass
def test_two():
    pass
def test_three():
    pass

and if you run this you get:

$ pytest test_module.py 
============== test session starts =================
platform linux2 -- Python 2.6.5 -- pytest-1.4.0a1
test path 1: test_module.py

test_module.py ..

!!!! Exit: decided to stop the test run !!!!!!!!!!!!
============= 2 passed in 0.08 seconds =============

You can also put the py.test.exit() call inside a test or into a project-specific plugin.

Sidenote: py.test natively supports py.test --maxfail=NUM to implement stopping after NUM failures.

Sidenote2: py.test has only limited support for running tests in the traditional unittest.TestCase style.

回复收藏 0 原文

有木有妳兜一样 2024-10-02 06:45:48

这是我过了一段时间想出的另一个答案：

首先，我添加了一个新的异常：

class StopTests(Exception):
"""
Raise this exception in a test to stop the test run.

"""
    pass

然后我向我的子测试类添加了一个新的 assert：

def assertStopTestsIfFalse(self, statement, reason=''):
    try:
        assert statement            
    except AssertionError:
        result.addFailure(self, sys.exc_info())

最后我覆盖了 run 函数将其包含在 testMethod() 调用的正下方：

except StopTests:
    result.addFailure(self, sys.exc_info())
    result.stop()

我更喜欢这个，因为现在任何测试都能够停止所有测试，并且没有特定于 cpython 的代码。

Here's another answer I came up with after a while:

First, I added a new exception:

class StopTests(Exception):
"""
Raise this exception in a test to stop the test run.

"""
    pass

then I added a new assert to my child test class:

def assertStopTestsIfFalse(self, statement, reason=''):
    try:
        assert statement            
    except AssertionError:
        result.addFailure(self, sys.exc_info())

and last I overrode the run function to include this right below the testMethod() call:

except StopTests:
    result.addFailure(self, sys.exc_info())
    result.stop()

I like this better since any test now has the ability to stop all the tests, and there is no cpython-specific code.

回复收藏 0 原文

巨坚强 2024-10-02 06:45:48

目前，您只能停止套件级别的测试。一旦进入 TestCase，迭代测试时就不会使用 TestResult 的 stop() 方法。

与您的问题有些相关，如果您使用的是 python 2.7，则在使用 python -m unittest 调用测试时可以使用 -f/--failfast 标志。这将在第一次失败时停止测试。

请参阅25.3.2.1。 failfast、catch 和 buffer 命令行选项

您还可以考虑使用 Nose 运行测试并使用 -x, --stop 标志提前停止测试。

回复收藏 0 原文

他不在意 2024-10-02 06:45:48

OP 是关于 python 2.7 的。向前跳过十年，对于 python 3.1 和上面，在 python unittest 中跳过测试的方法已经升级了，但是文档可以使用一些说明（恕我直言）：文档

涵盖以下内容：

在第一次失败后跳过所有测试：使用 failfast （仅当您真的不想继续任何进一步的测试时有用，包括在其他不相关的 TestCase 类中）
跳过 TestCase 类中的所有测试：使用 @unittest.skip() 装饰类 等。
跳过 TestCase 中的单个方法：使用 @unittest.skip() 装饰方法，等等
有条件地跳过方法或类：使用 @unittest.skipIf() 装饰 或 @unittest.skipUnless() 等。
有条件地跳过一个方法，但直到该方法中的某些内容运行为止：在方法内使用 self.skipTest() （这将跳过该方法，并且仅跳过该方法，而不是后续方法）

该文档不涵盖以下内容（截至撰写本文时）：

如果 setUpClass 方法内满足条件，则跳过 TestCase 类中的所有测试：这篇文章的解决方案 raise unittest.SkipTest("跳过所有在此类中进行测试”）（可能还有另一种方法，但我不知道）
在第一个测试中满足条件后，跳过 TestCase 类中的所有后续测试方法，但仍继续测试其他测试方法不相关的 TestCase 类。为此，我提出以下解决方案...

该解决方案假设您在测试方法中间遇到“不良状态”，并且只能在测试方法中注意到（即，它不是可以的）无论出于何种原因，都已在 setUpClass 方法中确定）。事实上，setUpClass 方法是确定初始条件不正确时是否继续的最佳位置，但有时（正如我遇到的那样）直到运行某些测试方法时您才知道。此解决方案假设测试方法按字母顺序排列，并且您不想在遇到“不良”状态后运行的后续测试方法按字母顺序排列。

import unittest

class SkipMethodsConditionally(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        #this class variable maintains whether or not test methods should continue
        cls.should_continue = True
        #this class variable represents the state of your system. Replace with function of your own
        cls.some_bad_condition = False

    def setUp(self) -> None:
        """setUp runs before every single test method in this class"""
        if not self.__class__.should_continue:
            self.skipTest("no reason to go on.")

    def test_1_fail(self):
        #Do some work here. Let's assume you encounter a "bad state,"" that could 
        #only be noticed in this first test method only, (i.e., it's not something that
        #can be placed in the setUpClass method, for whatever reason)
        self.__class__.some_bad_condition = True

        if self.__class__.some_bad_condition:
            self.__class__.should_continue = False

        self.assertTrue(False,"this test should fail, rendering the rest of the tests irrelevant")

    def test_2_pass(self):
        self.assertFalse(self.__class__.some_bad_condition,"this test would pass normally if run, but should be skipped, because it would fail")

上述测试将产生以下输出：

test_1_fail (__main__.SkipMethodsConditionally) ... FAIL
test_2_pass (__main__.SkipMethodsConditionally) ... skipped 'no reason to go on.'
----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=1, skipped=1)

The OP was about python 2.7. Skip ahead a decade, and for python 3.1 and above, the way to skip tests in python unittest has had an upgrade, but the documentation could use some clarification (IMHO):

The documentation covers the following:

Skip All tests after first failure: use failfast (only useful if you really don't want to continue any further tests at all, including in other unrelated TestCase classes)
Skip All tests in a TestCase class: decorate class with @unittest.skip(), etc.
Skip a single method within a TestCase: decorate method with @unittest.skip(), etc.
Conditionally skip a method or a class: decorate with @unittest.skipIf() or @unittest.skipUnless() etc.
Conditionally skip a method, but not until something within that method runs: use self.skipTest() inside the method (this will skip that method, and ONLY that method, not subsequent methods)

The documentation does not cover the following (as of this writing):

Skip all tests within a TestCase class if a condition is met inside the setUpClass method: solution from this post raise unittest.SkipTest("skip all tests in this class") (there may be another way, but I'm unaware)
Skip all subsequent test methods in a TestCase class after a condition is met in one of the first tests, but still continue to test other unrelated TestCase classes. For this, I propose the following solution...

This solution assumes that you encounter the "bad state" in the middle of a test method, and which could only be noticed in a test method ONLY (i.e., it is not something that could have been determined in the setUpClass method, for whatever reason). Indeed the setUpClass method is the best location for determining whether to proceed if the initial conditions aren't right, but sometimes (as I've encountered) you just don't know until you run some test method. This solution assumes that test methods are in alphabetical order and that subsequent tests methods that you don't want to run after encountering the "bad" state follow alphabetically.

import unittest

class SkipMethodsConditionally(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        #this class variable maintains whether or not test methods should continue
        cls.should_continue = True
        #this class variable represents the state of your system. Replace with function of your own
        cls.some_bad_condition = False

    def setUp(self) -> None:
        """setUp runs before every single test method in this class"""
        if not self.__class__.should_continue:
            self.skipTest("no reason to go on.")

    def test_1_fail(self):
        #Do some work here. Let's assume you encounter a "bad state,"" that could 
        #only be noticed in this first test method only, (i.e., it's not something that
        #can be placed in the setUpClass method, for whatever reason)
        self.__class__.some_bad_condition = True

        if self.__class__.some_bad_condition:
            self.__class__.should_continue = False

        self.assertTrue(False,"this test should fail, rendering the rest of the tests irrelevant")

    def test_2_pass(self):
        self.assertFalse(self.__class__.some_bad_condition,"this test would pass normally if run, but should be skipped, because it would fail")

The above test will yield the following output:

test_1_fail (__main__.SkipMethodsConditionally) ... FAIL
test_2_pass (__main__.SkipMethodsConditionally) ... skipped 'no reason to go on.'
----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=1, skipped=1)

回复收藏 0 原文

又怨 2024-10-02 06:45:48

尽管您不会获得到目前为止运行的测试的常规测试报告，但从 TestCase 方法中停止测试运行的一个非常简单的方法就是引发 KeyboardInterrupt方法内部。

通过查看 CPython 的代码，您可以了解如何仅允许 KeyboardInterrupt 在 unittest 的测试运行程序中冒泡此处在testPartExecutor()内。

回复收藏 0 原文

一桥轻雨一伞开 2024-10-02 06:45:48

在 unittest.TestSuite 的测试循环中，开始时有一个 break 条件：

class TestSuite(BaseTestSuite):

    def run(self, result, debug=False):
        topLevel = False
        if getattr(result, '_testRunEntered', False) is False:
            result._testRunEntered = topLevel = True

        for test in self:
            if result.shouldStop:
                break

所以我使用像这样的自定义测试套件：

class CustomTestSuite(unittest.TestSuite):
    """ This variant registers the test result object with all ScriptedTests,
        so that a failed Loign test can abort the test suite by setting result.shouldStop
        to True
    """
    def run(self, result, debug=False):
        for test in self._tests:
            test.result = result

        return super(CustomTestSuite, self).run(result, debug)

带有像这样的自定义测试结果类这：

class CustomTestResult(TextTestResult):
    def __init__(self, stream, descriptions, verbosity):
        super(CustomTestResult, self).__init__(stream, descriptions, verbosity)
        self.verbosity = verbosity
        self.shouldStop = False

我的测试类就像：

class ScriptedTest(unittest.TestCase):
    def __init__(self, environment, test_cfg, module, test):
        super(ScriptedTest, self).__init__()
        self.result = None

在某些条件下，我然后中止测试套件；例如，测试套件从登录开始，如果失败，我不必尝试其余的：

    try:
        test_case.execute_script(test_command_list)
    except AssertionError as e:
        if test_case.module == 'session' and test_case.test == 'Login':
            test_case.result.shouldStop = True
            raise TestFatal('Login failed, aborting test.')
        else:
            raise sys.exc_info()

然后我按以下方式使用测试套件：

    suite = CustomTestSuite()

    self.add_tests(suite)

    result = unittest.TextTestRunner(verbosity=self.environment.verbosity, stream=UnitTestLoggerStream(self.logger),
                                     resultclass=CustomTestResult).run(suite)

我不确定是否有更好的方法来做到这一点，但它对于我的测试表现正确。

In the test loop of unittest.TestSuite, there is a break condition at the start:

class TestSuite(BaseTestSuite):

    def run(self, result, debug=False):
        topLevel = False
        if getattr(result, '_testRunEntered', False) is False:
            result._testRunEntered = topLevel = True

        for test in self:
            if result.shouldStop:
                break

So I am using a custom test suite like this:

class CustomTestSuite(unittest.TestSuite):
    """ This variant registers the test result object with all ScriptedTests,
        so that a failed Loign test can abort the test suite by setting result.shouldStop
        to True
    """
    def run(self, result, debug=False):
        for test in self._tests:
            test.result = result

        return super(CustomTestSuite, self).run(result, debug)

with a custom test result class like this:

class CustomTestResult(TextTestResult):
    def __init__(self, stream, descriptions, verbosity):
        super(CustomTestResult, self).__init__(stream, descriptions, verbosity)
        self.verbosity = verbosity
        self.shouldStop = False

and my test classes are like:

class ScriptedTest(unittest.TestCase):
    def __init__(self, environment, test_cfg, module, test):
        super(ScriptedTest, self).__init__()
        self.result = None

Under certain conditions, I then abort the test suite; for example, the test suite starts with a login, and if that fails, I do not have to try the rest:

    try:
        test_case.execute_script(test_command_list)
    except AssertionError as e:
        if test_case.module == 'session' and test_case.test == 'Login':
            test_case.result.shouldStop = True
            raise TestFatal('Login failed, aborting test.')
        else:
            raise sys.exc_info()

Then I use the test suite in the following way:

    suite = CustomTestSuite()

    self.add_tests(suite)

    result = unittest.TextTestRunner(verbosity=self.environment.verbosity, stream=UnitTestLoggerStream(self.logger),
                                     resultclass=CustomTestResult).run(suite)

I'm not sure if there is a better way to do it, but it behaves correctly for my tests.

回复收藏 0 原文

好倦 2024-10-02 06:45:48

我查看了 TestCase 类并决定对其进行子类化。该类只是重写run()。我复制了该方法，并从原始类的第 318 行开始添加了以下内容：

# this is CPython specific. Jython and IronPython may do this differently
if testMethod.func_code.co_argcount == 2:
    testMethod(result)
else:
    testMethod()

它有一些 CPython 特定代码，用于判断测试方法是否可以接受另一个参数，但由于我到处都使用 CPython，所以这不是问题为我。

I looked at the TestCase class and decided to subclass it. The class just overrides run(). I copied the method and starting at line 318 in the original class added this:

# this is CPython specific. Jython and IronPython may do this differently
if testMethod.func_code.co_argcount == 2:
    testMethod(result)
else:
    testMethod()

It has some CPython specific code in there to tell if the test method can accept another parameter, but since I'm using CPython everywhere, this isn't an issue for me.

回复收藏 0 原文