当前位置： 文江博客 话题详情

问题使螺纹与硒和报告 - 港口合作

发布于 2025-01-23 09:58:17 字数 11222 浏览 0 评论 0 原文

我遇到了一种我根本不了解的行为。 在我决定附加reportportal之前，用硒实现螺纹tcpserver的工作正常...

为了简单起见 - 我试图将代码煮沸至重现问题所需的最低限度：

import http.server
import socketserver
from typing import Optional
from selenium.webdriver.chrome.webdriver import WebDriver


class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
    pass


def title_extractor(
    headless: bool = True
):
    import concurrent.futures

    with concurrent.futures.ThreadPoolExecutor() as executor:
        host, port = "localhost", 8050

        with ThreadedTCPServer((host, port), http.server.SimpleHTTPRequestHandler) as server:
            print(f"serving at port {port}")

            # Start a thread for the server
            print('Starting server ...')
            executor.submit(server.serve_forever)
            # Start a thread for selenium
            print('Starting page_worker ...')
            future_nonce = executor.submit(page_worker, None, headless)
            print('... page_worker done')

            done_set, not_done_set = concurrent.futures.wait(
                [future_nonce],
                timeout=300,
                return_when=concurrent.futures.ALL_COMPLETED
            )
            for future in done_set:
                if future.done():
                    print('encountered future.done()')
                    pass
                if future.cancelled():
                    print('encountered future.cancelled()')
                    pass
                if future.exception(timeout=3):
                    print(f'encountered future.exception(): {str(future.exception())}')
                    result = None
                    continue
                    # raise RuntimeError(str(future.exception()))

                print(f'future: {str(future)}')
                result = future.result(timeout=3)
                print('... future result finished')
                print(f"Result:\t{result}")

            print('Shutting down server ...')
            server.shutdown()
            executor.shutdown(wait=False)
            print('... server and executor shut down')

            return result


def page_worker(
    driver: Optional[WebDriver] = None,
    headless: bool = True
) -> Optional[str]:
    import selenium
    from selenium.webdriver.chrome.options import Options

    terminate_driver = False

    options = Options()
    options.add_argument("--no-sandbox")
    options.add_argument("--ignore-certificate-errors")
    if headless:
        options.add_argument("--headless")

    # Start a new driver and stop it afterwards at the end of the worker call
    if driver is None:
        terminate_driver = True
        url = "https://google.com"
        print(f"Starting chrome driver with URL={url}\n"
              f"headless={headless}\toptions={str(options)}")
        driver = selenium.webdriver.Chrome(options=options)
        driver.get(url)
        print(f"driver={driver}")

    title = driver.title

    if terminate_driver:
        print("Stopping driver")
        driver.quit()

    print(f"Worker terminating")

    return title


def test__page_worker_with_report_portal():
    title = title_extractor(headless=True)
    print(f"Worker finished. Page title: {title}")

这是呼叫（没有ReportPortal集成）作品：

python3 -m pytest -show -capture = all -s -v ./tests/debugging/noncewithrp.py

以下是结果：

(venv) bartoszc@bartoszclnx:~/PycharmProjects/ryd-tests$ python3 -m pytest --show-capture=all -s -v ./Tests/Debugging/NonceWithRP.py
MARKERS: {''}
======================================================= test session starts =======================================================
platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.0.0 -- .../venv/bin/python3
cachedir: .pytest_cache
rootdir: ..., configfile: pytest.ini
plugins: allure-pytest-2.9.45, reportportal-5.1.0, repeat-0.9.1
collected 1 item                                                                                                                  

Tests/Debugging/NonceWithRP.py::test__page_worker_with_report_portal serving at port 8050
Starting server ...
Starting page_worker ...
Starting chrome driver with URL=https://google.com
headless=True   options=<selenium.webdriver.chrome.options.Options object at 0x7fa3149e8eb0>... page_worker done


---------------------------------------------------------- live log call ----------------------------------------------------------
DEBUG 17:38:10: POST http://localhost:58501/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "args": ["--no-sandbox", "--ignore-certificate-errors", "--headless"]}}}, "desiredCapabilities": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "args": ["--no-sandbox", "--ignore-certificate-errors", "--headless"]}}}
DEBUG 17:38:10: Starting new HTTP connection (1): localhost:58501
DEBUG 17:38:10: http://localhost:58501 "POST /session HTTP/1.1" 200 752
DEBUG 17:38:10: Finished Request
DEBUG 17:38:10: POST http://localhost:58501/session/cbf04df70cb13b87b74624492c5682ae/url {"url": "https://google.com"}
DEBUG 17:38:11: http://localhost:58501 "POST /session/cbf04df70cb13b87b74624492c5682ae/url HTTP/1.1" 200 14
DEBUG 17:38:11: Finished Request
driver=<selenium.webdriver.chrome.webdriver.WebDriver (session="cbf04df70cb13b87b74624492c5682ae")>
DEBUG 17:38:11: GET http://localhost:58501/session/cbf04df70cb13b87b74624492c5682ae/title {}
DEBUG 17:38:11: http://localhost:58501 "GET /session/cbf04df70cb13b87b74624492c5682ae/title HTTP/1.1" 200 18
DEBUG 17:38:11: Finished Request
Stopping driver
DEBUG 17:38:11: DELETE http://localhost:58501/session/cbf04df70cb13b87b74624492c5682ae {}
DEBUG 17:38:11: http://localhost:58501 "DELETE /session/cbf04df70cb13b87b74624492c5682ae HTTP/1.1" 200 14
DEBUG 17:38:11: Finished Request
Worker terminating
encountered future.done()
future: <Future at 0x7fa314b0e340 state=finished returned str>
... future result finished
Result: Google
Shutting down server ...
... server and executor shut down
Worker finished. Page title: Google
PASSED

以下是reporterportal intemantation new call in Reportportal intemantation，它破裂：

<代码： python3 -m pytest -reportportal -show -capture = all -s -v ./tests/debugging/noncewithrp.py

以下是结果：

MARKERS: {''}

------------------------------------------------------ live log sessionstart ------------------------------------------------------
DEBUG 17:41:37: ReportPortal - Init service: endpoint=...
DEBUG 17:41:37: Starting new HTTP connection (1): ...
DEBUG 17:41:37: ... "GET /api/v1/testchuck/settings HTTP/1.1" 200 None
DEBUG 17:41:37: ReportPortal - Start launch: request_body=...
DEBUG 17:41:37: ... "POST /api/v2/testchuck/launch HTTP/1.1" 201 None
DEBUG 17:41:37: start_launch - ID: ...
DEBUG 17:41:37: ReportPortal - Launch started: id=...
DEBUG 17:41:37: Starting new HTTPS connection (1): www.google-analytics.com:443
DEBUG 17:41:38: https://www.google-analytics.com:443 "POST /collect HTTP/1.1" 200 35
======================================================= test session starts =======================================================
platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.0.0 -- .../venv/bin/python3
cachedir: .pytest_cache
rootdir: ..., configfile: pytest.ini
plugins: allure-pytest-2.9.45, reportportal-5.1.0, repeat-0.9.1
collected 1 item                                                                                                                  
------------------------------------------------------- live log collection -------------------------------------------------------
DEBUG 17:41:38: ReportPortal - Start TestItem: request_body={'attributes': [], 'name': 'Tests/Debugging/NonceWithRP.py::test__page_worker_with_report_portal', 'description': None, 'start_time': '1650555698079', 'item_type': 'STEP', 'code_ref': 'Tests/Debugging/NonceWithRP.py:test__page_worker_with_report_portal', 'parameters': None, 'parent_item_id': None, 'test_case_id': 'Tests/Debugging/NonceWithRP.py:test__page_worker_with_report_portal'}
DEBUG 17:41:38: ... "POST /api/v2/testchuck/item HTTP/1.1" 201 None
DEBUG 17:41:38: start_test_item - ID: ...

Tests/Debugging/NonceWithRP.py::test__page_worker_with_report_portal serving at port 8050
Starting server ...
Starting page_worker ...
Starting chrome driver with URL=https://google.com
headless=True   options=<selenium.webdriver.chrome.options.Options object at 0x7fcac84b5370>
... page_worker done

---------------------------------------------------------- live log call ----------------------------------------------------------
DEBUG 17:41:38: POST http://localhost:56813/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "args": ["--no-sandbox", "--ignore-certificate-errors", "--headless"]}}}, "desiredCapabilities": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "args": ["--no-sandbox", "--ignore-certificate-errors", "--headless"]}}}
encountered future.done()
encountered future.exception(): 'NoneType' object has no attribute 'log'
Shutting down server ...
... server and executor shut down
Worker finished. Page title: None
PASSED
--------------------------------------------------------- live log finish ---------------------------------------------------------
DEBUG 17:41:39: ReportPortal - Finish TestItem: request_body={'end_time': '1650555699144', 'status': 'PASSED', 'issue': None, 'item_id': '6647d85e-4c35-4798-acc5-17b18dcb5c48'}
DEBUG 17:41:39: ... "PUT /api/v2/testchuck/item/6647d85e-4c35-4798-acc5-17b18dcb5c48 HTTP/1.1" 200 None
DEBUG 17:41:39: finish_test_item - ID: 6647d85e-4c35-4798-acc5-17b18dcb5c48
DEBUG 17:41:39: response message: Accepted finish request for test item ID = 6647d85e-4c35-4798-acc5-17b18dcb5c48
----------------------------------------------------- live log sessionfinish ------------------------------------------------------
DEBUG 17:41:39: ReportPortal - Finish launch: request_body={'end_time': '1650555699384'}
DEBUG 17:41:39: ... "PUT /api/v2/testchuck/launch/1c4b1457-4e0a-4804-8749-391f77899385/finish HTTP/1.1" 200 None
DEBUG 17:41:39: finish_launch - ID: 1c4b1457-4e0a-4804-8749-391f77899385
DEBUG 17:41:39: response message: <reportportal_client.static.defines._PresenceSentinel object at 0x7fcac9ed9cd0>
DEBUG 17:41:39: Waiting for worker <reportportal_client.core.worker.APIWorker object at 0x7fcac8e27550> to completeprocessing batches.
DEBUG 17:41:39: [APIWorker] Received {ControlCommand.STOP} command
DEBUG 17:41:39: [APIWorker] Processing {ControlCommand.STOP} command
DEBUG 17:41:39: [APIWorker] Exiting due to {ControlCommand.STOP} command

看来，记录似乎存在问题由Web驱动器调用。 我不明白的是 - 当log.debug消息开始时，为什么Future_nonce终止为“完成” - 仅在第一个调试消息之后？

为了最终我需要做的事情 - 我确实需要具有多线程解决方案，其中一个线程在页面上使用，另一个线程在页面上运行Selenium Automation。上面的代码只是重现问题的最小实施。

有人可以帮助我获得相同的行为（含义：page_worker的完整执行，返回页面标题），但是在此“基础架构”中使用ReportPortal集成？

原文

I've encountered a behavior that I simply do not understand.
Implementing a ThreadedTCPServer with selenium was working fine until I decided to attach reportportal...

For the sake of simplicity - I've tried to boil down the code to the minimum necessary to reproduce the issue:

import http.server
import socketserver
from typing import Optional
from selenium.webdriver.chrome.webdriver import WebDriver


class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
    pass


def title_extractor(
    headless: bool = True
):
    import concurrent.futures

    with concurrent.futures.ThreadPoolExecutor() as executor:
        host, port = "localhost", 8050

        with ThreadedTCPServer((host, port), http.server.SimpleHTTPRequestHandler) as server:
            print(f"serving at port {port}")

            # Start a thread for the server
            print('Starting server ...')
            executor.submit(server.serve_forever)
            # Start a thread for selenium
            print('Starting page_worker ...')
            future_nonce = executor.submit(page_worker, None, headless)
            print('... page_worker done')

            done_set, not_done_set = concurrent.futures.wait(
                [future_nonce],
                timeout=300,
                return_when=concurrent.futures.ALL_COMPLETED
            )
            for future in done_set:
                if future.done():
                    print('encountered future.done()')
                    pass
                if future.cancelled():
                    print('encountered future.cancelled()')
                    pass
                if future.exception(timeout=3):
                    print(f'encountered future.exception(): {str(future.exception())}')
                    result = None
                    continue
                    # raise RuntimeError(str(future.exception()))

                print(f'future: {str(future)}')
                result = future.result(timeout=3)
                print('... future result finished')
                print(f"Result:\t{result}")

            print('Shutting down server ...')
            server.shutdown()
            executor.shutdown(wait=False)
            print('... server and executor shut down')

            return result


def page_worker(
    driver: Optional[WebDriver] = None,
    headless: bool = True
) -> Optional[str]:
    import selenium
    from selenium.webdriver.chrome.options import Options

    terminate_driver = False

    options = Options()
    options.add_argument("--no-sandbox")
    options.add_argument("--ignore-certificate-errors")
    if headless:
        options.add_argument("--headless")

    # Start a new driver and stop it afterwards at the end of the worker call
    if driver is None:
        terminate_driver = True
        url = "https://google.com"
        print(f"Starting chrome driver with URL={url}\n"
              f"headless={headless}\toptions={str(options)}")
        driver = selenium.webdriver.Chrome(options=options)
        driver.get(url)
        print(f"driver={driver}")

    title = driver.title

    if terminate_driver:
        print("Stopping driver")
        driver.quit()

    print(f"Worker terminating")

    return title


def test__page_worker_with_report_portal():
    title = title_extractor(headless=True)
    print(f"Worker finished. Page title: {title}")

Here's the call (without reportportal integration) that works:

python3 -m pytest --show-capture=all -s -v ./Tests/Debugging/NonceWithRP.py

Here's the result:

(venv) bartoszc@bartoszclnx:~/PycharmProjects/ryd-tests$ python3 -m pytest --show-capture=all -s -v ./Tests/Debugging/NonceWithRP.py
MARKERS: {''}
======================================================= test session starts =======================================================
platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.0.0 -- .../venv/bin/python3
cachedir: .pytest_cache
rootdir: ..., configfile: pytest.ini
plugins: allure-pytest-2.9.45, reportportal-5.1.0, repeat-0.9.1
collected 1 item                                                                                                                  

Tests/Debugging/NonceWithRP.py::test__page_worker_with_report_portal serving at port 8050
Starting server ...
Starting page_worker ...
Starting chrome driver with URL=https://google.com
headless=True   options=<selenium.webdriver.chrome.options.Options object at 0x7fa3149e8eb0>... page_worker done


---------------------------------------------------------- live log call ----------------------------------------------------------
DEBUG 17:38:10: POST http://localhost:58501/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "args": ["--no-sandbox", "--ignore-certificate-errors", "--headless"]}}}, "desiredCapabilities": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "args": ["--no-sandbox", "--ignore-certificate-errors", "--headless"]}}}
DEBUG 17:38:10: Starting new HTTP connection (1): localhost:58501
DEBUG 17:38:10: http://localhost:58501 "POST /session HTTP/1.1" 200 752
DEBUG 17:38:10: Finished Request
DEBUG 17:38:10: POST http://localhost:58501/session/cbf04df70cb13b87b74624492c5682ae/url {"url": "https://google.com"}
DEBUG 17:38:11: http://localhost:58501 "POST /session/cbf04df70cb13b87b74624492c5682ae/url HTTP/1.1" 200 14
DEBUG 17:38:11: Finished Request
driver=<selenium.webdriver.chrome.webdriver.WebDriver (session="cbf04df70cb13b87b74624492c5682ae")>
DEBUG 17:38:11: GET http://localhost:58501/session/cbf04df70cb13b87b74624492c5682ae/title {}
DEBUG 17:38:11: http://localhost:58501 "GET /session/cbf04df70cb13b87b74624492c5682ae/title HTTP/1.1" 200 18
DEBUG 17:38:11: Finished Request
Stopping driver
DEBUG 17:38:11: DELETE http://localhost:58501/session/cbf04df70cb13b87b74624492c5682ae {}
DEBUG 17:38:11: http://localhost:58501 "DELETE /session/cbf04df70cb13b87b74624492c5682ae HTTP/1.1" 200 14
DEBUG 17:38:11: Finished Request
Worker terminating
encountered future.done()
future: <Future at 0x7fa314b0e340 state=finished returned str>
... future result finished
Result: Google
Shutting down server ...
... server and executor shut down
Worker finished. Page title: Google
PASSED

Here's the call with reportportal integration, where it breaks:

python3 -m pytest --reportportal --show-capture=all -s -v ./Tests/Debugging/NonceWithRP.py

Here's the result:

MARKERS: {''}

------------------------------------------------------ live log sessionstart ------------------------------------------------------
DEBUG 17:41:37: ReportPortal - Init service: endpoint=...
DEBUG 17:41:37: Starting new HTTP connection (1): ...
DEBUG 17:41:37: ... "GET /api/v1/testchuck/settings HTTP/1.1" 200 None
DEBUG 17:41:37: ReportPortal - Start launch: request_body=...
DEBUG 17:41:37: ... "POST /api/v2/testchuck/launch HTTP/1.1" 201 None
DEBUG 17:41:37: start_launch - ID: ...
DEBUG 17:41:37: ReportPortal - Launch started: id=...
DEBUG 17:41:37: Starting new HTTPS connection (1): www.google-analytics.com:443
DEBUG 17:41:38: https://www.google-analytics.com:443 "POST /collect HTTP/1.1" 200 35
======================================================= test session starts =======================================================
platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.0.0 -- .../venv/bin/python3
cachedir: .pytest_cache
rootdir: ..., configfile: pytest.ini
plugins: allure-pytest-2.9.45, reportportal-5.1.0, repeat-0.9.1
collected 1 item                                                                                                                  
------------------------------------------------------- live log collection -------------------------------------------------------
DEBUG 17:41:38: ReportPortal - Start TestItem: request_body={'attributes': [], 'name': 'Tests/Debugging/NonceWithRP.py::test__page_worker_with_report_portal', 'description': None, 'start_time': '1650555698079', 'item_type': 'STEP', 'code_ref': 'Tests/Debugging/NonceWithRP.py:test__page_worker_with_report_portal', 'parameters': None, 'parent_item_id': None, 'test_case_id': 'Tests/Debugging/NonceWithRP.py:test__page_worker_with_report_portal'}
DEBUG 17:41:38: ... "POST /api/v2/testchuck/item HTTP/1.1" 201 None
DEBUG 17:41:38: start_test_item - ID: ...

Tests/Debugging/NonceWithRP.py::test__page_worker_with_report_portal serving at port 8050
Starting server ...
Starting page_worker ...
Starting chrome driver with URL=https://google.com
headless=True   options=<selenium.webdriver.chrome.options.Options object at 0x7fcac84b5370>
... page_worker done

---------------------------------------------------------- live log call ----------------------------------------------------------
DEBUG 17:41:38: POST http://localhost:56813/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "args": ["--no-sandbox", "--ignore-certificate-errors", "--headless"]}}}, "desiredCapabilities": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "args": ["--no-sandbox", "--ignore-certificate-errors", "--headless"]}}}
encountered future.done()
encountered future.exception(): 'NoneType' object has no attribute 'log'
Shutting down server ...
... server and executor shut down
Worker finished. Page title: None
PASSED
--------------------------------------------------------- live log finish ---------------------------------------------------------
DEBUG 17:41:39: ReportPortal - Finish TestItem: request_body={'end_time': '1650555699144', 'status': 'PASSED', 'issue': None, 'item_id': '6647d85e-4c35-4798-acc5-17b18dcb5c48'}
DEBUG 17:41:39: ... "PUT /api/v2/testchuck/item/6647d85e-4c35-4798-acc5-17b18dcb5c48 HTTP/1.1" 200 None
DEBUG 17:41:39: finish_test_item - ID: 6647d85e-4c35-4798-acc5-17b18dcb5c48
DEBUG 17:41:39: response message: Accepted finish request for test item ID = 6647d85e-4c35-4798-acc5-17b18dcb5c48
----------------------------------------------------- live log sessionfinish ------------------------------------------------------
DEBUG 17:41:39: ReportPortal - Finish launch: request_body={'end_time': '1650555699384'}
DEBUG 17:41:39: ... "PUT /api/v2/testchuck/launch/1c4b1457-4e0a-4804-8749-391f77899385/finish HTTP/1.1" 200 None
DEBUG 17:41:39: finish_launch - ID: 1c4b1457-4e0a-4804-8749-391f77899385
DEBUG 17:41:39: response message: <reportportal_client.static.defines._PresenceSentinel object at 0x7fcac9ed9cd0>
DEBUG 17:41:39: Waiting for worker <reportportal_client.core.worker.APIWorker object at 0x7fcac8e27550> to completeprocessing batches.
DEBUG 17:41:39: [APIWorker] Received {ControlCommand.STOP} command
DEBUG 17:41:39: [APIWorker] Processing {ControlCommand.STOP} command
DEBUG 17:41:39: [APIWorker] Exiting due to {ControlCommand.STOP} command

It seems that there is a problem with the logging that seems to be invoked by the web-driver.
What I don't understand is - why does the future_nonce terminate as 'done' when the log.debug messages begin - and only after the first debug message?

For the sake of what I need to do in the end - I do need to have the multi-threaded solution, where one thread serves the page, another thread operates selenium automation on the page. The code above is just a minimal implementation to reproduce the issue.

Could someone please help me in obtaining the same behavior (meaning: a complete execution of the page_worker, returning the page title), but with reportportal integration in this 'infrastructure' ?

如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论， 你可以免费 注册 一个本站的账号。

评论（1）

蓝眼睛不忧郁 2025-01-30 09:58:17

这是一个错误的内部报告门户网站客户端，该端子已固定在版本 5.2.2.2
请使用客户端的最新版本： https://pypi.org/project/project/reporportalal-client/

This was a bug inside Report Portal client which is fixed in version 5.2.2
Please use the latest version of the client: https://pypi.org/project/reportportal-client/

回复 原文
~没有更多了~

关于作者

瑕疵

暂无简介

文章
评论
26 人气
发私信

相关话题

更多

热门标签

操作系统 程序设计 IT运维 Linux系统管理 JavaScript 服务器应用 solaris C/C++ PHP Shell BSD Vue.js aix Oracle Python HTML 系统管理 HTML5 CSS 前端
更多

推荐作者

檐前雨

文章 0 评论 0

鹿港巷口少年归

文章 0 评论 0

qq_32QL4xcD

文章 0 评论 0

sum_

文章 0 评论 0

DLL

文章 0 评论 0

唐婉

文章 0 评论 0

更多

友情链接

文江博客
    我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站，即表示您同意使用 Cookies 和您的相关数据。
    原文