selenium.common.exceptions.webdriverexception:消息:Chromedriver意外退出。状态代码是:255使用Dockerfile

发布于 2025-01-23 14:56:25 字数 5019 浏览 0 评论 0原文

我有一个Python Webscraping应用程序,可以在我的Localhost(MacOS,M1 Silicon)上完美无缺。

我试图使用容器将其发布到Azure中。

问题:

使用Dockerfile构建应用程序时 - 容器化图像会产生错误:

selenium.common.exceptions.WebDriverException: Message: Service /root/.wdm/drivers/chromedriver/linux64/100.0.4896.60/chromedriver unexpectedly exited. Status code was: 255

Dockerfile:

这是我创建的Dockerfile:

FROM python:3.9-buster
# FROM --platform=linux/amd64  python:3.9

ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

RUN apt-get update \
    && apt-get -y install gcc make \
    && rm -rf /var/lib/apt/lists/*s

# RUN apt-get update
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
RUN apt-get install -y chromium
# RUN apt-get install -y chromium-browser

# install manually all the missing libraries
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils

# install chrome
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
# RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install

RUN dpkg -i google-chrome-stable_current_amd64.deb --fix-missing; apt-get -fy install


RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
RUN apt-get -y update

RUN apt-get update && apt-get install -y wget bzip2 libxtst6 packagekit-gtk3-module libx11-xcb-dev libdbus-glib-1-2 libxt6 libpci-dev && rm -rf /var/lib/apt/lists/*

#download and install chrome
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install

RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
RUN apt-get install -y chromium

RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main' | tee /etc/apt/sources.list.d/google-chrome.list
RUN apt update -y
RUN apt install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
# RUN apt install -y google-chrome-stable


RUN apt-get install -yqq unzip
RUN wget -O /tmp/chromedriver.zip http://chromedriver.storage.googleapis.com/`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE`/chromedriver_linux64.zip
RUN unzip /tmp/chromedriver.zip chromedriver -d /usr/local/bin/
RUN python3 --version
RUN pip3 --version
RUN pip install --no-cache-dir --upgrade pip


#install python dependencies
COPY requirements.txt requirements.txt
RUN pip install -r ./requirements.txt

#some envs
ENV APP_HOME /app
ENV PORT 5000

#set workspace
WORKDIR ${APP_HOME}

#copy local files
COPY . .

CMD exec gunicorn --bind :${PORT} --workers 1 --threads 8 main:app
# CMD exec gunicorn -b :$PORT main:app
# EXPOSE 8080
# CMD ["gunicorn", "--bind", "0.0.0.0:8080","--timeout", "90", "main:main"]
# CMD ["python3", "main.py"]
#build using:
# docker build -t python-webscraper .
# docker run --rm -p 3500:5000 python-webscraper
# docker run -p 3500:5000 python-webscraper

呼叫 CORMEDRIVER来自代码

chromedriver_autoinstaller.install()  # Check if the current version of chromedriver exists
    # and if it doesn't exist, download it automatically,
    # then add chromedriver to path
    chrome_options = Options()
    chrome_options.add_argument("--no-sandbox")
    # chrome_options = webdriver.ChromeOptions()
    chrome_options.add_argument("--headless")
    # chrome_options.add_argument("window-size=1400,2100")
    chrome_options.add_argument('--disable-gpu')
    chrome_options.add_argument("--disable-setuid-sandbox")
    chrome_options.add_argument('--disable-dev-shm-usage')  # Not used

如您所见,

,我使用的是无头选项,而没有沙盒选项可以传递到驱动程序中。 我如何构建容器

我感谢我使用MacOS,M1芯片,因此我尝试使用以下两种方法来构建:

docker buildx build --platform linux/amd64 -t app2 .

或者

docker build -t app2

我只是使用

 docker run -it  app2

摘要

  1. 来运行它我正在创建我的应用程序的本地容器化图像以在将其发布到Azure之前对其进行测试。
  2. Dockerfile为我创建了容器。
  3. 当我在本地测试(运行图像)时 - 启动Chrome驱动程序时,Web刮擦应用程序会失败。

我现在真的很绝望,并且尝试了很多搜索和重构,没有任何运气。

I have a Python Webscraping app that works flawlessly on my localhost (MacOS, M1 Silicon).

I am trying to publish it into Azure using a Container.

The problem:

When I build my app using a Dockerfile - the containerised image produces an error:

selenium.common.exceptions.WebDriverException: Message: Service /root/.wdm/drivers/chromedriver/linux64/100.0.4896.60/chromedriver unexpectedly exited. Status code was: 255

The Dockerfile:

Here is the Dockerfile that I've created:

FROM python:3.9-buster
# FROM --platform=linux/amd64  python:3.9

ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

RUN apt-get update \
    && apt-get -y install gcc make \
    && rm -rf /var/lib/apt/lists/*s

# RUN apt-get update
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
RUN apt-get install -y chromium
# RUN apt-get install -y chromium-browser

# install manually all the missing libraries
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils

# install chrome
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
# RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install

RUN dpkg -i google-chrome-stable_current_amd64.deb --fix-missing; apt-get -fy install


RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
RUN apt-get -y update

RUN apt-get update && apt-get install -y wget bzip2 libxtst6 packagekit-gtk3-module libx11-xcb-dev libdbus-glib-1-2 libxt6 libpci-dev && rm -rf /var/lib/apt/lists/*

#download and install chrome
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install

RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
RUN apt-get install -y chromium

RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main' | tee /etc/apt/sources.list.d/google-chrome.list
RUN apt update -y
RUN apt install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
# RUN apt install -y google-chrome-stable


RUN apt-get install -yqq unzip
RUN wget -O /tmp/chromedriver.zip http://chromedriver.storage.googleapis.com/`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE`/chromedriver_linux64.zip
RUN unzip /tmp/chromedriver.zip chromedriver -d /usr/local/bin/
RUN python3 --version
RUN pip3 --version
RUN pip install --no-cache-dir --upgrade pip


#install python dependencies
COPY requirements.txt requirements.txt
RUN pip install -r ./requirements.txt

#some envs
ENV APP_HOME /app
ENV PORT 5000

#set workspace
WORKDIR ${APP_HOME}

#copy local files
COPY . .

CMD exec gunicorn --bind :${PORT} --workers 1 --threads 8 main:app
# CMD exec gunicorn -b :$PORT main:app
# EXPOSE 8080
# CMD ["gunicorn", "--bind", "0.0.0.0:8080","--timeout", "90", "main:main"]
# CMD ["python3", "main.py"]
#build using:
# docker build -t python-webscraper .
# docker run --rm -p 3500:5000 python-webscraper
# docker run -p 3500:5000 python-webscraper

Calling ChromeDriver from Code

chromedriver_autoinstaller.install()  # Check if the current version of chromedriver exists
    # and if it doesn't exist, download it automatically,
    # then add chromedriver to path
    chrome_options = Options()
    chrome_options.add_argument("--no-sandbox")
    # chrome_options = webdriver.ChromeOptions()
    chrome_options.add_argument("--headless")
    # chrome_options.add_argument("window-size=1400,2100")
    chrome_options.add_argument('--disable-gpu')
    chrome_options.add_argument("--disable-setuid-sandbox")
    chrome_options.add_argument('--disable-dev-shm-usage')  # Not used

As you can see, I'm using the headless option and no sandbox options to pass into the driver.

How am I building the container

I appreciate Im using MacOs , M1 chip, so I've tried to build using the following 2 ways:

docker buildx build --platform linux/amd64 -t app2 .

or

docker build -t app2

Then I just run it using

 docker run -it  app2

Summary

  1. Im creating a local containerised image of my app to test it before publishing it to Azure.
  2. The Dockerfile creates the container for me.
  3. When I test it (run the image) locally - the web scraping app fails when launching Chrome Driver.

I'm now really desperate and have tried a lot of searching and refactoring without any luck.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

原谅过去的我 2025-01-30 14:56:25

我也有同样的问题。问题在于码头虚拟环境中。添加“ options.add_argument(' - disable-dev-shm-usage')”参数对我有帮助。必须安装镀铬。

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service


service = Service(executable_path=ChromeDriverManager().install())
options = Options()
options.add_argument('--disable-blink-features=AutomationControlled') ## to avoid getting detected
options.add_argument('headless')
options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome(service=service, options=options)"

I had the same issue. The problem was in the docker virtual environment. Adding the "options.add_argument('--disable-dev-shm-usage')" argument helped me. Chrome must be installed.

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service


service = Service(executable_path=ChromeDriverManager().install())
options = Options()
options.add_argument('--disable-blink-features=AutomationControlled') ## to avoid getting detected
options.add_argument('headless')
options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome(service=service, options=options)"
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文