selenium.common.exceptions.webdriverexception:消息:Chromedriver意外退出。状态代码是:255使用Dockerfile
我有一个Python Webscraping应用程序,可以在我的Localhost(MacOS,M1 Silicon)上完美无缺。
我试图使用容器将其发布到Azure中。
问题:
使用Dockerfile构建应用程序时 - 容器化图像会产生错误:
selenium.common.exceptions.WebDriverException: Message: Service /root/.wdm/drivers/chromedriver/linux64/100.0.4896.60/chromedriver unexpectedly exited. Status code was: 255
Dockerfile:
这是我创建的Dockerfile:
FROM python:3.9-buster
# FROM --platform=linux/amd64 python:3.9
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
RUN apt-get update \
&& apt-get -y install gcc make \
&& rm -rf /var/lib/apt/lists/*s
# RUN apt-get update
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
RUN apt-get install -y chromium
# RUN apt-get install -y chromium-browser
# install manually all the missing libraries
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
# install chrome
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
# RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install
RUN dpkg -i google-chrome-stable_current_amd64.deb --fix-missing; apt-get -fy install
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
RUN apt-get -y update
RUN apt-get update && apt-get install -y wget bzip2 libxtst6 packagekit-gtk3-module libx11-xcb-dev libdbus-glib-1-2 libxt6 libpci-dev && rm -rf /var/lib/apt/lists/*
#download and install chrome
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
RUN apt-get install -y chromium
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main' | tee /etc/apt/sources.list.d/google-chrome.list
RUN apt update -y
RUN apt install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
# RUN apt install -y google-chrome-stable
RUN apt-get install -yqq unzip
RUN wget -O /tmp/chromedriver.zip http://chromedriver.storage.googleapis.com/`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE`/chromedriver_linux64.zip
RUN unzip /tmp/chromedriver.zip chromedriver -d /usr/local/bin/
RUN python3 --version
RUN pip3 --version
RUN pip install --no-cache-dir --upgrade pip
#install python dependencies
COPY requirements.txt requirements.txt
RUN pip install -r ./requirements.txt
#some envs
ENV APP_HOME /app
ENV PORT 5000
#set workspace
WORKDIR ${APP_HOME}
#copy local files
COPY . .
CMD exec gunicorn --bind :${PORT} --workers 1 --threads 8 main:app
# CMD exec gunicorn -b :$PORT main:app
# EXPOSE 8080
# CMD ["gunicorn", "--bind", "0.0.0.0:8080","--timeout", "90", "main:main"]
# CMD ["python3", "main.py"]
#build using:
# docker build -t python-webscraper .
# docker run --rm -p 3500:5000 python-webscraper
# docker run -p 3500:5000 python-webscraper
呼叫 CORMEDRIVER来自代码
chromedriver_autoinstaller.install() # Check if the current version of chromedriver exists
# and if it doesn't exist, download it automatically,
# then add chromedriver to path
chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
# chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
# chrome_options.add_argument("window-size=1400,2100")
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument("--disable-setuid-sandbox")
chrome_options.add_argument('--disable-dev-shm-usage') # Not used
如您所见,
,我使用的是无头选项,而没有沙盒选项可以传递到驱动程序中。 我如何构建容器
我感谢我使用MacOS,M1芯片,因此我尝试使用以下两种方法来构建:
docker buildx build --platform linux/amd64 -t app2 .
或者
docker build -t app2
我只是使用
docker run -it app2
摘要
- 来运行它我正在创建我的应用程序的本地容器化图像以在将其发布到Azure之前对其进行测试。
- Dockerfile为我创建了容器。
- 当我在本地测试(运行图像)时 - 启动Chrome驱动程序时,Web刮擦应用程序会失败。
我现在真的很绝望,并且尝试了很多搜索和重构,没有任何运气。
I have a Python Webscraping app that works flawlessly on my localhost (MacOS, M1 Silicon).
I am trying to publish it into Azure using a Container.
The problem:
When I build my app using a Dockerfile - the containerised image produces an error:
selenium.common.exceptions.WebDriverException: Message: Service /root/.wdm/drivers/chromedriver/linux64/100.0.4896.60/chromedriver unexpectedly exited. Status code was: 255
The Dockerfile:
Here is the Dockerfile that I've created:
FROM python:3.9-buster
# FROM --platform=linux/amd64 python:3.9
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
RUN apt-get update \
&& apt-get -y install gcc make \
&& rm -rf /var/lib/apt/lists/*s
# RUN apt-get update
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
RUN apt-get install -y chromium
# RUN apt-get install -y chromium-browser
# install manually all the missing libraries
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
# install chrome
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
# RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install
RUN dpkg -i google-chrome-stable_current_amd64.deb --fix-missing; apt-get -fy install
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
RUN apt-get -y update
RUN apt-get update && apt-get install -y wget bzip2 libxtst6 packagekit-gtk3-module libx11-xcb-dev libdbus-glib-1-2 libxt6 libpci-dev && rm -rf /var/lib/apt/lists/*
#download and install chrome
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
RUN apt-get install -y chromium
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main' | tee /etc/apt/sources.list.d/google-chrome.list
RUN apt update -y
RUN apt install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
# RUN apt install -y google-chrome-stable
RUN apt-get install -yqq unzip
RUN wget -O /tmp/chromedriver.zip http://chromedriver.storage.googleapis.com/`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE`/chromedriver_linux64.zip
RUN unzip /tmp/chromedriver.zip chromedriver -d /usr/local/bin/
RUN python3 --version
RUN pip3 --version
RUN pip install --no-cache-dir --upgrade pip
#install python dependencies
COPY requirements.txt requirements.txt
RUN pip install -r ./requirements.txt
#some envs
ENV APP_HOME /app
ENV PORT 5000
#set workspace
WORKDIR ${APP_HOME}
#copy local files
COPY . .
CMD exec gunicorn --bind :${PORT} --workers 1 --threads 8 main:app
# CMD exec gunicorn -b :$PORT main:app
# EXPOSE 8080
# CMD ["gunicorn", "--bind", "0.0.0.0:8080","--timeout", "90", "main:main"]
# CMD ["python3", "main.py"]
#build using:
# docker build -t python-webscraper .
# docker run --rm -p 3500:5000 python-webscraper
# docker run -p 3500:5000 python-webscraper
Calling ChromeDriver from Code
chromedriver_autoinstaller.install() # Check if the current version of chromedriver exists
# and if it doesn't exist, download it automatically,
# then add chromedriver to path
chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
# chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
# chrome_options.add_argument("window-size=1400,2100")
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument("--disable-setuid-sandbox")
chrome_options.add_argument('--disable-dev-shm-usage') # Not used
As you can see, I'm using the headless option and no sandbox options to pass into the driver.
How am I building the container
I appreciate Im using MacOs , M1 chip, so I've tried to build using the following 2 ways:
docker buildx build --platform linux/amd64 -t app2 .
or
docker build -t app2
Then I just run it using
docker run -it app2
Summary
- Im creating a local containerised image of my app to test it before publishing it to Azure.
- The Dockerfile creates the container for me.
- When I test it (run the image) locally - the web scraping app fails when launching Chrome Driver.
I'm now really desperate and have tried a lot of searching and refactoring without any luck.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
我也有同样的问题。问题在于码头虚拟环境中。添加“ options.add_argument(' - disable-dev-shm-usage')”参数对我有帮助。必须安装镀铬。
I had the same issue. The problem was in the docker virtual environment. Adding the "options.add_argument('--disable-dev-shm-usage')" argument helped me. Chrome must be installed.