Facing an Error while using Custom Docker file

[faisalriazgoraya]

Docker file

SeleniumBase Docker Image

FROM ubuntu:18.04

#=======================================

Install Python and Basic Python Tools

#=======================================
RUN apt-get -o Acquire::Check-Valid-Until=false -o Acquire::Check-Date=false update
RUN apt-get install -y python3 python3-pip python3-setuptools python3-dev python-distribute
RUN alias python=python3
RUN echo "alias python=python3" >> ~/.bashrc

#=================================

Install Bash Command Line Tools

#=================================
RUN apt-get -qy --no-install-recommends install
sudo
unzip
wget
curl
libxi6
libgconf-2-4
vim
xvfb
&& rm -rf /var/lib/apt/lists/*

#================

Install Chrome

#================
RUN curl -sS -o - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - &&
echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list &&
apt-get -yqq update &&
apt-get -yqq install google-chrome-stable &&
rm -rf /var/lib/apt/lists/*

#===========================

Configure Virtual Display

#===========================
RUN set -e
RUN echo "Starting X virtual framebuffer (Xvfb) in background..."
RUN Xvfb -ac :99 -screen 0 1280x1024x16 > /dev/null 2>&1 &
RUN export DISPLAY=:99
RUN exec "$@"

#=======================

Update Python Version

#=======================
RUN apt-get update -y
RUN apt-get -qy --no-install-recommends install python3.8
RUN rm /usr/bin/python3
RUN ln -s python3.8 /usr/bin/python3

#=============================================

Allow Special Characters in Python Programs

#=============================================
RUN export PYTHONIOENCODING=utf8
RUN echo "export PYTHONIOENCODING=utf8" >> ~/.bashrc

#=====================

Download WebDrivers

#=====================
RUN wget https://github.com/mozilla/geckodriver/releases/download/v0.33.0/geckodriver-v0.33.0-linux64.tar.gz
RUN tar -xvzf geckodriver-v0.33.0-linux64.tar.gz
RUN chmod +x geckodriver
RUN mv geckodriver /usr/local/bin/
RUN wget https://chromedriver.storage.googleapis.com/72.0.3626.69/chromedriver_linux64.zip
RUN unzip chromedriver_linux64.zip
RUN chmod +x chromedriver
RUN mv chromedriver /usr/local/bin/

# Set the working directory inside the container

WORKDIR /app

Install Python dependencies

COPY requirements.txt .
RUN pip3 install -r requirements.txt

Copy your SeleniumBase scripts or application code

COPY main2.py ./

Set the entry point (replace 'main2.py' with your actual script)

CMD [ "python3", "main2.py" ]

Requirements File
requests~=2.31.0
beautifulsoup4~=4.12.2
seleniumbase~=4.22.0
selenium~=4.16.0

Python main2.py code
import time
from seleniumbase import Driver

proxy = "uqsqhklf:[email protected]:5868"
user_agent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) '
'Chrome/96.0.4664.45 Safari/537.36 '
driver = Driver(uc=True, agent=user_agent, headless=True)
driver.open("https://www.ibm.com/topics/natural-language-processing")
driver.sleep(30)
driver.save_screenshot('/tmp/screenshots/screenshot_{}.png'.format(int(time.time())))
driver.quit()

Error that i am getting

INIT_REPORT Init Duration: 10134.04 ms Phase: init Status: timeout

Warning: uc_driver not found. Getting it now:
*** chromedriver to download = 120.0.6099.109 (Latest Stable)
Downloading chromedriver-linux64.zip from:
https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/120.0.6099.109/linux64/chromedriver-linux64.zip ...
Cache folder (/home/sbx_user1051/.cache/selenium) cannot be created: Read-only file system (os error 30)
The chromedriver version (72.0.3626.69) detected in PATH at /usr/local/bin/chromedriver might not be compatible with the detected chrome version (120.0.6099.129); currently, chromedriver 120.0.6099.109 is recommended for chrome 120., so it is advised to delete the driver in PATH and retry
Traceback (most recent call last):
File "main2.py", line 7, in
driver = Driver(uc=True, agent=user_agent, headless=True)
File "/usr/local/lib/python3.8/dist-packages/seleniumbase/plugins/driver_manager.py", line 463, in Driver
driver = browser_launcher.get_driver(
File "/usr/local/lib/python3.8/dist-packages/seleniumbase/core/browser_launcher.py", line 1562, in get_driver
return get_local_driver(
File "/usr/local/lib/python3.8/dist-packages/seleniumbase/core/browser_launcher.py", line 3210, in get_local_driver
with uc_lock: # Avoid multithreaded issues
File "/usr/local/lib/python3.8/dist-packages/fasteners/process_lock.py", line 166, in enter
gotten = self.acquire()
File "/usr/local/lib/python3.8/dist-packages/fasteners/process_lock.py", line 144, in acquire
self._do_open()
File "/usr/local/lib/python3.8/dist-packages/fasteners/process_lock.py", line 106, in _do_open
made_basedir = _ensure_tree(basedir)
File "/usr/local/lib/python3.8/dist-packages/fasteners/process_lock.py", line 43, in _ensure_tree
os.makedirs(path)
File "/usr/lib/python3.8/os.py", line 221, in makedirs
mkdir(name, mode)
OSError: [Errno 30] Read-only file system: b'downloaded_files'
INIT_REPORT Init Duration: 22894.84 ms Phase: invoke Status: error Error Type: Runtime.ExitError
START RequestId: 6a36a7a0-c2d9-4341-8f8f-9f1d1ba3d50f Version: $LATEST
RequestId: 6a36a7a0-c2d9-4341-8f8f-9f1d1ba3d50f Error: Runtime exited with error: exit status 1Runtime.ExitError
END RequestId: 6a36a7a0-c2d9-4341-8f8f-9f1d1ba3d50f
REPORT RequestId: 6a36a7a0-c2d9-4341-8f8f-9f1d1ba3d50f Duration: 22896.00 ms Billed Duration: 22897 ms Memory Size: 2048 MB Max Memory Used: 109 MB
Warning: uc_driver not found. Getting it now:
*** chromedriver to download = 120.0.6099.109 (Latest Stable)
Downloading chromedriver-linux64.zip from:
https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/120.0.6099.109/linux64/chromedriver-linux64.zip ...
Cache folder (/home/sbx_user1051/.cache/selenium) cannot be created: Read-only file system (os error 30)
The chromedriver version (72.0.3626.69) detected in PATH at /usr/local/bin/chromedriver might not be compatible with the detected chrome version (120.0.6099.129); currently, chromedriver 120.0.6099.109 is recommended for chrome 120., so it is advised to delete the driver in PATH and retry
Traceback (most recent call last):
File "main2.py", line 7, in
driver = Driver(uc=True, agent=user_agent, headless=True)
File "/usr/local/lib/python3.8/dist-packages/seleniumbase/plugins/driver_manager.py", line 463, in Driver
driver = browser_launcher.get_driver(
File "/usr/local/lib/python3.8/dist-packages/seleniumbase/core/browser_launcher.py", line 1562, in get_driver
return get_local_driver(
File "/usr/local/lib/python3.8/dist-packages/seleniumbase/core/browser_launcher.py", line 3210, in get_local_driver
with uc_lock: # Avoid multithreaded issues
File "/usr/local/lib/python3.8/dist-packages/fasteners/process_lock.py", line 166, in enter
gotten = self.acquire()
File "/usr/local/lib/python3.8/dist-packages/fasteners/process_lock.py", line 144, in acquire
self._do_open()
File "/usr/local/lib/python3.8/dist-packages/fasteners/process_lock.py", line 106, in _do_open
made_basedir = _ensure_tree(basedir)
File "/usr/local/lib/python3.8/dist-packages/fasteners/process_lock.py", line 43, in _ensure_tree
os.makedirs(path)
File "/usr/lib/python3.8/os.py", line 221, in makedirs
mkdir(name, mode)
OSError: [Errno 30] Read-only file system: b'downloaded_files'
INIT_REPORT Init Duration: 1027.74 ms Phase: init Status: error Error Type: Runtime.ExitError

[mdmintz]

Based on you using an AWS Lambda Docker environment, (#2378), it is limiting you: OSError: [Errno 30] Read-only file system. There are related Stack Overflow questions about it here: https://stackoverflow.com/q/53810516/7058266 and https://stackoverflow.com/q/39383465/7058266. According to the second one, Only /tmp seems to be writable in AWS Lambda. (https://stackoverflow.com/a/42002539/7058266). Due to that limitation, you will need to make some modifications to your Dockerfile to accommodate. Some more info here: SeleniumHQ/selenium#12774 (comment)

You could try asking on Stack Overflow if you have trouble making modifications to your Dockerfile, as there are a lot of qualified AWS Lambda experts on there.

And don't open duplicate tickets for the same problem, as you did here:

• Docker file for AWS deployement #2377
• Docker file for AWS Python Deployement #2378