How to use Selenium on AWS lambda Python3 7
Why Stack Overflow recommends it
Stack Overflow users link it when answering how to get Selenium and chromedriver working on AWS Lambda, especially for layer packaging and pinned Python 3.7 version combinations.
Recommended by 2 people across 2 different months, 2021 to 2024.
What it is
A short screen-recorded tutorial by Soumil Shah showing how to run Selenium with headless Chromium inside an AWS Lambda function on the Python 3.7 runtime. It walks through packaging Selenium, chromedriver and headless-chromium as a Lambda layer, with the code on GitHub.
What people said
Recommends Ramón · 2021-08 · Stack Overflow · on “Debugging Selenium and Chromedriver on AWS Lambda (chrome not reachable)”
I found this to be really helpful: [this video] [LINK] The versions are the following: RUNTIME=python3.7 SELENIUM_VER=3.141.0 CHROME_BINARY_VER=v1.0.0-55 # based on Chromium 69.0.3497.81 CHROMEDRIVER_VER=2.43 # supports Chrome v69-71 Credits go to Soumil Nitin Shah. Best, Ramón
Recommends Utku Can · 2022-04 · Stack Overflow · on “How do I add selenium & chromedriver to an AWS Lambda function?”
I've been struggling adding selenium to the aws lambda for last couple days. I have a web scraping function (uses selenium and google api) which extracts data from a website and writes the outputs to a google spreadsheet. Let me explain what i did step by step and how i finally succeeded so you don't have to deal with it as much as me: 1- I tried to add selenium as a layer described here [this video] . What i ended up was, i was succesfull with adding selenium but deployment package is over 250mb (describe lambda quotaas here: How to increase the maximum size of the AWS lambda deployment package (RequestEntityTooLargeException)? [LINK] ) so it did not work. 2- To overcome deployment package size,
Cites mailo · 2021-09 · Stack Overflow · on “How to solve NameError:Options is not defined in AWS Lambda?”
Just had a similar issue so perhaps it can help you: Check if the selenium, chromdriver $ headless-chromium are zipped properly. so when you upload it as a layer to aws lambda, it has the right pathname for you to import selenium in your lambda code. you can download the file chrome_headless.zip from: [LINK] and then zip it again inside that folder, on your local machine: zip -r chrome_headless.zip chromedriver headless-chromium python then try to upload it to aws lambda layers, attach it to your function and test it. all credit goes to Soumil Nitin Shah with his great tutorial: [this video]
Cites Shubham · 2024-04 · Stack Overflow · on “How to run Selenium on AWS Lambda with Python versions greater than 3.7”
I'm trying to use Selenium with AWS Lambda for web scraping tasks using Python. However, I'm encountering difficulties when trying to run Selenium on Lambda with Python versions greater than 3.7. During my research, I found that many tutorials and resources only cover Python 3.7 or earlier versions, and the solutions provided are not compatible with newer Python versions. Can anyone provide guidance on how to properly set up and run Selenium on AWS Lambda using Python versions 3.8 or 3.9? I've tried various approaches, including using custom layers with pre-installed Selenium and WebDriver like ch, where I zip them and create two file namely python and chrome driver but I keep encountering i