"cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'" on AWS Lambda using a layer
Jun 6
To scrape an website using AWS Lambda and save the data on S3.
The issues I'm havingWhen I execute Lambda, the following error message appears.
Code{ "errorMessage": "Unable to import module 'lambda_function': cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_' (/opt/python/urllib3/util/ssl_.py)", "errorType": "Runtime.ImportModuleError", "requestId": "fb66bea9-cbad-4bd3-bd4d-6125454e21be", "stackTrace": [] }
The minimum Lambda code is as follows.
import requests
import boto3
def lambda_handler(event, context):
s3 = boto3.client('s3')
upload_res = s3.put_object(Bucket='horserace-dx', Key='/raw/a.html', Body='testtext')
return event
An layer was added to the Lambda. Files were save in python folder using the commands below , frozen in a zip file, then uploaded to AWS Lambda as a layer.
!pip install requests -t ./python --no-user
!pip install pandas -t ./python --no-user
!pip install beautifulsoup4 -t ./python --no-user
- The bucket
horserace-dxexists - The folder
rawexists - The role of the Lambda is properly set. It can read from and write to S3
- The runtime of the Lambda is Python 3.9. The python version of the local computer is 3.9.13.
I google "cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'" and found some suggestions. I made the layer with the following code and tried again in vain.
!pip install requests -t ./python --no-user
!pip install pandas -t ./python --no-user
!pip install beautifulsoup4 -t ./python --no-user
!pip install urllib3==1.26.15 -t ./python --no-user
So what should I do to achieve what I want to achieve? Any suggestions would be greatly appreciated.
1 answer
Accepted answer · original discussion
Jun 7
Execute the following commands.
pip install requests==2.25.0 -t ./python --no-user pip install beautifulsoup4 -t ./python --no-user pip install pytz -t ./python --no-user
On PyPI, download the following whl files from the pages of numpy and pandas
- numpy-1.24.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- pandas-2.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Unzip the files and move the contents to the
pythonfolder.Zip the
pythonfolder and upload it to AWS Lambda Layer.Set the layer to the Lambda.
Then the code runs without errors.
1 question comment
Use comments to ask for clarification. Post a solution as an answer.
Jun 6