Can you have an async handler in Lambda Python 3.6?
Feb 28
I've made Lambda functions before but not in Python. I know in Javascript Lambda supports the handler function being asynchronous, but I get an error if I try it in Python.
Here is the code I am trying to test:
async def handler(event, context):
print(str(event))
return {
'message' : 'OK'
}
And this is the error I get:
An error occurred during JSON serialization of response: <coroutine object handler at 0x7f63a2d20308> is not JSON serializable
Traceback (most recent call last):
File "/var/lang/lib/python3.6/json/__init__.py", line 238, in dumps
**kw).encode(obj)
File "/var/lang/lib/python3.6/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/var/lang/lib/python3.6/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/var/runtime/awslambda/bootstrap.py", line 149, in decimal_serializer
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <coroutine object handler at 0x7f63a2d20308> is not JSON serializable
/var/runtime/awslambda/bootstrap.py:312: RuntimeWarning: coroutine 'handler' was never awaited
errortype, result, fatal = report_fault(invokeid, e)
EDIT 2021:
Since this question seems to be gaining traction, I assume people are coming here trying to figure out how to get async to work with AWS Lambda as I was. The bad news is that even now more than a year later, there still isn't any support by AWS to have an asynchronous handler in a Python-based Lambda function. (I have no idea why, as NodeJS-based Lambda functions can handle it perfectly fine.)
The good news is that since Python 3.7, there is a simple workaround in the form of asyncio.run:
import asyncio
def lambda_handler(event, context):
# Use asyncio.run to synchronously "await" an async function
result = asyncio.run(async_handler(event, context))
return {
'statusCode': 200,
'body': result
}
async def async_handler(event, context):
# Put your asynchronous code here
await asyncio.sleep(1)
return 'Success'
Note: The selected answer says that using asyncio.run is not the proper way of starting an asynchronous task in Lambda. In general, they are correct because if some other resource in your Lambda code creates an event loop (a database/HTTP client, etc.), it's wasteful to create another loop and it's better to operate on the existing loop using asyncio.get_event_loop.
However, if an event loop does not yet exist when your code begins running, asyncio.run becomes the only (simple) course of action.
python1 answer
Accepted answer · original discussion
May 26
Not at all. Async Python handlers are not supported by AWS Lambda.
If you need to use async/await functionality in your AWS Lambda, you have to define an async function in your code (either in Lambda files or a Lambda Layer) and call asyncio.get_event_loop().run_until_complete(your_async_handler()) inside your regular sync Lambda handler:
import asyncio
import aioboto3
# To reduce execution time for subsequent invocations,
# open a reusable resource in a global scope
dynamodb = aioboto3.Session().resource('dynamodb')
async def async_handler(event, context):
# Put your asynchronous code here
table = await dynamodb.Table('test')
await table.put_item(
Item={'pk': 'test1', 'col1': 'some_data'},
)
return {'statusCode': 200, 'body': '{"ok": true}'}
# Point to this function as a handler in the Lambda configuration
def lambda_handler(event, context):
loop = asyncio.get_event_loop()
# DynamoDB resource defined above is attached to this loop:
# if you use asyncio.run instead
# you will encounter "Event loop closed" exception
return loop.run_until_complete(async_handler(event, context))
Please note that asyncio.run (introduced in Python 3.7) is not a proper way to call an async handler in AWS Lambda execution environment since Lambda tries to reuse the execution context for subsequent invocations. The problem here is that asyncio.run creates a new EventLoop and closes the previous one. If you have opened any resources or created coroutines attached to the closed EventLoop from previous Lambda invocation you will get «Event loop closed» error. asyncio.get_event_loop().run_until_complete allows you to reuse the same loop. See related StackOverflow question.
AWS Lambda documentation misleads its readers a little by introducing synchronous and asynchronous invocations. Do not mix it up with sync/async Python functions. Synchronous refers to invoking AWS Lambda with further waiting for the result (blocking operation). The function is called immediately and you get the response as soon as possible. Whereas using an asynchronous invocation you ask Lambda to schedule the function execution and do not wait for the response at all. When the time comes, Lambda still will call the handler function synchronously.
3 question comments
Use comments to ask for clarification. Post a solution as an answer.
Sep 14
asyncio.get_event_loop(), it threw an error since there was no existing event loop to get. So you would manually have to create the event loop as well as close it, which was a real pain to manage, which is why I used asyncio.run as it handled creating and closing a loop for you. I ran it several times on every supported Python runtime 3.7+ and never got an Event loop closed exception.Sep 20