You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Not able to import pyarrow in aws lambda function. #39465
I am trying to create a lambda function in aws, which requires pyarrow as a dependency. i have installed pyarrow along with other dependency and created a zip folder in my local machine. after that i have created layer on aws with that zip folder. but when i am trying to run the lambda function, it is giving me this below error.
{
"errorMessage": "Unable to import module 'lambda_function': No module named 'pyarrow.lib'",
"errorType": "Runtime.ImportModuleError",
"requestId": "4eb1a49d-751f-4e11-ad1e-2fd75319ccfb",
"stackTrace": []
}
Import statements in my lambda function looks like this,
import boto3
import pyarrow as pa
import pyarrow.parquet as pq
import redis
can anyone help me on this?? although i am able to import other dependencies like redis but not pyarrow.
No module named 'pyarrow.lib' means the compiled part of pyarrow in the layer could not be loaded, which usually happens when the zip was built for a different OS, architecture or Python version than the Lambda runtime. I get the same error locally by importing a pyarrow wheel built for Python 3.11 from Python 3.12. Building the layer for the same platform and Python version as the function, or using the AWS SDK for pandas layer mentioned above, should resolve it.
As this is a packaging question rather than a bug in Arrow and there has been no follow-up, I'm closing this; the user mailing list or GitHub Discussions are the best place for further help.
Activity
WillAyd commentedon Jan 9, 2025
Are you uploading the PyArrow source or a pre-compiled version to lambda?
FWIW AWS distributes a Lambda Layer that already has pyarrow in it. Any reason why you don't want to use that?
https://aws-sdk-pandas.readthedocs.io/en/stable/layers.html
dixithak commentedon Jun 7, 2025
@JainamRaval were you able to resolve this? I tried this via cdk and docker, but didn't work
thisisnic commentedon Nov 20, 2026
No module named 'pyarrow.lib'means the compiled part of pyarrow in the layer could not be loaded, which usually happens when the zip was built for a different OS, architecture or Python version than the Lambda runtime. I get the same error locally by importing a pyarrow wheel built for Python 3.11 from Python 3.12. Building the layer for the same platform and Python version as the function, or using the AWS SDK for pandas layer mentioned above, should resolve it.As this is a packaging question rather than a bug in Arrow and there has been no follow-up, I'm closing this; the user mailing list or GitHub Discussions are the best place for further help.