Getting PIL/Pillow 4.2.1 to upload properly to AWS

问题:

Background

I have been struggling for the past few days to deploy a Lambda that uses Pillow, and I am deploying using Python 3.6. It may be noteworthy also that I am developing this on a Windows 10 environment.

First Attempts

I began by having pip install my packages strictly in my workspace by doing the following:

pip3 install pillow -t "D:\Work and Projects\...\...\<projectdir>\pillow"

I have other packages, and tried installing the packages in the same manor, one of them specifically was praw and I did so by:

pip3 install praw -t "D:\Work and Projects\...\...\<projectdir>\praw"

After zipping the contents of my project together, I uploaded my package up to Lambda and upon my first test I received the error:

Unable to import module 'motw_lambda': cannot import name '_imaging'

I then removed the Pillow package in an attempt to see where this issue was stemming from (Pillow or praw or one of the other packages). With Pillow removed, the execution succeeded. I then removed the pillow package in my package and tried:

pip3 install pillow -t "D:\Work and Projects\...\...\<projectdir>\PIL"

and

pip3 install pillow -t "D:\Work and Projects\...\...\<projectdir>\Pillow"

But got the same error with the package '_imaging'.

Further Attempts

I then followed the directions per this resource and this. I also tried using virualenv and even lambda-uploader.

Strange enough, I get the same error! I am all out of options here, and feel either I am doing something dumb, or that this is not possible on Lambda-Python3.6 currently (Although I can't image someone else hasn't used pillow in a py3.6-lambda yet...)

Any info, help, or generic resources would be appreciated!

回答1:

Basically, you have to compile the libraries (eg, PIL) either using Docker or, even better, an EC2 instance.

1. Launch an Docker container like this: docker run --rm -it -v "%cd%:/code" lambci/lambda:build-python3.6 sh

2. Inside there, navigate to the /code dir (cd /code), create a virtualenv (virtualenv env), activate it (source env/bin/activate) and finally, install your library (pip install pillow).

3. Once you have installed your library, you can exit the container. The secret here is to move your package library to the root folder (where your main .py file is). For example, move the folder env/lib/python3.6/site-packages/PIL to the root.

Then, zip your PIL folder together with your .py file and you're set!

I just did it and documented it in full here: https://learn-serverless.org/post/deploying-pillow-aws-lambda/

回答2:

You can use a precompiled version of the PIL available here: https://github.com/Miserlou/lambda-packages

Just extract PIL folder to the deployment package and it should work.

回答3:

For anyone else also new to aws python and running into this issue, you can use the layers feature, and there are existing layers here you can link to and this worked for me.

https://github.com/keithrozario/Klayers

Pillow specifically on us-east-1:

arn:aws:lambda:us-east-1:770693421928:layer:Klayers-python38-Pillow:2