python+docker_通过Docker创建Lambda层:Python +层+ Docker

东方建修
2023-12-01

python+docker

序幕 (Prologue)

If you read Deploying to AWS Lambda: Python + Layers + CloudWatch, we created our Lambda Layer zip on a RHEL8 EC2 instance. Note that we did this because we needed to install the packages on a Linux OS, not Mac OS which I was on. If you are on Linux, you probably don’t need to use Docker or an EC2 instance and can just install and zip locally. For the remainder of this post, we’ll assume that you are not on a Linux OS. So, if you don’t want to launch an EC2 instance, another way to create the zip is via Docker.

如果您阅读Deploying to AWS Lambda:Python + Layers + CloudWatch ,我们在RHEL8 EC2实例上创建了Lambda Layer zip。 请注意,我们这样做是因为我们需要在Linux操作系统(而不是我所在的Mac OS)上安装软件包。 如果您使用的是Linux,则可能不需要使用Docker或EC2实例,而只需在本地安装和压缩即可。 在本文的其余部分中,我们假设您不在Linux操作系统上。 因此,如果您不想启动EC2实例,则创建zip的另一种方法是通过Docker。

Some fundamental knowledge of Docker, Bash, and Python is a prerequisite for this post, but perhaps you may still be able to follow along.

对Docker,Bash和Python的一些基本了解是这篇文章的先决条件,但也许您仍然可以继续学习。

步骤0:安装Docker (Step 0: Install Docker)

I will be proceeding with this tutorial on Mac OS but I think even if you are not on Mac OS, this post should still point you in the right direction.

我将在Mac OS上进行本教程的学习,但我认为即使您不在Mac OS上,本帖子也应为您指明正确的方向。

Go ahead and install Docker on your machine.

继续并在您的机器上安装Docker

步骤1:建立Dockerfile (Step 1: Create Dockerfile)

Once you’re able to verify that you have it properly installed, pick any directory on your local machine to create our package. For example, on my Macbook, I’m going to be creating the layer in /Users/Nakul/Documents/Docker. Now in this directory, we’re going to create a Dockerfile — to do so, simply create a file called Dockerfile with vi Dockerfile.

一旦能够确认已正确安装它,请在本地计算机上选择任何目录以创建我们的软件包。 例如,在Macbook上,我将在/Users/Nakul/Documents/Docker创建该层。 现在在此目录中,我们将创建一个Dockerfile -为此,只需使用vi Dockerfile创建一个名为Dockerfile的文件。

In this Dockerfile, we’re going to input the following:

在此Dockerfile中,我们将输入以下内容:

FROM amazonlinux:2.0.20200602.0RUN yum update -yRUN yum install -y \python3-pip \zip \RUN yum -y clean allRUN python3.7 -m pip install — upgrade pip

Save the file.

保存文件。

We don’t want or need to do too much in this Dockerfile. The FROM command simply specifies which image we want to download from Docker hub, we just want any Linux distro so I went with Amazon Linux (you can also pick Debian, etc). The tag you see was pulled from the official Amazon Linux page on Docker hub.

我们不想或不需要在此Dockerfile中做太多事情。 FROM命令仅指定我们要从Docker集线器下载的映像,我们只需要任何Linux发行版,因此我选择了Amazon Linux(您也可以选择Debian等)。 您看到的标签是从Docker Hub上官方Amazon Linux页面提取的

Now, once we’ve done that, we want to make sure we can install our Python libraries once we build and run this image, so we need to RUN a few commands. The commands you see above would be similar to what you might run on an EC2 Instance.

现在,一旦完成,我们要确保在构建和运行该映像后就可以安装Python库,因此我们需要运行一些命令。 您在上面看到的命令将类似于您在EC2实例上运行的命令。

See the references at the end of this post to learn about YUM and YUM commands.

请参阅本文结尾处的参考资料,以了解YUM和YUM命令。

第2步:构建并运行Docker映像 (Step 2: Build and Run Docker Image)

Now in that directory, type

现在在该目录中,键入

docker build -t amzlinuxpy37 .

And press Enter. This will build the image from the Dockerfile we just wrote (and will also tag it as amzlinuxpy37).

然后按Enter。 这将从我们刚刚编写的Dockerfile构建映像(并将其标记为amzlinuxpy37)。

Now, to run the Docker image, in the same directory (/Docker), type

现在,要运行Docker映像,请在同一目录( /Docker )中键入

docker run -it amzlinuxpy37 bash

What we’re doing here is running a container based on the image we built via the Dockerfile in the directory. The -it is to flag that we want to immediately open up an interactive shell — so you’ll see the command prompt say bash-4.2# or something like that.

我们在这里所做的是基于通过目录中的Dockerfile构建的映像运行一个容器。 -it表示我们要立即打开一个交互式shell,因此您将在命令提示符下看到bash-4.2#或类似名称。

You can run an ls to see what directories we have but we won’t be bothering with any of them. I made a directory called layer_dir in the home (/) directory of the container. I’d recommend you do the same so you’re not distracted by all the sub-directories in the top level directory.

您可以运行ls来查看我们拥有的目录,但是我们不会打扰任何一个。 我在容器的主目录( / )中layer_dir了一个名为layer_dir的目录。 我建议您执行相同的操作,以免您被顶级目录中的所有子目录分散注意力。

Next, simply enter that directory with a cd layer_dir command.

接下来,只需使用cd layer_dir命令输入该目录。

步骤3:安装Python模块 (Step 3: Install Python Modules)

Now, in here is where we’ll build the package with all our modules. Since we won’t be using this container for anything else besides creating our Lambda layer, it’s not mandatory to use a virtual environment (but note that in most cases, it’s a best practice to do so to keep your project dependencies isolated and intact).

现在,在这里,我们将使用所有模块来构建软件包。 由于除了创建Lambda层之外,我们不会将该容器用于其他任何用途,因此使用虚拟环境不是强制性的(但请注意,在大多数情况下,这样做是使项目依赖项保持隔离和完整的最佳做法) 。

With that in mind, we should pause and remember why we are in this container. We want to install Python modules and zip them up so we can upload the zip as a Lambda Layer.

考虑到这一点,我们应该暂停并记住为什么要进入此容器。 我们要安装Python模块并压缩它们,以便我们可以将zip上传为Lambda层。

If you read my Lambda Layers post or perused AWS documentation, you’ll note that in order for AWS to comprehend our package, the modules must either be installed in python/*module installed here* or python/lib/python3.x/site-packages/*module installed here*.

如果您阅读我的Lambda Layers帖子细读过的AWS文档 ,您会注意到,为了使AWS理解我们的软件包,必须将模块安装在python/*module installed here*python/lib/python3.x/site-packages/*module installed here*

The quickest (but not only) way here is to make a python directory with mkdir python.

这里最快(但不是唯一)的方法是使用mkdir python创建一个python目录。

So now our directory structure on the container should be like so: /layer_dir/python.

因此,现在我们在容器上的目录结构应类似于: /layer_dir/python

We can remain in layer_dir and install a package directly into the python sub-directory with the following command:

我们可以保留在layer_dir并使用以下命令将软件包直接安装到python子目录中:

pip3 install requests -t /layer_dir/python

If you navigate into the python folder, you should see requests and related modules installed. Now back in layer_dir, let’s go ahead and zip up the python directory (of course, if you have more to install, go ahead, this is just a simple tutorial).

如果导航到python文件夹,则应该看到requests和相关模块已安装。 现在回到layer_dir ,让我们继续压缩python目录(当然,如果您要安装更多内容,请继续,这只是一个简单的教程)。

Run the following command:

运行以下命令:

zip -r docker_layer.zip .

The docker_layer.zip parameter can be changed to anything, it is just the name of the zip. The . will zip up everything in the directory we are in. Since we are in layer_dir and the only item in it is the python sub-directory, that is what will be zipped up (so when you upload the zip, the top level directory is python).

可以将docker_layer.zip参数更改为任何内容,它只是zip的名称。 的. 将压缩所有目录中的内容。由于我们位于layer_dir并且其中唯一的项目是python子目录,因此将被压缩(因此,当您上传zip时,顶层目录是python )。

If you ls in layer_dir, you’ll see the zip at the same level as the python sub-directory. You can modify docker_layer.zip with a path as well, but we don’t care where the zip is located, as long as we have it.

如果lslayer_dir,你会看到拉链处于同一水平的python子目录。 您也可以使用路径修改docker_layer.zip ,但是只要有它,我们就不在乎zip的位置。

Ok, so how do we get this zip onto our local machine so we can upload it?

好的,我们如何将这个zip压缩到本地计算机上以便上载它?

步骤4:将Zip从容器复制到本地计算机 (Step 4: Copy Zip from Container to Local Machine)

Open up another terminal window and navigate to the directory where your Dockerfile is (for me: Users/Nakul/Documents/Docker).

打开另一个终端窗口 ,然后导航到Dockerfile所在的目录(对我来说: Users/Nakul/Documents/Docker )。

In here, we are going to run a docker copy command.

在这里,我们将运行docker copy命令。

The basic syntax is below:

基本语法如下:

docker cp <container ID>:*source_path* *destination_path*

For me, that will be:

对我来说,将是:

docker cp <container ID>:/layer_dir/docker_layer.zip ./

You must use the container ID and not the image tag. To get the container ID, simply run docker ps to list all running instances and then copy/paste the text (this command wouldn’t have worked if you stayed in the other terminal window — still in your container).

您必须使用容器ID而不是图像标签。 要获取容器ID,只需运行docker ps列出所有正在运行的实例,然后复制/粘贴文本(如果您停留在另一个终端窗口中-仍在容器中,则此命令将无效)。

The ./ simply means the destination path will be the directory we are already in.

./只是意味着目标路径将是我们已经在的目录。

The zip should show up in your local directory after running the command, feel free to double check that if you unzip it, the top level directory is python. If it is, then great!

运行该命令后,该zip文件应显示在您的本地目录中,请仔细检查是否将其解压缩,顶级目录是python。 如果是,那就太好了!

I won’t go into how to actually upload the layer and attach it to a Lambda function as that has been covered in the following post.

我将不讨论如何实际上传图层并将其附加到Lambda函数,这将在以下文章中介绍

But hey, next time you need to upload a layer, you don’t need to spin up an EC2 instance, you can just create your layer in Docker!

但是,嘿,下一次您需要上传层时,您不需要启动EC2实例,您只需在Docker中创建您的层即可!

Note: Docker’s official documentation is quite extensive — if you want to continue playing with Docker, it might be best to just Google your way towards what you want to accomplish. Hopefully this post was a good starting point for using Docker.

注意:Docker的官方文档非常丰富—如果您想继续使用Docker,最好以Google的方式来实现自己的目标。 希望本文是使用Docker的良好起点。

谢谢阅读! (Thanks for reading!)

翻译自: https://levelup.gitconnected.com/creating-a-lambda-layer-via-docker-python-layers-docker-e4e318717822

python+docker

 类似资料: