AWS SageMaker TensorFlow Serving - 终端节点故障 - CloudWatch 日志参考：“NET_LOG：进入事件循环...”

Question

这是我第一次使用sagemaker来服务我自己的自定义张量流模型，所以我一直在使用媒体文章来让我开始：

如何为AWS SageMaker创建TensorFlow服务容器＜br＞如何将Docker映像推送到AWS ECS存储库＜br＞使用TensorFlow服务部署AWS Sage Maker容器＜br>如何使用Tensor Flow服务对SageMakendpoint进行预测

我设法创建了我的服务容器，成功地将其推送到ECR，并从我的docker映像创建了sagemaker模型。但是，当我尝试创建endpoint时，它开始创建，但在3-5分钟后以失败消息结束:

生产变体默认的主容器未通过ping运行状况检查。请检查此endpoint的CloudWatch日志。

故障图像

然后我检查了我的云观察日志，看起来像这样...

云监视日志

...以“NET_LOG：进入事件循环......”结尾

我试图在谷歌上搜索更多关于使用tf服务部署sagemaker模型的日志消息，但找不到任何有用的解决方案。

为了给出更多的背景，在遇到这个问题之前，我遇到了另外两个问题:

>

"FileSystemStoragePathSource遇到文件系统访问错误：找不到基路径

MODEL_PATH/MODEL_NAME/MODEL_NAME"

我通过以下链接解决了这两个问题:

[文档]TensorFlowModelendpoint需要export/Servo文件夹结构，但没有记录

失败原因：生产变体All流量的主容器未通过ping健康检查。

同样值得注意的是，我的Tensorflow模型是使用TF版本2.0创建的（因此我需要docker容器）。我只使用 AWS CLI 来执行我的 tensorflow 服务，而不是 sagemaker SDK。

以下是我的 shell 脚本的片段：

nginx.config

events {
    # determines how many requests can simultaneously be served
    # https://www.digitalocean.com/community/tutorials/how-to-optimize-nginx-configuration
    # for more information
    worker_connections 2048;
}

http {
  server {
    # configures the server to listen to the port 8080
    # Amazon SageMaker sends inference requests to port 8080.
    # For more information: https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-inference-code.html#your-algorithms-inference-code-container-response
    listen 8080 deferred;

    # redirects requests from SageMaker to TF Serving
    location /invocations {
      proxy_pass http://localhost:8501/v1/models/pornilarity_model:predict;
    }

    # Used by SageMaker to confirm if server is alive.
    # https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-inference-code.html#your-algorithms-inference-algo-ping-requests
    location /ping {
      return 200 "OK";
    }
  }
}

文档文件

# RUN pip install sagemaker-containers

# Installing NGINX, used to reverse proxy the predictions from SageMaker to TF Serving
RUN apt-get update && apt-get install -y --no-install-recommends nginx git

# Copy our model folder to the container 
# NB: Tensorflow serving requires you manually assign version numbering to models e.g. model_path/1/
# see below links: 

# https://stackoverflow.com/questions/45544928/tensorflow-serving-no-versions-of-servable-model-found-under-base-path
# https://github.com/aws/sagemaker-python-sdk/issues/599
COPY pornilarity_model /opt/ml/model/export/Servo/1/

# Copy NGINX configuration to the container
COPY nginx.conf /opt/ml/code/nginx.conf

# Copies the hosting code inside the container
# COPY serve.py /opt/ml/code/serve.py

# Defines serve.py as script entrypoint
# ENV SAGEMAKER_PROGRAM serve.py

# starts NGINX and TF serving pointing to our model
ENTRYPOINT service nginx start | tensorflow_model_server --rest_api_port=8501 \
 --model_name=pornilarity_model \
 --model_base_path=/opt/ml/model/export/Servo/

构建和推动

%%sh

# The name of our algorithm
ecr_repo=sagemaker-tf-serving
docker_image=sagemaker-tf-serving

cd container

# chmod a+x container/serve.py

account=$(aws sts get-caller-identity --query Account --output text)

# Get the region defined in the current configuration (default to us-west-2 if none defined)
region=$(aws configure get region)
region=${region:-eu-west-2}

fullname="${account}.dkr.ecr.${region}.amazonaws.com/${ecr_repo}:latest"

# If the repository doesn't exist in ECR, create it.

aws ecr describe-repositories --repository-names "${ecr_repo}" > /dev/null 2>&1

if [ $? -ne 0 ]
then
    aws ecr create-repository --repository-name "${ecr_repo}" > /dev/null
fi

# Get the login command from ECR and execute it directly
$(aws ecr get-login --region ${region} --no-include-email)

# Build the docker image locally with the image name and then push it to ECR
# with the full name.

docker build -t ${docker_image} .
# docker tag ${docker_image} ${fullname}
docker tag ${docker_image}:latest ${fullname}

docker push ${fullname}

创建SageMaker模型

#!/usr/bin/env bash

CONTAINER_NAME="Pornilarity-Container"
MODEL_NAME=pornilarity-model-v1

# the role named created with
# https://gist.github.com/mvsusp/599311cb9f4ee1091065f8206c026962
ROLE_NAME=AmazonSageMaker-ExecutionRole-20191202T133391

# the name of the image created with
# https://gist.github.com/mvsusp/07610f9cfecbec13fb2b7c77a2e843c4
ECS_IMAGE_NAME=sagemaker-tf-serving
# the role arn of the role
EXECUTION_ROLE_ARN=$(aws iam get-role --role-name ${ROLE_NAME} | jq -r .Role.Arn)

# the ECS image URI
ECS_IMAGE_URI=$(aws ecr describe-repositories --repository-name ${ECS_IMAGE_NAME} |\
jq -r .repositories[0].repositoryUri)

# defines the SageMaker model primary container image as the ECS image
PRIMARY_CONTAINER="ContainerHostname=${CONTAINER_NAME},Image=${ECS_IMAGE_URI}"

# Createing the model
aws sagemaker create-model --model-name ${MODEL_NAME} \
--primary-container=${PRIMARY_CONTAINER}  --execution-role-arn ${EXECUTION_ROLE_ARN}

endpoint配置

#!/usr/bin/env bash

MODEL_NAME=pornilarity-model-v1

ENDPOINT_CONFIG_NAME=pornilarity-model-v1-config

ENDPOINT_NAME=pornilarity-v1-endpoint

PRODUCTION_VARIANTS="VariantName=Default,ModelName=${MODEL_NAME},"\
"InitialInstanceCount=1,InstanceType=ml.c5.large"

aws sagemaker create-endpoint-config --endpoint-config-name ${ENDPOINT_CONFIG_NAME} \
--production-variants ${PRODUCTION_VARIANTS}

aws sagemaker create-endpoint --endpoint-name ${ENDPOINT_NAME} \
--endpoint-config-name ${ENDPOINT_CONFIG_NAME}

Docker容器文件夹结构

├── container
│   ├── Dockerfile
│   ├── nginx.conf
│   ├── pornilarity_model
│   │   ├── assets
│   │   ├── saved_model.pb
│   │   └── variables
│   │       ├── variables.data-00000-of-00002
│   │       ├── variables.data-00001-of-00002
│   │       └── variables.index

任何指导都将不胜感激！！

Answer 1

路径上的 Web 服务器

必须公开活跃度响应，如果一切都在运行，返回200。现在它不见了，Sagemaker不认为它是一个有效的推理容器。就是这么简单:)

web服务器已经为您提供了容器，在您的情况下，Tensorfolow服务容器可在以下站点公开获取：

763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.4.1-cpu-py37-ubuntu18.04

这是us-East和CPU类型推断的示例。您可以添加模型 /opt/ml/model/就像您在示例docker中所做的那样。

对于所有可用的容器，请查看AWS的深度学习容器