python3.7.3+apache web+flask+redis+pytorch_model

硬件配置

滴滴云CPU服务器，ubuntu16.04

环境配置

详细步骤参考https://www.pyimagesearch.com/2018/02/05/deep-learning-production-keras-redis-flask-apache/，此处，仅仅就本人安装过程中出现的一些问题予以记录。

源码编译安装python3.7.3

强调必须源码编译，除了安装python之外，还需要安装python-dev；非常重要，否则在源码编译安装libapache2-mod-wsgi-py3的.so，有可能无法使用。

#!/bin/bash
apt-get update
apt-get -y upgrade
apt-get install -y curl wget git tar
# vim/emacs
apt-get install -y vim emacs
apt install -y build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev libsqlite3-dev
apt install -y liblzma-dev
#尤其要注意，./configure --enable-shared，生成libpython3.7m.so.1.0的动态库
cd /usr/src ; wget https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz ; tar -xf Python-3.7.3.tar.xz ; cd Python-3.7.3 ; ./configure --enable-optimizations --enable-shared ; make altinstall
# replace python version to have 3.7.3 as default
rm -f /usr/bin/python
rm -f /usr/bin/python3
ln -s /usr/local/bin/python3.7 /usr/bin/python
ln -s /usr/local/bin/python3.7 /usr/bin/python3
ln -s /usr/local/bin/python3.7 /usr/local/bin/python
ln -s /usr/local/bin/python3.7 /usr/local/bin/python3
# create links to pip3.7
ln -s /usr/local/bin/pip3.7 /usr/bin/pip
ln -s /usr/local/bin/pip3.7 /usr/bin/pip3
ln -s /usr/local/bin/pip3.7 /usr/local/bin/pip
ln -s /usr/local/bin/pip3.7 /usr/local/bin/pip3
# 下面这些库是为了方便使用opencv4.0
apt update
apt install -y libsm6 libxext6 libxrender-dev
apt install -y libgl1-mesa-glx
pip3 install --upgrade pip
pip3 config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

上述sh文件是参考下述GPU镜像文件，一并附录在此，供日后制作GPU镜像时使用。

FROM nvidia/cuda:10.2-runtime-ubuntu16.04
MAINTAINER [e-mail]
RUN apt-get update
RUN apt-get -y upgrade
# curl/wget/git
RUN apt-get install -y curl wget git tar
# vim/emacs
RUN apt-get install -y vim emacs

RUN apt install -y build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev libsqlite3-dev
RUN apt install -y liblzma-dev
RUN cd /usr/src ; wget https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz ; tar -xf Python-3.7.3.tar.xz ; cd Python-3.7.3 ; ./configure --enable-optimizations --enable-shared ; make altinstall

# be sure it's 3.7 and not 3.6
RUN ! ls /usr/local/bin/python3.7 && ls /usr/src/Python-3.7.3/python && cp /usr/src/Python-3.7.3/python /usr/local/bin/python3.7 ; exit 0

# replace python version to have 3.7.3 as default
RUN rm -f /usr/bin/python
RUN rm -f /usr/bin/python3
RUN ln -s /usr/local/bin/python3.7 /usr/bin/python
RUN ln -s /usr/local/bin/python3.7 /usr/bin/python3
RUN ln -s /usr/local/bin/python3.7 /usr/local/bin/python
RUN ln -s /usr/local/bin/python3.7 /usr/local/bin/python3

# create links to pip3.7
RUN ln -s /usr/local/bin/pip3.7 /usr/bin/pip
RUN ln -s /usr/local/bin/pip3.7 /usr/bin/pip3
RUN ln -s /usr/local/bin/pip3.7 /usr/local/bin/pip
RUN ln -s /usr/local/bin/pip3.7 /usr/local/bin/pip3

ENV CUDNN_VERSION 8.2.0.53

LABEL com.nvidia.cudnn.version="${CUDNN_VERSION}"

RUN apt-get update && apt-get install -y --no-install-recommends \
    libcudnn8=$CUDNN_VERSION-1+cuda10.2 \
    && apt-mark hold libcudnn8 && \
    rm -rf /var/lib/apt/lists/*

RUN apt update
RUN apt install -y libsm6 libxext6 libxrender-dev
RUN apt install -y libgl1-mesa-glx
RUN pip3 install --upgrade pip
RUN pip3 config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

运行上述sh文件，安装好python3.7之后，在/usr/local/lib目录下会有libpython3.7*.so.*的文件，但是没有libpython3.7.so的文件，所以，此处需要建立一个软链接，供源码编译libapache2-mod-wsgi-py3时使用。

cd /usr/local/lib
#必须链接到libpython3.7m.so.1.0上
ln -s libpython3.7.so libpython3.7m.so.1.0

再将/usr/local/lib添加到环境变量LD_LIBRARY_PATH中，方法是：

vim ~/.bashrc
#添加语句，并退出
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
source ~/.bashrc

至此，python3.7安装完成，可以进行下述步骤。

源码编译安装redis数据库

wget http://download.redis.io/redis-stable.tar.gz
tar xvzf redis-stable.tar.gz
cd redis-stable
make
make install

安装apache web server

apt install apache2-dev

源码编译安装libapache2-mod-wsgi-py3.7.3

参考https://modwsgi.readthedocs.io/en/develop/user-guides/quick-installation-guide.html，如果wget下载失败，就手动下载，我这里下载的是版本3.4，下载完成后解压得到文件夹mod_wsgi-3.4

cd mod_wsgi-3.4
make clean
#此处需要指定python解释器的版本，因为在安装python3.7的时候，默认python就是python3.7，所以不需要额外指定，具体可以参考上面给出的参考网页
./configure
make
make install

完成之后，一定要到/usr/lib/apache2/modules/下，确认mod_wsgi.so是否正确链接到我们指定的python版本，方法是：

ldd /usr/lib/apache2/modules/mod_wsgi.so

如果python3.7是严格按照上述步骤安装，应该都没有问题。此时,libapache2-mod-wsgi-py3.7.3的python版本和我们使用的python版本一致，就可以正确使用。如果版本不一致，则会出现找不到python包的问题；

安装完成后，完成后续设置：

vim ~/.bashrc
#添加语句，并退出
export LD_LIBRARY_PATH=/usr/lib/apache2/modules:$LD_LIBRARY_PATH
source ~/.bashrc
#这一步是非常关键的，否则在启动apache2的服务时，会出现各种变量找不到的问题
source /etc/apache2/envvars
#下面这一步是导入wsgi包，否则在配置文件
a2enmod wsgi

在/etc/apache2/apache2.conf中定义：

vim /etc/apache2/apache2.conf
#定义servername，有人也提出需要在httpd.conf中定义，但我这里因为调用的是apache2.conf文件，所以，必须在apache2.conf里定义；具体用那个配置文件，在service apache2 restart时，可以查看出错信息
ServerName localhost:80

创建服务运行的虚拟环境

建议在虚拟环境中运行自己的服务。方法：

virtualenv my_env -p /usr/local/bin/python3.7
source my_env/bin/activate
pip install flask
pip install gevent
pip install requests
pip install redis

至此，环境已经配置好。接下来，需要告诉apache2服务，应该指向那个flask app（由python编写）。这其实是两个步骤：
步骤一：apache2服务在启动时，需要知道加载那个wsgi脚本；为了叙述方便，假定这个脚本名字为my_api_app.wsgi；将其路径配置在文件/etc/apache2/sites-available/000-default.conf中，方法是：

#当没有a2enmod wsgi时，需要运行下面的命令，以加载mod_wsgi库
#LoadModule wsgi_module /usr/lib/apache2/modules/mod_wsgi.so
#下面这个变量是指向python解释器的，因为我加了之后，出现错误，网上有人建议不要加，所以就注释掉了，原因不明确
#WSGIPythonHome /usr/bin
#指定python包的安装位置，否则在运行python脚本时，会报找不到包的错误，注意此处的my_env就是配置的python虚拟环境
WSGIPythonPath /home/my_env/lib/python3.7/site-packages
<VirtualHost *:80>
        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        #ServerName www.example.com

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html
		#注意看my_api_app.wsgi,my_python_webservice
        WSGIDaemonProcess my_api_app threads=10
        WSGIScriptAlias / /var/www/html/my_python_webservice/my_api_app.wsgi

        <Directory /var/www/html/my_python_webservice>
                WSGIProcessGroup my_api_app
                WSGIApplicationGroup %{GLOBAL}
                Order deny,allow
                Allow from all
        </Directory>

        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
        #Include conf-available/serve-cgi-bin.conf
</VirtualHost>

步骤二：在my_api_app.wsgi里面，加载python编写的flask app；见下述代码：

#my_api_app.wsgi
import sys
#/var/www/是apache2 web服务器的默认路径，请求来了之后，默认就在这个路径下找服务运行，我们应该把我们的服务放在这个路径下，但为了方便，一般是建立一个链接，链接到我们自己的服务my_python_webservice上
sys.path.insert(0,"/var/www/my_python_webservice")
#my_web_server.py是服务my_python_webservice下的一个python脚本，它创建了一个flask app
from my_web_server import app as application

启动服务

启动时务必保证是在虚拟环境下,否则有错

#启动apache2指向的flask应用程序，每次flask的python程序有变化，都需要重启
service apache2 restart
#如果服务启动错误，可以通过下面的命令来查看
apache2 --help
#如果服务启动成功，但脚本有错误，可以通过下面的命令来监控
tail -f /var/log/apache2/error.log
#启动redis-server
redis-server
#启动pytorch_model服务
cd /var/www/my_python_webservice
python my_pytorch_model.py

附件

my_web_server.py

import numpy as np
import settings
import flask
import redis
import uuid
import time
import json
import io
# initialize our Flask application and Redis server
app = flask.Flask(__name__)
db = redis.StrictRedis(host=settings.REDIS_HOST,
	port=settings.REDIS_PORT, db=settings.REDIS_DB)

@app.route("/")
def homepage():
	return "Welcome to the my_web_server!"
@app.route("/predict", methods=["POST"])
def predict():
	# initialize the data dictionary that will be returned from the
	# view
	data = {"success": False}
	# ensure an image was properly uploaded to our endpoint
	if flask.request.method == "POST":
		data = json.loads(flask.request.data)
		if "url" in data:
			# print("enter into inference...")
			# read the image in PIL format and prepare it for
			# classification
			url = data['url']
			# generate an ID for the classification then add the
			# classification ID + image to the queue
			k = str(uuid.uuid4())
			d = {"id": k, "url": url}
			db.rpush(settings.URL_QUEUE, json.dumps(d))
			# keep looping until our model server returns the output
			# predictions
			while True:
				# attempt to grab the output predictions
				output = db.get(k)
				# check to see if our model has classified the input
				# image
				if output is not None:
					# add the output predictions to our data
					# dictionary so we can return it to the client
					data["predictions"] = json.loads(output)
					# delete the result from the database and break
					# from the polling loop
					db.delete(k)
					break
				# sleep for a small amount to give the model a chance
				# to classify the input image
				time.sleep(settings.CLIENT_SLEEP)
			# indicate that the request was a success
			data["success"] = True
	# return the data dictionary as a JSON response
	# print(data)
	return flask.jsonify(data)
# for debugging purposes, it's helpful to start the Flask testing
# server (don't use this for production
if __name__ == "__main__":
	print("* Starting web service...")
	app.run()

my_pytorch_model.py

from my_model import MyModel
import numpy as np
import settings
import redis
import time
import json
# connect to Redis server
db = redis.StrictRedis(host=settings.REDIS_HOST,
	port=settings.REDIS_PORT, db=settings.REDIS_DB)
def classify_process():
	# load the pre-trained Keras model (here we are using a model
	# pre-trained on ImageNet and provided by Keras, but you can
	# substitute in your own networks just as easily)
	print("* Loading model...")
	model = MyModel()
	print("* Model loaded")
	# continually pool for new images to classify
	while True:
		# attempt to grab a batch of images from the database, then
		# initialize the image IDs and batch of images themselves
		queue = db.lrange(settings.URL_QUEUE, 0,
			settings.BATCH_SIZE - 1)
		imageIDs = []
		# loop over the queue
		model_input = {"example_list":[]}
		for q in queue:
			q = json.loads(q)
			image_url = q['url']			model_input["url_list"].append({"url":image_url})
			imageIDs.append(q["id"])
		if len(imageIDs) > 0:
			# classify the batch
			print("* image size: {}".format(len(model_input["url_list"])))
			model_output = model.predict(model_input)
			print(model_output)
			# loop over the image IDs and their corresponding set of
			# results from our model
			for (imageID, resultSet) in zip(imageIDs, model_output["predict_list"]):
				# initialize the list of output predictions
				# store the output predictions in the database, using
				# the image ID as the key so we can fetch the results
				print(imageID,resultSet)
				db.set(imageID, json.dumps(resultSet))
			# remove the set of images from our queue
			db.ltrim(settings.URL_QUEUE, len(imageIDs), -1)
		# sleep for a small amount
		time.sleep(settings.SERVER_SLEEP)
# if this is the main thread of execution start the model server
# process
if __name__ == "__main__":
	classify_process()

client.py

from threading import Thread
import requests
import time
import json
# initialize the Keras REST API endpoint URL along with the input
# image path
API_URL = "http://localhost/predict"
IMAGE_URL = "https://sdf/0064C656-F7B3-44a0-9220-6614E9F6587F.png"
# initialize the number of requests for the stress test along with
# the sleep amount between requests
NUM_REQUESTS = 10
SLEEP_COUNT = 0.05
def call_predict_endpoint(n):
    # load the input image and construct the payload for the request
    payload = {"url": IMAGE_URL}
    # submit the request
    r = requests.post(API_URL, data=json.dumps(payload)).json()
    # ensure the request was sucessful
    if r["success"]:
        print("[INFO] thread {} OK".format(n))
        print(r)
    # otherwise, the request failed
    else:
        print("[INFO] thread {} FAILED".format(n))
# loop over the number of threads
for i in range(0, NUM_REQUESTS):
    # start a new thread to call the API
    t = Thread(target=call_predict_endpoint, args=(i,))
    t.daemon = True
    t.start()
    time.sleep(SLEEP_COUNT)
# insert a long sleep so we can wait until the server is finished
# processing the images
time.sleep(100)

settings.py

# initialize Redis connection settings
REDIS_HOST = "localhost"
REDIS_PORT = 6379
REDIS_DB = 0
# initialize constants used for server queuing
URL_QUEUE = "url_queue"
BATCH_SIZE = 32
SERVER_SLEEP = 0.25
CLIENT_SLEEP = 0.25