尝试使用Torch/PyTorch创建docker映像时出现MemoryError

Question

我遵循本教程为烧瓶应用程序创建Docker映像。应用程序取决于Torch/PyTorch。因此，我的requirements.txt文件如下所示。

flask
flask-cors
pytorch
torchvision
pandas

我的Dockerfile如下所示。

FROM ubuntu:latest
LABEL My Company "[e-mail]"
RUN apt-get update -y
RUN apt-get install -y python-pip python-dev build-essential
COPY . /app
WORKDIR /app
RUN pip install -vvv --no-cache-dir -r requirements.txt
ENTRYPOINT ["python3"]
CMD ["app.py"]

当我输入命令docker build-t flask sample one:latest时 我在Mac上收到以下错误消息（当pip尝试安装torch时发生）。

Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 620, in _prepare_file
    session=self.session, hashes=hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 821, in unpack_url
    hashes=hashes
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 659, in unpack_http_url
    hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 882, in _download_http_url
    _download_url(resp, link, content_file, hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 603, in _download_url
    hashes.check_against_chunks(downloaded_chunks)
  File "/usr/lib/python2.7/dist-packages/pip/utils/hashes.py", line 46, in check_against_chunks
    for chunk in chunks:
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 571, in written_chunks
    for chunk in chunks:
  File "/usr/lib/python2.7/dist-packages/pip/utils/ui.py", line 139, in iter
    for x in it:
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 560, in resp_read
    decode_content=False):
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 436, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 384, in read
    data = self._fp.read(amt)
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/filewrapper.py", line 63, in read
    self._close()
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/filewrapper.py", line 50, in _close
    self.__callback(self.__buf.getvalue())
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/controller.py", line 275, in cache_response
    self.serializer.dumps(request, response, body=body),
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/serialize.py", line 55, in dumps
    "body": _b64_encode_bytes(body),
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/serialize.py", line 12, in _b64_encode_bytes
    return base64.b64encode(b).decode("ascii")
MemoryError
The command '/bin/sh -c pip install -vvv -r requirements.txt' returned a non-zero code: 2

我不确定内存错误是从哪里来的。我试图用16GB内存在Macbook Pro上构建docker图像。关于正在发生的事情或如何使用Python创建docker映像，有什么想法吗？

在AWSEC2实例上，我得到以下错误。

Running setup.py install for pytorch: started
    Running command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-SeBh33/pytorch/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-5jxyuS-record/install-record.txt --single-version-externally-managed --compile
    Traceback (most recent call last):
      File "", line 1, in 
      File "/tmp/pip-build-SeBh33/pytorch/setup.py", line 13, in 
        raise Exception(message)
    Exception: You should install pytorch from http://pytorch.org
    Running setup.py install for pytorch: finished with status 'error'
Cleaning up...
  Removing source in /tmp/pip-build-SeBh33/pytorch
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-SeBh33/pytorch/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-5jxyuS-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-SeBh33/pytorch/
Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 360, in run
    prefix=options.prefix_path,
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/usr/lib/python2.7/dist-packages/pip/req/req_install.py", line 878, in install
    spinner=spinner,
  File "/usr/lib/python2.7/dist-packages/pip/utils/__init__.py", line 725, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-SeBh33/pytorch/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-5jxyuS-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-SeBh33/pytorch/
The command '/bin/sh -c pip install -vvv --no-cache-dir -r requirements.txt' returned a non-zero code: 1

对发生了什么有什么想法吗？我只是试图用这几个依赖项构建一个docker映像，这样我就可以从烧瓶应用程序中提供一个PyTorch模型。我不确定是否有这样做的教程（还找不到）。




Answer 1

如果有人感兴趣，我可以将PyTorch安装到docker容器中，如下所示。

我修改了要求。txt显示如下所示。

flask
flask-cors
pandas

然后我修改了Dockerfile，如下所示。这很奇怪，并没有遵循使用pip安装Python包的传统方式，但它可以工作。注意，我必须先安装Python3，然后安装PyTorch wheel，最后安装piptorchvision。

FROM ubuntu:latest
LABEL My Company "[e-mail]"
RUN apt-get update -y
RUN apt-get install -y python-pip python-dev build-essential
RUN apt-get install -y python3.6
RUN apt-get install -y python3-pip
COPY . /app
WORKDIR /app
RUN pip3 install -vvv --no-cache-dir -r requirements.txt
RUN pip3 install http://download.pytorch.org/whl/cpu/torch-0.4.1-cp36-cp36m-linux_x86_64.whl
RUN pip3 install torchvision
ENTRYPOINT ["python3"]
CMD ["app.py"]