nvidia cuda工具包更新9.0版本记录

因工作需要，对公司2台服务器升级cuda到9.0版本。本文记录其过程。

介绍

cuda在nvidia官方网站有下载，最新版本为9.2，但公司采用的tensorflow只使用9.0，为了版本的兼容，决定还是选用9.0版本。官网默认使用最新版本，旧版本位于：https://developer.nvidia.com/cuda-toolkit-archive。9.0版本地址：https://developer.nvidia.com/cuda-90-download-archive。提供Windows、Linux和MacOS三大平台安装包。而Linux平台支持CentOS、Ubuntu、Fedora、RHEL等等主流发行版本。Ubuntu只支持16.04和17.04版本。本文采用16.04版本。一共需要下载5个文件，一个安装包cuda_9.0.176_384.81_linux.run，四个补丁包，名称为cuda_9.0.176.X_linux.run，其中X为1~4。

下载

cuda包下载比较麻烦，去年曾经安装过，在md5问题上卡了很久，各种下载工具都尝试过，但md5都不正确，所以安装失败。还是托在深圳工作的前同事帮忙下载，再传回来的。现在认识到docker，就使用docker来做中转。
下载docker：

$ docker pull latelee/cuda:tar-9.0

9.0安装包位于容器根目录下。

$ docker run -it -v /home:/home --rm latelee/cuda:tar-9.0 bash
root@98dfc8d4f016:/# ls
bin                       cuda_9.0.176.2_linux.run  cuda_9.0.176_384.81_linux.run  home   media  proc  sbin  tmp
boot                      cuda_9.0.176.3_linux.run  dev                            lib    mnt    root  srv   usr
cuda_9.0.176.1_linux.run  cuda_9.0.176.4_linux.run  etc                            lib64  opt    run   sys   var

然后将.run文件拷贝到宿主机目录中。

安装

在安装前，先查看内核：

uname -a
Linux localhost 4.13.0-38-generic #43~16.04.1-Ubuntu SMP Wed Mar 14 17:48:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

再查看nvidia显卡驱动版本：

$ nvidia-smi
Thu Sep 13 09:56:26 2018       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.130                Driver Version: 384.130                   |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 1070    Off  | 00000000:01:00.0 Off |                  N/A |
| 79%   82C    P2   131W / 200W |   2865MiB /  8112MiB |    100%      Default |
+-------------------------------+----------------------+----------------------+

上面表明驱动是正常的，由于已经安装了驱动，在安装cuda时就不需要再安装了。
先卸载旧版本：

cd /usr/local/cuda-8.0/bin
sudo ./uninstall_cuda_8.0.pl

再进行安装：

sudo ./cuda_9.0.176_384.81_linux.run

首先显示协议，按q退出。下面根据步骤来安装，如下所示(各步骤给出说明)：

Do you accept the previously read EULA?
accept/decline/quit: accept  # 输入accept表示同意协议

Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 384.81?
(y)es/(n)o/(q)uit: n # 输入n不安装nvidia驱动，因为已经安装过了。

Install the CUDA 9.0 Toolkit?
(y)es/(n)o/(q)uit: y # 输入y安装cuda工具包

Enter Toolkit Location
[ default is /usr/local/cuda-9.0 ]: # 直接回车即可，使用默认目录

Do you want to install a symbolic link at /usr/local/cuda?
(y)es/(n)o/(q)uit: y # 输入y，安装链接目录

Install the CUDA 9.0 Samples?
(y)es/(n)o/(q)uit: y # 输入y安装示例

Enter CUDA Samples Location
[ default is /home/latelee ]: # 使用默认目录，直接回车

Installing the CUDA Toolkit in /usr/local/cuda-9.0 ...
Missing recommended library: libGLU.so
Missing recommended library: libX11.so
Missing recommended library: libXi.so
Missing recommended library: libXmu.so

Installing the CUDA Samples in /home/latelee ...
Copying samples to /home/latelee/NVIDIA_CUDA-9.0_Samples now...
Finished copying samples.

===========
= Summary =
===========

Driver:   Not Selected  # 由于没选择驱动，会这样打印信息出来
Toolkit:  Installed in /usr/local/cuda-9.0
Samples:  Installed in /home/latelee, but missing recommended libraries

# 设置PATH和库目录
Please make sure that
-   PATH includes /usr/local/cuda-9.0/bin
-   LD_LIBRARY_PATH includes /usr/local/cuda-9.0/lib64, or, add /usr/local/cuda-9.0/lib64 to /etc/ld.so.conf and run ldconfig as root

To uninstall the CUDA Toolkit, run the uninstall script in /usr/local/cuda-9.0/bin

Please see CUDA_Installation_Guide_Linux.pdf in /usr/local/cuda-9.0/doc/pdf for detailed information on setting up CUDA.

***WARNING: Incomplete installation! This installation did not install the CUDA Driver. A driver of version at least 384.00 is required for CUDA 9.0 functionality to work.
To install the driver using this installer, run the following command, replacing <CudaInstaller> with the name of this run file:
    sudo <CudaInstaller>.run -silent -driver

Logfile is /tmp/cuda_install_5468.log

补丁

下面安装补丁文件：

sudo ./cuda_9.0.176.1_linux.run
sudo ./cuda_9.0.176.2_linux.run
sudo ./cuda_9.0.176.3_linux.run
sudo ./cuda_9.0.176.4_linux.run

按q退出协议，输入accept，再直接回车即可，每个一补丁都如此。

环境变量

由于本文是升级，而且，安装目录做了链接，所以无须任何修改即可。如果是首次安装，则要设置环境变量，——正如上面提示信息所述那样。设置环境变量有很多种方式，这里只提一种，由于是服务器，所以最好是全局设置，所有开发人员都能使用。打开/etc/profile文件，在最后添加：

# cuda-9.0
export PATH=$PATH:/usr/local/cuda/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64

重新登陆，可执行source /etc/profile，均可生效。

验证

查看nvcc编译器版本以确认cuda版本：

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Sep__1_21:08:03_CDT_2017
Cuda compilation tools, release 9.0, V9.0.176

最后一行表示cuda已经是9.0了。接着编译并运行示例代码，进入相应示例代码目录：

$ cd /home/latelee/NVIDIA_CUDA-9.0_Samples/1_Utilities/deviceQuery

编译并运行：

$ make -j
$ ./deviceQuery

输出信息如下：

./deviceQuery Starting...

CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Device 0: "GeForce GTX 1070"
  CUDA Driver Version / Runtime Version          9.0 / 9.0
  CUDA Capability Major/Minor version number:    6.1
  Total amount of global memory:                 8113 MBytes (8506769408 bytes)
  (15) Multiprocessors, (128) CUDA Cores/MP:     1920 CUDA Cores
  GPU Max Clock rate:                            1709 MHz (1.71 GHz)
  Memory Clock rate:                             4004 Mhz
  Memory Bus Width:                              256-bit
  L2 Cache Size:                                 2097152 bytes
  Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
  Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       49152 bytes
  Total number of registers available per block: 65536
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  2048
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size    (x,y,z): (2147483647, 65535, 65535)
  Maximum memory pitch:                          2147483647 bytes
  Texture alignment:                             512 bytes
  Concurrent copy and kernel execution:          Yes with 2 copy engine(s)
  Run time limit on kernels:                     No
  Integrated GPU sharing Host Memory:            No
  Support host page-locked memory mapping:       Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Disabled
  Device supports Unified Addressing (UVA):      Yes
  Supports Cooperative Kernel Launch:            Yes
  Supports MultiDevice Co-op Kernel Launch:      Yes
  Device PCI Domain ID / Bus ID / location ID:   0 / 1 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 9.0, CUDA Runtime Version = 9.0, NumDevs = 1
Result = PASS

最后的PASS表示程序测试通过。

小结

如果已经安装过nvidia显卡驱动，在安装cuda时就不能选择安装驱动，否则，因为驱动已经被加载，无法安装成功。需要先卸载，但制裁重新安装比较麻烦，考虑到年代较近（去年年底才安装），因为就不升级驱动了。

李迟 2018.9.14 周五 夜