KVM+WebVirtMgr部署安装笔记

一、安装epel源
yum install wget -y
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
yum -y install epel-release
yum clean all
yum makecache

替换源了之后才有python-pip 这个安装包
二、做好网卡桥接(这里省略，参考Centos学习笔记)
配置网卡桥接：
配置方法：
桥接之前需要关闭systemctl disable NetworkManager systemctl stop NetworkManager
1、宿主机，工作网卡为eth0
cd /etc/sysconfig/network-script/ ##进入宿主机网卡文件目录
cp ifcfg-eth0/root/ifcfg-eth0.bak ##备份宿主机网卡文件，用于日后有需要的时候备份

2、修改宿主机工作网卡文件。vim ifcfg-eth0
[root@localhost network-scripts]# vim ifcfg-eth0
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=eth0
DEVICE=eth0
ONBOOT=yes
BRIDGE=br0

3、新增宿主机，网桥设备br0
vim ifcfg-br0 #模式为静态，类型是网桥，不受NetworkManager控制，定义IP等
[root@localhost network-scripts]# vim ifcfg-br0
TYPE=Bridge
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=br0
DEVICE=br0
ONBOOT=yes
IPADDR=192.168.10.246
PREFIX=24
GATEWAY=192.168.10.254
DNS1=8.8.8.8
NM_CONTROLLED=no

4、重启network
systemctl restart network

5、查看桥接状况
brctl show
[root@localhost network-scripts]# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.00e04c6be82b no eth0
virbr0 8000.5254001338d1 yes virbr0-nic
br0设备，绑定了eth0网卡成功。如果stp enabled为no，可以执行brctl stp br0 on 打开。
另外可以使用 route -n 查看路由情况
[root@localhost network-scripts]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.10.254 0.0.0.0 UG 0 0 0 br0
169.254.0.0 0.0.0.0 255.255.0.0 U 1006 0 0 br0
192.168.10.0 0.0.0.0 255.255.255.0 U 0 0 0 br0
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0

上图可以看到，所有地址，网关为 192.168.10.254 ，通过br0，进行通信。

删除桥接
1、按顺序执行如下
1）关闭虚拟网卡br0
[root@node1 ~]# ifdown br0
2）删除桥接
[root@node1 ~]# brctl delbr br0
3）删除br0的配置文件
[root@node1 ~]# rm ifcfg-br0
4）重新给物理网卡eth0配置ip
[root@node1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.0.111
NETWORK=255.255.255.0
GATEWAY=192.168.0.254
5）重启网络服务network
systemctl restart network

三、安装KVM
yum install qemu-kvm libvirt libvirt-python libguestfs-tools virt-install virt-manager python-virtinst libvirt-client virt-viewer -y
1、启动libvirt
[root@localhost ~]# systemctl restart libvirtd
[root@localhost ~]# systemctl status libvirtd
● libvirtd.service - Virtualization daemon
Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
Active: active (running) since 五 2019-12-06 12:36:12 +07; 33s ago
Docs: man:libvirtd(8)
2、测试
[root@localhost ~]# virsh -c qemu:///system list
Id 名称 状态

[root@localhost ~]# virsh --version
4.5.0
[root@localhost ~]# virt-install --version
1.5.0
[root@localhost ~]# ln -s /usr/libexec/qemu-kvm /usr/bin/qemu-kvm
[root@localhost ~]# lsmod |grep kvm
kvm_intel 170086 0
kvm 566340 1 kvm_intel
irqbypass 13503 1 kvm

四、部署webvirtmgr
1、安装依赖包
yum install git python-pip libvirt-python libxml2-python python-websockify supervisor nginx -y
2、从git-hub中下载相关的webvirtmgr代码
[root@localhost ~]# cd /usr/local/src/
[root@localhost src]# git clone git://github.com/retspen/webvirtmgr.git
3、安装webvirtmgr
[root@localhost src]# cd webvirtmgr
[root@localhost webvirtmgr]# pip install -r requirements.txt
4、检查sqlite3 （备注：自带不需要安装，导入模块检查一下。）
[root@localhost webvirtmgr]# python
Python 2.7.5 (default, Aug 4 2017, 00:39:18)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux2
Type “help”, “copyright”, “credits” or “license” for more information.

import sqlite3
exit()

5、初始化账号
[root@localhost webvirtmgr]# ./manage.py syncdb
WARNING:root:No local_settings file found.
Creating tables …
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_groups
Creating table auth_user_user_permissions
Creating table auth_user
Creating table django_content_type
Creating table django_session
Creating table django_site
Creating table servers_compute
Creating table instance_instance
Creating table create_flavor
You just installed Django’s auth system, which means you don’t have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (leave blank to use ‘root’): admin
Email address: ***@.com
Password:
Password (again):
Superuser created successfully.
Installing custom SQL …
Installing indexes …
Installed 6 object(s) from 1 fixture(s)

./manage.py collectstatic #生成配置文件
./manage.py createsuperuser #创建超级管理员用户：

6、拷贝web到 相关目录
[root@localhost webvirtmgr]# mkdir -pv /var/www
mkdir: 已创建目录 “/var/www”
[root@localhost webvirtmgr]# cp -Rv /usr/local/src/webvirtmgr /var/www/webvirtmgr

7、设置ssh
[root@localhost webvirtmgr]# ssh-keygen -t rsa //产生公私钥
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 这里不要输入名字，直接回车
Enter passphrase (empty for no passphrase): 这里输入密码
Enter same passphrase again: 这里重复输入密码
Your identification has been saved in kvmrsa.
Your public key has been saved in kvmrsa.pub.
The key fingerprint is:
SHA256:NDxQIMH+SVBSxPrZ/7G0wTvzPXbnnl2H3WKbS3vxaVM [e-mail]
The key’s randomart image is:
±–[RSA 2048]----+
| .+**o. |
| oo.o |
| . o = |
| o … o |
| + +S |
| = . . +E|
| . = = X|
| …oO %X|
| .=+O*X|
±—[SHA256]-----+

如果root的家目录没有.ssh
执行：ssh localhost就生成.ssh目录

[root@localhost ~]# ssh-copy-id 192.168.10.246
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: “/root/.ssh/id_rsa.pub”
The authenticity of host ‘192.168.10.246 (192.168.10.246)’ can’t be established.
ECDSA key fingerprint is SHA256:p066y9w2mzxh1CY0Ku+8ANcyLNlImv8hFtpxWx93QFI.
ECDSA key fingerprint is MD5:72:b9:4d:16:bc:0e:ba:25:f9:38:e6:b3:43:17:b2:a6.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed – if you are prompted now it is to install the new keys
root@192.168.10.246’s password:

Number of key(s) added: 1

Now try logging into the machine, with: “ssh ‘192.168.10.246’”
and check to make sure that only the key(s) you wanted were added.

8、编辑nginx配置文件
[root@localhost ~]# cd /etc/nginx/
[root@localhost nginx]# mv nginx.conf /tmp
[root@localhost nginx]# cp nginx.conf.default nginx.conf
[root@localhost nginx]# vim nginx.conf
添加：
http {
include mime.types;
default_type application/octet-stream;
include /etc/nginx/conf.d/*.conf;
9、添加 /etc/nginx/conf.d/webvirtmgr.conf 配置文件
[root@localhost conf.d]# vim webvirtmgr.conf
server {
listen 80 default_server;

   server_name $hostname;
   #access_log /var/log/nginx/webvirtmgr_access_log;

   location /static/ {
            root /var/www/webvirtmgr/webvirtmgr; # or /srv instead of /var
            expires max;
   }

   location / {
            proxy_pass http://127.0.0.1:8000;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-for $proxy_add_x_forwarded_for;
            proxy_set_header Host $host:$server_port;
            proxy_set_header X-Forwarded-Proto $remote_addr;
            proxy_connect_timeout 600;
            proxy_read_timeout 600;
            proxy_send_timeout 600;
            client_max_body_size 1024M; # Set higher depending on your needs
   }

}
10、重启nginx服务
[root@localhost conf.d]# systemctl restart nginx

11、修改防火墙规则
[root@ops ~]# vim /etc/sysconfig/selinux
…
SELINUX=disabled
#临时生效
[root@localhost conf.d]# setenforce 0 设置这里感觉没什么作用

12、授权
chown -R nginx:nginx /var/www/webvirtmgr

13、设置 supervisor (如果iptables防火墙开启的话，就必须要开通80、8000、6080端口访问)
在supervisord.d目录下新建.ini文件
[root@test]# vim /etc/supervisord.conf //在文件末尾添加,注意将默认的python改为python2，因为上面只有用这个版本执行才不报错！
[program:webvirtmgr]
command=/usr/bin/python2 /var/www/webvirtmgr/manage.py run_gunicorn -c /var/www/webvirtmgr/conf/gunicorn.conf.py //启动8000端口
directory=/var/www/webvirtmgr
autostart=true
autorestart=true
logfile=/var/log/supervisor/webvirtmgr.log
log_stderr=true
user=nginx

[program:webvirtmgr-console]
command=/usr/bin/python2 /var/www/webvirtmgr/console/webvirtmgr-console //启动6080端口（这是控制台vnc端口）
directory=/var/www/webvirtmgr
autostart=true
autorestart=true
stdout_logfile=/var/log/supervisor/webvirtmgr-console.log
redirect_stderr=true
user=nginx

14、检查
[root@test]#vim /var/www/webvirtmgr/conf/gunicorn.conf.py //确保下面bind绑定的是本机的8000端口，这个在nginx配置中定义了，被代理的端口
bind = ‘127.0.0.1:8000’
15、设置开机启动
[root@localhost etc]# systemctl enable supervisord.service

#重启服务
[root@webvirtmg nginx]# systemctl restart supervisord
[root@webvirtmg nginx]# systemctl status supervisord
这个服务是个守护进程配置文件在 vim /etc/supervisord.conf
与第13点相关联

virsh使用qemu+tcp访问远程libvirtd
因为ssh的不能访问 所以使用tcp进行对远程libvirtd进行连接访问，例如
virsh -c qemu+tcp://example.com/system
修改文件vim /etc/sysconfig/libvirtd，用来启用tcp的端口
LIBVIRTD_CONFIG=/etc/libvirt/libvirtd.conf
LIBVIRTD_ARGS="–listen"
修改文件vim /etc/libvirt/libvirtd.conf
listen_tls = 0
listen_tcp = 1
tcp_port = “16509”
listen_addr = “0.0.0.0”
auth_tcp = “none”
运行 libvirtd
service libvirtd restart
systemctl restart libvirtd.service
如果没起效果(我的就没有生效  )，那么使用命令行:
libvirtd --daemon --listen --config /etc/libvirt/libvirtd.conf
查看运行进程
[root@ddd run]# ps aux | grep libvirtd
root 16563 1.5 0.1 925880 7056 ? Sl 16:01 0:28 libvirtd -d -l --config /etc/libvirt/libvirtd.conf
查看端口
[root@ddd run]# netstat -apn | grep tcp
测试：
[root@localhost libvirt]# virsh -c qemu+tcp://192.168.10.246/system
欢迎使用 virsh，虚拟化的交互式终端。
输入：‘help’ 来获得命令的帮助信息
‘quit’ 退出
virsh # quit

错误问题解决方案：
1、Cannot recv data: Host key verification failed.: Connection reset by peer
这个错误是因为 nginx没有家目录导致的
[root@localhost /]# cd /home/
[root@localhost home]# mkdir nginx
[root@localhost home]# chown nginx.nginx nginx/
[root@localhost home]# chmod 700 nginx/ -R
[root@localhost home]# su - nginx -s /bin/bash
-bash-4.2$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/var/lib/nginx/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /var/lib/nginx/.ssh/id_rsa.
Your public key has been saved in /var/lib/nginx/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:5M4ulBY1hCAqZHif4+yeEzVy66b92nLHQE88u08G5TI [e-mail]
The key’s randomart image is:
±–[RSA 2048]----+
|.o. … o. |
|+… . o |
|o. . . …o . |
|. = =o. +o |
| o = SoEo. |
| + =o. o+ |
| . = oo .o |
| oo=o. +o |
| .++o+. … |
±—[SHA256]-----+
-bash-4.2$ touch ~/.ssh/config && echo -e “StrictHostKeyChecking=no\nUserKnownHostsFile=/dev/null” >> ~/.ssh/config
-bash-4.2$ chmod 0600 ~/.ssh/config

将nginx用户的ssh-key上传到kvm服务器上（这里kvm和WebVirtMgr部署在同一台机器上）
[root@localhost home]# su - nginx -s /bin/bash
上一次登录：五 12月 6 15:47:09 +07 2019pts/1 上
-bash-4.2$ ssh-copy-id root@192.168.10.246
/bin/ssh-copy-id: INFO: Source of key(s) to be installed: “/var/lib/nginx/.ssh/id_rsa.pub”
/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed – if you are prompted now it is to install the new keys
Warning: Permanently added ‘192.168.10.246’ (ECDSA) to the list of known hosts.
root@192.168.10.246’s password:

Number of key(s) added: 1

Now try logging into the machine, with: “ssh ‘root@192.168.10.246’”
and check to make sure that only the key(s) you wanted were added.

这里因为是本机SSH到本机，所以需要在nginx用户下生成公钥
[root@openstack ops]# cd /home/
[root@openstack home]# mkdir nginx
[root@openstack home]# chown nginx.nginx nginx/
[root@openstack home]# chmod 700 nginx/ -R

我的nginx家目录很奇怪在/var/lib/nginx
先登录到nginx用户下：
su - nginx -s /bin/bash
生成密钥：
-bash-4.2$ pwd
/var/lib/nginx
-bash-4.2$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/var/lib/nginx/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /var/lib/nginx/.ssh/id_rsa.
Your public key has been saved in /var/lib/nginx/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:zEgRR/YEEtFAChXWmi+gq4MnylDqafpmI9SqlT6MRmo [e-mail]
The key’s randomart image is:
±–[RSA 2048]----+
| …++OB=… |
| o …=.o |
| .o. . |
| . o. + |
| .o. … S |
|.= o. . |
|B++ . |
|%EO |
|&%oo |
±—[SHA256]-----+

上传密钥到要登陆的服务器上，并指定服务器上要用这个用户登录的用户
我这里要用root用户免密登录：
-bash-4.2$ ssh-copy-id root@192.168.10.246
/bin/ssh-copy-id: INFO: Source of key(s) to be installed: “/var/lib/nginx/.ssh/id_rsa.pub”
/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed – if you are prompted now it is to install the new keys
root@192.168.10.246’s password:

Number of key(s) added: 1

Now try logging into the machine, with: “ssh ‘root@192.168.10.246’”
and check to make sure that only the key(s) you wanted were added.

这样你在服务器root用户家目录下：
[root@localhost .ssh]# more authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC82YDbLwuuzy7Gqs8G043M91YAl08RiIrAod475BaoptiD19TX9WQKP+a11Cd1tYtnzIcCS1pOIoYA6dgekP3b40PiQjt9ChPTndZVyOOKaSa4YmXdn0f5h6FLKp8BIBHTAMwyn/ABi7smDV8t/0uCa8ze7gjk64xicJyAumGWmy3vE688IyzZBmc1ul6u6TA8l272
g+FxGCLd7ALj5BNbNMDqc/r3XuycEMO37UtgsaxhuhanC70NlgFyGf7u+6upNN1JT3wLOvhseKDpndyMXhGLUdLF6AufK9/5SLP1i4GRagjG6pRiUGdCn2oDBY7RJ08OVov5dh+wgwukDQS3 [e-mail]
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg50T7lhKwSbYeo+kBlEBlqWvrtvCBqznQaWssHqtHeuc7IYO3hAqHalVq2c1gF3cLKXOwVHQTu53bCejILolGjKjbpnMlgRoFp6Ryo8K4kL+nwu1huW8qnkXWNU1CdXs2ddWM/CrXcK7feWroPPPE4THlagCLFEYjNFf8eazIzsV/FJKxZ3UpiVSkjWdOM4u6NIxA
gDbQlbxLLTMhwu8OtGFXap1a18Ss0YfsBs/Ovrd0snyiU4CaujESH1aATPfbQsbRecmFQ5RVegz12WSxUjG472siHke561MKajKPWO1+8ag7zSDbMdGZIKXFbKQaMoGXtWQZwlDk6n/JdNE1 [e-mail]

逻辑就是客户端用id_rsa和id_rsa.pub在登录服务器时会这个文件做校验，你在复制这个公钥时必须复制你要登录的这个用户下的"username"/.ssh/目录下，公钥会追加在这个authorized_keys 文件里面，这样登录时就可以免密了

关于授权的问题，我在安装libvirt之后没有vim /etc/polkit-1/localauthority/50-local.d/50-libvirt-remote-access.pkla这个文件，后来我新建一个文件，把如下配置拷贝进去：
[Remote libvirt SSH access]
Identity=unix-user:root #注意这里采用的是root用户
Action=org.libvirt.unix.manage
ResultAny=yes
ResultInactive=yes
ResultActive=yes
但是具体是否有用，也不知道，
完了之后授权：chown -R root.root /etc/polkit-1/localauthority/50-local.d/50-libvirt-remote-access.pkla