ubuntu18 LXD 体验

ubuntu18 安装lxd

Update ubuntu

apt update -y && apt upgrade -y 

Add new user

adduser <user>

例如：

root@ubuntu-18:~# adduser fang
Adding user `fang' ...
Adding new group `fang' (1001) ...
Adding new user `fang' (1001) with group `fang' ...
Creating home directory `/home/fang' ...
Copying files from `/etc/skel' ...
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Changing the user information for fang

apt remove LXD & add the LXD Group

$ apt remove --purge lxd lxd-client

$ groupadd --system lxd

$ usermod -G lxd -a <user>

Update the permessions

newgrp lxd

install LXD

snap install lxd

or

snap install lxd --channel=3.0/stable

Install zfsutils-linux

sudo apt install zfsutils-linux

LXD initialization process

sudo lxd init

例如：

root@ubuntu-18:~# sudo lxd init
Would you like to use LXD clustering? (yes/no) [default=no]:
Do you want to configure a new storage pool? (yes/no) [default=yes]:
Name of the new storage pool [default=default]:
Name of the storage backend to use (btrfs, dir, lvm, zfs, ceph) [default=zfs]:
Create a new ZFS pool? (yes/no) [default=yes]:
Would you like to use an existing empty block device (e.g. a disk or partition)? (yes/no) [default=no]:
Size in GB of the new loop device (1GB minimum) [default=30GB]:
Would you like to connect to a MAAS server? (yes/no) [default=no]:
Would you like to create a new local network bridge? (yes/no) [default=yes]:
What should the new bridge be called? [default=lxdbr0]:
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
Would you like the LXD server to be available over the network? (yes/no) [default=no]:
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]:

LXD 常用配置

lanunch instance

lxc launch imageserver:imagename   instancename

例如：
fang@ubuntu-18:~$ lxc launch images:centos/7/amd64 centos7
Creating centos7
Starting centos7

fang@ubuntu-18:~$ lxc launch ubuntu:18.04 CTNAME
Creating CTNAME
Starting CTNAME

fang@ubuntu-18:~$ lxc image list
+-------+--------------+--------+---------------------------------------------+--------------+-----------+----------+-----------------------------+
| ALIAS | FINGERPRINT  | PUBLIC |                 DESCRIPTION                 | ARCHITECTURE |   TYPE    |   SIZE   |         UPLOAD DATE         |
+-------+--------------+--------+---------------------------------------------+--------------+-----------+----------+-----------------------------+
|       | 5a4a2ba32e1f | no     | Centos 7 amd64 (20210831_07:08)             | x86_64       | CONTAINER | 84.44MB  | Sep 1, 2021 at 6:49am (UTC) |
+-------+--------------+--------+---------------------------------------------+--------------+-----------+----------+-----------------------------+
|       | a7c9fc967dea | no     | ubuntu 18.04 LTS amd64 (release) (20210825) | x86_64       | CONTAINER | 193.78MB | Sep 1, 2021 at 6:01am (UTC) |
+-------+--------------+--------+---------------------------------------------+--------------+-----------+----------+-----------------------------+

进入instance

lxc exec centos7 -- /bin/bash

例如:
fang@ubuntu-18:~$ lxc exec centos7 -- /bin/bash
[root@centos7 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
16: eth0@if17: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:b2:89:9b brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.229.20.232/24 brd 10.229.20.255 scope global dynamic eth0
       valid_lft 3337sec preferred_lft 3337sec
    inet6 fd42:621a:e06c:52ee:216:3eff:feb2:899b/64 scope global mngtmpaddr dynamic
       valid_lft 3401sec preferred_lft 3401sec
    inet6 fe80::216:3eff:feb2:899b/64 scope link
       valid_lft forever preferred_lft forever
[root@centos7 ~]# cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)


fang@ubuntu-18:~$ lxc exec CTNAME -- lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.5 LTS
Release:        18.04
Codename:       bionic
fang@ubuntu-18:~$ lxc exec CTNAME -- ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
14: eth0@if15: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:15:8f:00 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.229.20.83/24 brd 10.229.20.255 scope global dynamic eth0
       valid_lft 2075sec preferred_lft 2075sec
    inet6 fd42:621a:e06c:52ee:216:3eff:fe15:8f00/64 scope global dynamic mngtmpaddr noprefixroute
       valid_lft 3246sec preferred_lft 3246sec
    inet6 fe80::216:3eff:fe15:8f00/64 scope link
       valid_lft forever preferred_lft forever

Limit cpu cores

lxc config set centos7 limits.cpu  4


例如: 没限制之前instance资源会直接公用宿主机
fang@ubuntu-18:~$ cat /proc/cpuinfo| grep "processor"| wc -l
80
fang@ubuntu-18:~$
fang@ubuntu-18:~$
fang@ubuntu-18:~$ lxc exec centos7 --  cat /proc/cpuinfo| grep "processor"| wc -l
80

    添加限制
fang@ubuntu-18:~$ lxc config set centos7 limits.cpu 4
fang@ubuntu-18:~$
fang@ubuntu-18:~$ lxc exec centos7 --  cat /proc/cpuinfo| grep "processor"| wc -l
4

Limit cpu usage(max%)

lxc config set xxx  limits.cpu.allowance  10%

例如：
fang@ubuntu-18:~$ lxc exec CTNAME --  cat /proc/cpuinfo| grep "processor"| wc -l
80
fang@ubuntu-18:~$
fang@ubuntu-18:~$ lxc config set CTNAME limits.cpu.allowance 10%
fang@ubuntu-18:~$
fang@ubuntu-18:~$
fang@ubuntu-18:~$ lxc exec CTNAME --  cat /proc/cpuinfo| grep "processor"| wc -l
80

#######会发现还是宿主机的总数，这只有在负载状态才会这样！！！

limit memory/RAM

lxc config set xxx limits.memory 

例如：
fang@ubuntu-18:~$  lxc exec centos7 --  free -g
              total        used        free      shared  buff/cache   available
Mem:            376           0         376           0           0         376
Swap:             0           0           0
fang@ubuntu-18:~$ lxc config set centos7 limits.memory 9192MB
fang@ubuntu-18:~$
fang@ubuntu-18:~$
fang@ubuntu-18:~$ lxc exec centos7 --  free -g
              total        used        free      shared  buff/cache   available
Mem:              8           0           8           0           0           8
Swap:             0           0           0

#####内存限制默认是‘hard’ ，  还可以设置‘soft’

set disksize

lxc config device add xxx root disk path=/ pool=default  size=50GB

例如：
fang@ubuntu-18:~$ lxc exec centos7 --  df -hT
Filesystem                 Type      Size  Used Avail Use% Mounted on
default/containers/centos7 zfs        27G  208M   27G   1% /
none                       tmpfs     492K  4.0K  488K   1% /dev
udev                       devtmpfs  189G     0  189G   0% /dev/tty
tmpfs                      tmpfs     100K     0  100K   0% /dev/lxd
tmpfs                      tmpfs     100K     0  100K   0% /dev/.lxd-mounts
tmpfs                      tmpfs     189G     0  189G   0% /dev/shm
tmpfs                      tmpfs     189G  8.4M  189G   1% /run
tmpfs                      tmpfs     189G     0  189G   0% /sys/fs/cgroup
fang@ubuntu-18:~$ lxc config device add centos7 root disk path=/ pool=default size=50GB
Device root added to centos7
fang@ubuntu-18:~$
fang@ubuntu-18:~$ lxc exec centos7 --  df -hT
Filesystem                 Type      Size  Used Avail Use% Mounted on
default/containers/centos7 zfs        27G  208M   27G   1% /
none                       tmpfs     492K  4.0K  488K   1% /dev
udev                       devtmpfs  189G     0  189G   0% /dev/tty
tmpfs                      tmpfs     100K     0  100K   0% /dev/lxd
tmpfs                      tmpfs     100K     0  100K   0% /dev/.lxd-mounts
tmpfs                      tmpfs     189G     0  189G   0% /dev/shm
tmpfs                      tmpfs     189G  8.4M  189G   1% /run
tmpfs                      tmpfs     189G     0  189G   0% /sys/fs/cgroup

####### 发现instance，并没有扩大，
####查看 配置信息   lxc config show --expanded  xxx
fang@ubuntu-18:~$ lxc config show --expanded centos7
architecture: x86_64
config:
  image.architecture: amd64
  image.description: Centos 7 amd64 (20210831_07:08)
  image.os: Centos
  image.release: "7"
  image.serial: "20210831_07:08"
  image.type: squashfs
  image.variant: default
  limits.cpu: "4"
  limits.memory: 9192MB
  volatile.base_image: 5a4a2ba32e1fcf1db0d78c563c68bb370ca5b3d620955d1488a6aaf9e304fa64
  volatile.eth0.host_name: veth70bf358d
  volatile.eth0.hwaddr: 00:16:3e:b2:89:9b
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.power: RUNNING
  volatile.uuid: 26f40cd2-85de-4d2c-abdf-f42941831119
devices:
  eth0:
    name: eth0
    network: lxdbr0
    type: nic
  root:
    path: /
    pool: default
    size: 50GB
    type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""

####配置已生效，思考， 是否是pool池问题！

fang@ubuntu-18:~$ zpool list
NAME      SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
default  27.8G   602M  27.2G         -     0%     2%  1.00x  ONLINE  -

####考虑扩大pool
方法一：

root@ubuntu-18:~# zpool set autoexpand=on default
root@ubuntu-18:~#
root@ubuntu-18:~# zpool get autoexpand default
NAME     PROPERTY    VALUE   SOURCE
default  autoexpand  on      local

方法二：

Set disksize for ALL containers

lxc profile device set default root size XXXGB

Limit Disk I/O (40MB read / 20MB write)

lxc config device set XXX  root limits.read  40MB

lxc config device set XXX root limits.write  20MB

查看配置

lxc config show --expanded XXX

例如：
root@ubuntu-18:~# lxc config show --expanded centos7
architecture: x86_64
config:
  image.architecture: amd64
  image.description: Centos 7 amd64 (20210901_07:08)
  image.os: Centos
  image.release: "7"
  image.serial: "20210901_07:08"
  volatile.base_image: 93e5ef398fc99d396573adbabbb46b69bee8b2146a7968c484d700865606a2f5
  volatile.eth0.host_name: veth3Y6G8W
  volatile.eth0.hwaddr: 00:16:3e:d7:f4:93
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.power: RUNNING
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: lxdbr0
    type: nic
  root:
    limits.read: 40MB
    limits.write: 20MB
    path: /
    pool: default
    size: 100GB
    type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""

LXC ----network ----bridge

宿主机(ubuntu18)桥接网卡配置：

root@ubuntu-18:~# cat /etc/netplan/50-cloud-init.yaml
network:
    ethernets:
        enp95s0f0:
            dhcp4: false
        enp95s0f1:
            dhcp4: false
    bridges:
      br0:
        interfaces: [enp95s0f0]
        dhcp4: no
        addresses: [192.168.150.3/24]
        gateway4: 192.168.150.254
        nameservers:
          addresses: [114.114.114.114]
        parameters:
                stp: true
                forward-delay: 4
    version: 2

默认container 网络模式：

lxc list

##
root@ubuntu-18:~# lxc list
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+
|  NAME  |  STATE  |       IPV4        |                     IPV6                      |    TYPE    | SNAPSHOTS |
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+
| ubuntu | RUNNING | 10.83.7.44 (eth0) | fd42:1db8:daf6:9bd9:216:3eff:fe26:7436 (eth0) | PERSISTENT | 0         |
+--------+---------+-------------------+-----------------------------------------------+------------+-----------+

attach the container to the bridge

lxc config device add ubuntu  eth0 nic nictype=bridged parent=br0 name=eth0

##
root@ubuntu-18:~# lxc list
+--------+---------+------+------+------------+-----------+
|  NAME  |  STATE  | IPV4 | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+------+------+------------+-----------+
| ubuntu | RUNNING |      |      | PERSISTENT | 0         |
+--------+---------+------+------+------------+-----------+

## 进入container
root@ubuntu-18:~# lxc exec ubuntu bash
root@ubuntu:~#

root@ubuntu:~#
root@ubuntu:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
23: eth0@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:26:74:36 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::216:3eff:fe26:7436/64 scope link
       valid_lft forever preferred_lft forever

## 配置网络
root@ubuntu:~# cat /etc/netplan/50-cloud-init.yaml
# This file is generated from information provided by the datasource.  Changes
# to it will not persist across an instance reboot.  To disable cloud-init's
# network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    version: 2
    ethernets:
        eth0:
            dhcp4: false
            addresses:
            -  192.168.150.103/24
            gateway4: 192.168.150.254
            nameservers:
              addresses:
              - 114.114.114.114

root@ubuntu:~# netplan apply
root@ubuntu:~#
root@ubuntu:~#
root@ubuntu:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
23: eth0@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:26:74:36 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.150.103/24 brd 192.168.150.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe26:7436/64 scope link
       valid_lft forever preferred_lft forever


## exit 
root@ubuntu-18:~# lxc list
+--------+---------+------------------------+------+------------+-----------+
|  NAME  |  STATE  |          IPV4          | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+------------------------+------+------------+-----------+
| ubuntu | RUNNING | 192.168.150.103 (eth0) |      | PERSISTENT | 0         |
+--------+---------+------------------------+------+------------+-----------+

这里有借鉴：
https://thenewstack.io/how-to-create-a-bridged-network-for-lxd-containers/