etcd

穆洋
2023-12-01
  • 程序启动的用户                    etcd
  • 程序systemd service文件     见下面
  • 主目录                                   etcd没有这个
  • 主目录/二进制                       三个二进制放在/usr/local/etcd
  • 主目录/配置                          配置在/etc/etcd.conf.yml         
  • 主目录/数据                          /var/lib/etcd
  • 主目录/日志                         
  • 动态库或插件

install:

1,安装脚本: (安装脚本可与2配置文件一起使用,做自动化静态集群部署)

这里有一个问题是,如果直接用etcd用户启动etcd,在systemctl start etcd时,是没有权限创建/var/lib/etcd下面的各种数据文件的。

所以,这里先用root启动起来,文件创建出来。然后停掉etcd service。再然后更改/var/lib/etcd的权限 chown -R etcd:etcd /var/lib/etcd,在编辑

/usr/lib/systemd/system/etcd.service 加入User=etcd,Group=etcd ,再启动etcd service。

#!/bin/sh

groupadd etcd
useradd -r -g etcd -s /sbin/nologin etcd
wget https://mirrors.huaweicloud.com/etcd/v3.5.1/etcd-v3.5.1-linux-amd64.tar.gz
tar -zxvf etcd-v3.5.1-linux-amd64.tar.gz
mkdir -p /usr/local/etcd
cd etcd-v3.5.1-linux-amd64
cp etcd etcdctl etcdutl /usr/local/etcd
chown -R etcd:etcd /usr/local/etcd

sudo cat >>/etc/yum.repos.d/nginx.repo<<EOF
[Unit]
Description=etcd process
After=network-online.target
Wants=network-online.target

[Service]

Type=simple
Restart=on-failure
ExecStart=/usr/local/etcd/etcd --config-file /etc/etcd.conf.yml

[Install]
WantedBy=multi-user.target
EOF

systemctl enable etcd
systemctl daemon-reload

sudo cp etcd.conf.yml /etc
systemctl start etcd
systemctl status etcd

systemctl stop etcd


chown -R etcd:etcd /var/lib/etcd
sudo cat >>/etc/yum.repos.d/nginx.repo<<EOF
[Unit]
Description=etcd process
After=network-online.target
Wants=network-online.target

[Service]

User=etcd
Group=etcd

Type=simple
Restart=on-failure
ExecStart=/usr/local/etcd/etcd --config-file /etc/etcd.conf.yml

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload

systemctl start etcd
systemctl status etcd

2,配置文件:

# This is the configuration file for the etcd server.

# Human-readable name for this member.
name: 'etcd1'

# Path to the data directory.
data-dir: /var/lib/etcd

# Path to the dedicated wal directory.
wal-dir: /var/lib/etcd

# Number of committed transactions to trigger a snapshot to disk.
snapshot-count: 10000

# Time (in milliseconds) of a heartbeat interval.
heartbeat-interval: 100

# Time (in milliseconds) for an election to timeout.
election-timeout: 1000

# Raise alarms when backend size exceeds the given quota. 0 means use the
# default quota.
quota-backend-bytes: 0

# List of comma separated URLs to listen on for peer traffic.
listen-peer-urls: http://192.168.:2380

# List of comma separated URLs to listen on for client traffic.
listen-client-urls: http://192.168:2379

# Maximum number of snapshot files to retain (0 is unlimited).
max-snapshots: 5

# Maximum number of wal files to retain (0 is unlimited).
max-wals: 5

# Comma-separated white list of origins for CORS (cross-origin resource sharing).
cors:

# List of this member's peer URLs to advertise to the rest of the cluster.
# The URLs needed to be a comma-separated list.
initial-advertise-peer-urls: http://192.168:2380

# List of this member's client URLs to advertise to the public.
# The URLs needed to be a comma-separated list.
advertise-client-urls: http://192.168:2379

# Discovery URL used to bootstrap the cluster.
discovery:

# Valid values include 'exit', 'proxy'
discovery-fallback: 'proxy'

# HTTP proxy to use for traffic to discovery service.
discovery-proxy:

# DNS domain used to bootstrap initial cluster.
discovery-srv:

# Initial cluster configuration for bootstrapping.
initial-cluster: etcd1=http://192.168.1:2380,etcd2=http://192.16:2380,etcd3=http://192.168.:2380

# Initial cluster token for the etcd cluster during bootstrap.
initial-cluster-token: 'etcd-cluster1'

# Initial cluster state ('new' or 'existing').
initial-cluster-state: 'new'

# Reject reconfiguration requests that would cause quorum loss.
strict-reconfig-check: false

# Accept etcd V2 client requests
enable-v2: true

# Enable runtime profiling data via HTTP server
enable-pprof: true

# Valid values include 'on', 'readonly', 'off'
proxy: 'off'

# Time (in milliseconds) an endpoint will be held in a failed state.
proxy-failure-wait: 5000

# Time (in milliseconds) of the endpoints refresh interval.
proxy-refresh-interval: 30000

# Time (in milliseconds) for a dial to timeout.
proxy-dial-timeout: 1000

# Time (in milliseconds) for a write to timeout.
proxy-write-timeout: 5000

# Time (in milliseconds) for a read to timeout.
proxy-read-timeout: 0

client-transport-security:
# Path to the client server TLS cert file.
cert-file:

# Path to the client server TLS key file.
key-file:

# Enable client cert authentication.
client-cert-auth: false

# Path to the client server TLS trusted CA cert file.
trusted-ca-file:

# Client TLS using generated certificates
auto-tls: false

peer-transport-security:
# Path to the peer server TLS cert file.
cert-file:

# Path to the peer server TLS key file.
key-file:

# Enable peer client cert authentication.
client-cert-auth: false

# Path to the peer server TLS trusted CA cert file.
trusted-ca-file:

# Peer TLS using generated certificates.
auto-tls: false

# The validity period of the self-signed certificate, the unit is year.
self-signed-cert-validity: 1

# Enable debug-level logging for etcd.
log-level: debug

logger: zap

# Specify 'stdout' or 'stderr' to skip journald logging even when running under systemd.
log-outputs: [stderr]

# Force to create a new one member cluster.
force-new-cluster: false

auto-compaction-mode: periodic
auto-compaction-retention: "1"

解释说明:

创建etcd的用户和组:(创建service文件的时候需要使用)

groupadd etcd

useradd -r -g etcd -s /sbin/nologin etcd

wget https://github.com/etcd-io/etcd/releases/download/v3.5.1/etcd-v3.5.1-linux-amd64.tar.gz

或者国内华为镜像

wget https://mirrors.huaweicloud.com/etcd/v3.5.1/etcd-v3.5.1-linux-amd64.tar.gz

tar -zxvf etcd-v3.5.1-linux-amd64.tar.gz

mkdir -p  /usr/local/etcd

cd etcd-v3.5.1-linux-amd64

cp etcd etcdctl etcdutl /usr/local/etcd

 chown -R etcd:etcd /usr/local/etcd  (这里把二进制给与etcd用户?)
 chown -R etcd:etcd /var/lib/etcd (二进制程序给etcd用户和组,数据目录也给etcd用户和组)

vi /usr/lib/systemd/system/etcd.service

[Unit]
Description=etcd process
After=network-online.target
Wants=network-online.target

[Service]

User=etcd
Group=etcd

Type=simple
Restart=on-failure
ExecStart=/usr/local/etcd/etcd --config-file /etc/etcd.conf.yml

[Install]
WantedBy=multi-user.target

systemctl enable etcd

systemc service:

[Unit]

这里有问题,没有after网络
Description=etcd process

[Service]
Type=simple
Restart=on-failure
ExecStart=/usr/local/etcd/etcd --config-file /etc/etcd.conf.yml

[Install]
WantedBy=multi-user.target

systemctl daemon-reload

systemctl enable etcd

systemctl start etcd

systemctl status etcd

/usr/local/etcd/etcd --version

 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

使用:

etcdctl --endpoints=ip地址:2379 put greeting "Hello, etcd"

etcdctl --endpoints=IP地址:2379 get greeting

管理:

角色控制,就两步,一,root用户初始化。二,使用已初始化的root用户添加管理新用户

etcdctl  --endpoints=ip:2379   user list     列出用户,如果没设置会为空

etcdctl  --endpoints=ip:2379   member  list  列出集群成员 

etcdctl --endpoints=ip:2379    role list          列出角色 如果没设置会为空。

etcdctl --endpoints=:2379 role add root    添加一个root角色
1008 etcdctl --endpoints=:2379 role list    可以看到root角色列出来
1009 etcdctl --endpoints=:2379 user add root   添加一个root用户(这个必须最先做,要在开启角色控制之前,其实就是最开始的root用户初始化,和mysql一样)
1011 etcdctl --endpoints=:2379 user grant-role root(用户名) root(角色名) 给某个用户加某个角色(root用户必须给root角色)
1013 etcdctl --endpoints=:2379 auth status 查看访问控制状态,没开启会为false
1014 etcdctl --endpoints=:2379 auth enable 开启授权,访问控制
1016 etcdctl --endpoints=:2379 auth status  因为已经开启访问控制,没有指定用户名和密码,所有报错
1017 etcdctl --endpoints=:2379 user list      因为已经开启访问控制,没有指定用户名和密码,所有报错

etcdctl --endpoints=:2379 --user root user list   加了--user option,此时需要输入root密码,就可以看到用户列表

etcdctl  --help

etcdctl [flags],flags其实就是[options] [commands]

比如etcdctl --endpoints=ip:2379(--endpoints option) --user=root(--user options) user (etcdctl 的command)list(sub command,也就是user的command) 

 工作流:

角色控制,就两步,一,root用户初始化。二,使用已初始化的root用户添加管理新用户

/usr/local/etcd/etcdctl --endpoints=$ENDPOINTS  role add root 

user add root  

user grant-role root  root(角色名) 给某个用户加某个角色(root用户必须给root角色)

auth enable

--user root user list  

创建一个app用户,etcdctl --endpoints=:2379 user add app

一个app角色,role add app

并把app用户和app角色绑定:user grant-role root  root

在给app角色赋予以config开头的key的读写权限,

 etcdctl --user=root --endpoints=ip:2379  role grant-permission app --prefix=true readwrite /config/

 etcdctl  --endpoints=:2379  --user=root role get app   查看app角色被赋予了哪些权限。

运行时配置:

https://etcd.io/docs/v3.5/op-guide/runtime-configuration/

 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

集群:

几个url,两个节点间用的,两个对外客户端用的,这四个url都可以配置成逗号分割的列表形式:

节点间2380,客户端2379

--initial-advertise-peer-urls http://10.0.1.12:2380 \

--advertise-client-urls http://10.0.1.12:2379
--listen-peer-urls http://10.0.1.12:2380 \
--listen-client-urls http://10.0.1.12:2379,http://127.0.0.1:2379 \

节点间服务发现url:initial-advertise-peer-urls。做静态集群初始化配置–initial-cluster 时需要使用:

节点间心跳,数据同步url:listen-peer-urls

因为etcd是无主对等分布式架构,所以有集群对外提供服务的监听url集合:advertise-client-urls

单机监听客户端url:listen-client-urls

集群配置:
https://etcd.io/docs/v3.5/op-guide/clustering/

–name

  • Human-readable name for this member.
  • default: “default”
  • env variable: ETCD_NAME
  • This value is referenced as this node’s own entries listed in the --initial-cluster flag (e.g., default=http://localhost:2380). This needs to match the key used in the flag if using static bootstrapping. When using discovery, each member must have a unique name. Hostname or machine-id can be a good choice
  • 名字标识或者说节点名,当使用静态方式搭建集群时,做为key

–initial-cluster

  • Initial cluster configuration for bootstrapping.
  • default: “default=http://localhost:2380”
  • env variable: ETCD_INITIAL_CLUSTER
  • The key is the value of the --name flag for each node provided. The default uses default for the key because this is the default for the --name flag
初始集群配置,这里的url要注意,Note that the URLs specified in  initial-cluster are the  advertised peer URLs, i.e. they should match the value of  initial-advertise-peer-urls on the respective nodes.,是initial-advertise-peer-urls节点间服务发现url
集群标识:
initial-cluster-token  
etcd listens on  listen-client-urls to accept client traffic。Multiple URLs may be used to specify a number of addresses and ports to listen on. The etcd will respond to requests from any of the listed addresses and ports.
搭建三节点etcd集群:

On each machine, start etcd with these flags:(也可以分别在配置文件里指定)

$ etcd --name infra0(节点名) --initial-advertise-peer-urls http://10.0.1.10:2380(广播服务发现节点地址) \
  --listen-peer-urls http://10.0.1.10:2380(节点间心跳数据同步等地址) \
  --listen-client-urls http://10.0.1.10:2379,http://127.0.0.1:2379() \
  --advertise-client-urls http://10.0.1.10:2379(集群整体对外提供监听客户端地址列表广播客户端地址) \ 
--initial-cluster-token etcd-cluster-1(集群id名) \ --initial-cluster infra0=http://10.0.1.10:2380,infra1=http://10.0.1.11:2380,infra2=http://10.0.1.12:2380(集群静态成员列表) \
--initial-cluster-state new (初始化集群,Initial cluster state (“new” or “existing”). Set to new for all members present during initial static or DNS bootstrapping.)
$ etcd --name infra1 --initial-advertise-peer-urls http://10.0.1.11:2380 \
  --listen-peer-urls http://10.0.1.11:2380 \
  --listen-client-urls http://10.0.1.11:2379,http://127.0.0.1:2379 \
  --advertise-client-urls http://10.0.1.11:2379 \
  --initial-cluster-token etcd-cluster-1 \
  --initial-cluster infra0=http://10.0.1.10:2380,infra1=http://10.0.1.11:2380,infra2=http://10.0.1.12:2380 \
  --initial-cluster-state new
$ etcd --name infra2 --initial-advertise-peer-urls http://10.0.1.12:2380 \
  --listen-peer-urls http://10.0.1.12:2380 \
  --listen-client-urls http://10.0.1.12:2379,http://127.0.0.1:2379 \
  --advertise-client-urls http://10.0.1.12:2379 \
  --initial-cluster-token etcd-cluster-1 \
  --initial-cluster infra0=http://10.0.1.10:2380,infra1=http://10.0.1.11:2380,infra2=http://10.0.1.12:2380 \
  --initial-cluster-state new
查看集群成员:
1,   vi /etc/profile  配置etcd环境变量

#etcd
export etcd_host1=192.168.
export etcd_host2=192.168.6
export etcd_host3=192.168.7
ENDPOINTS=$etcd_host1:2379,$etcd_host2:2379,$etcd_host3:2379

source /etc/profile

2,放通防火墙:

ansible etcd -m shell -a "firewall-cmd --zone=public --add-port=2380/tcp --permanent"

ansible etcd -m shell -a "firewall-cmd --zone=public --add-port=2379/tcp --permanent"

ansible etcd -m shell -a " firewall-cmd --reload"

etcdctl --endpoints=$ENDPOINTS member list
etcdctl --endpoints=$ENDPOINTS endpoint status --write-out="table"

etcdctl --endpoints=$ENDPOINTS endpoint health --write-out="table"

节点上下线:
etcdctl --endpoints=$ENDPOINTS member add|remove



运行时配置:
比如集群节点上下线
https://etcd.io/docs/v3.5/op-guide/runtime-configuration/

-------------------------------------------------------------------------------------------------------------------------------------------
etcd配置文件:
配置文件参考:https://github.com/etcd-io/etcd/blob/release-3.4/etcd.conf.yml.sample
  • -advertise-client-urls
    • 就是客户端(etcdctl/curl等)跟etcd服务进行交互时请求的url
  • --listen-client-urls
    • 这个参数是etcd服务器自己监听时用的,也就是说,监听本机上的哪个网卡,哪个端口

–log-level  日志级别

  • Configures log level. Only supports debug, info, warn, error, panic, or fatal.
  • default: info
/etc/etcd.conf
# This is the configuration file for the etcd server.
  
 # Human-readable name for this member.
 name: 'default'
  
 # Path to the data directory.
 data-dir:
  
 # Path to the dedicated wal directory.
 wal-dir:
  
 # Number of committed transactions to trigger a snapshot to disk.
 snapshot-count: 10000
  
 # Time (in milliseconds) of a heartbeat interval.
 heartbeat-interval: 100
  
 # Time (in milliseconds) for an election to timeout.
 election-timeout: 1000
  
 # Raise alarms when backend size exceeds the given quota. 0 means use the
 # default quota.
 quota-backend-bytes: 0
  
 # List of comma separated URLs to listen on for peer traffic.
 listen-peer-urls: http://localhost:2380
  
 # List of comma separated URLs to listen on for client traffic.
 listen-client-urls: http://localhost:2379
  
 # Maximum number of snapshot files to retain (0 is unlimited).
 max-snapshots: 5
  
 # Maximum number of wal files to retain (0 is unlimited).
 max-wals: 5
  
 # Comma-separated white list of origins for CORS (cross-origin resource sharing).
 cors:
  
 # List of this member's peer URLs to advertise to the rest of the cluster.
 # The URLs needed to be a comma-separated list.
 initial-advertise-peer-urls: http://localhost:2380
  
 # List of this member's client URLs to advertise to the public.
 # The URLs needed to be a comma-separated list.
 advertise-client-urls: http://localhost:2379
  
 # Discovery URL used to bootstrap the cluster.
 discovery:
  
 # Valid values include 'exit', 'proxy'
 discovery-fallback: 'proxy'
  
 # HTTP proxy to use for traffic to discovery service.
 discovery-proxy:
  
 # DNS domain used to bootstrap initial cluster.
 discovery-srv:
  
 # Initial cluster configuration for bootstrapping.
 initial-cluster:
  
 # Initial cluster token for the etcd cluster during bootstrap.
 initial-cluster-token: 'etcd-cluster'
  
 # Initial cluster state ('new' or 'existing').
 initial-cluster-state: 'new'
  
 # Reject reconfiguration requests that would cause quorum loss.
 strict-reconfig-check: false
  
 # Accept etcd V2 client requests
 enable-v2: true
  
 # Enable runtime profiling data via HTTP server
 enable-pprof: true
  
 # Valid values include 'on', 'readonly', 'off'
 proxy: 'off'
  
 # Time (in milliseconds) an endpoint will be held in a failed state.
 proxy-failure-wait: 5000
  
 # Time (in milliseconds) of the endpoints refresh interval.
 proxy-refresh-interval: 30000
  
 # Time (in milliseconds) for a dial to timeout.
 proxy-dial-timeout: 1000
  
 # Time (in milliseconds) for a write to timeout.
 proxy-write-timeout: 5000
  
 # Time (in milliseconds) for a read to timeout.
 proxy-read-timeout: 0
  
 client-transport-security:
 # Path to the client server TLS cert file.
 cert-file:
  
 # Path to the client server TLS key file.
 key-file:
  
 # Enable client cert authentication.
 client-cert-auth: false
  
 # Path to the client server TLS trusted CA cert file.
 trusted-ca-file:
  
 # Client TLS using generated certificates
 auto-tls: false
  
 peer-transport-security:
 # Path to the peer server TLS cert file.
 cert-file:
  
 # Path to the peer server TLS key file.
 key-file:
  
 # Enable peer client cert authentication.
 client-cert-auth: false
  
 # Path to the peer server TLS trusted CA cert file.
 trusted-ca-file:
  
 # Peer TLS using generated certificates.
 auto-tls: false
  
 # The validity period of the self-signed certificate, the unit is year.
 self-signed-cert-validity: 1
  
 # Enable debug-level logging for etcd.
 log-level: debug
  
 logger: zap
  
 # Specify 'stdout' or 'stderr' to skip journald logging even when running under systemd.
 log-outputs: [stderr]
  
 # Force to create a new one member cluster.
 force-new-cluster: false
  
 auto-compaction-mode: periodic
 

auto-compaction-retention: "1"

etcd --config-file  /etc/etcd.conf

 ------------------------------------------------------------------------

使用:

etcdctl --endpoints=ip地址:2379 put greeting "Hello, etcd"

etcdctl --endpoints=IP地址:2379 get greeting

 类似资料:

相关阅读

相关文章

相关问答

相关文档