您的位置:首页 > 运维架构 > 网站架构

跟着炎炎盐实践k8s---Kubernetes 1.16.10 二进制高可用集群部署之ETCD部署

2020-07-01 16:57 1181 查看
这一节我们在三个主节点部署高可用的etcd集群,官方建议5节点或者7节点,我们三节点也是够用的。我们开始吧!
  • 一、环境准备

    10.13.33.29  etcd-0
    10.13.33.40  etcd-1
    10.13.33.38  etcd-2

    etcd 数据目录:

    ETCD_DATA_DIR="/data/k8s/etcd/data"

    etcd WAL 目录:

    ETCD_WAL_DIR="/data/k8s/etcd/wal"

    (建议是 SSD 磁盘分区,或者和 ETCD_DATA_DIR 不同的磁盘分区)

  • 二、部署安装ETCD

1、创建etcd相关证书

cd /opt/k8s/work
cat > etcd-csr.json <<EOF
{
"CN": "etcd",
"hosts": [
"127.0.0.1",
"10.13.33.38",
"10.13.33.29",
"10.13.33.40"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "BeiJing",
"L": "BeiJing",
"O": "k8s",
"OU": "4Paradigm"
}
]
}
EOF
###host字段指定授权使用该证书的etcd节点IP或子网列表,需要将etcd集群的3个节点都添加其中

###生成证书和私钥

cfssl gencert -ca=/opt/k8s/work/ca.pem \
-ca-key=/opt/k8s/work/ca-key.pem \
-config=/opt/k8s/work/ca-config.json \
-profile=kubernetes etcd-csr.json | cfssljson -bare etcd
ls etcd*pem

###分发到etcd节点

mkdir -p /etc/etcd/cert
cp etcd*.pem /etc/etcd/cert/

2、安装etcd

wget https://github.com/etcd-io/etcd/releases/download/v3.3.13/etcd-v3.3.13-linux-amd64.tar.gz         ##下载etcd组件
tar -xvf etcd-v3.3.13-linux-amd64.tar.gz                                                                                                  ##解压
cp etcd-v3.3.13-linux-amd64/etcd* /opt/k8s/bin/                                                                                     ##复制etcd命令到指定目录
chmod +x /opt/k8s/bin/*                                                                                                                          ##赋权
mkdir -p /data/k8s/etcd/data /data/k8s/etcd/wal                                                                                     ##创建etcd数据目录

3、创建etcd.service

cat > etcd.service.template <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.
109e
target
Wants=network-online.target
Documentation=https://github.com/coreos
[Service]
Type=notify
WorkingDirectory=/data/k8s/etcd/data
ExecStart=/opt/k8s/bin/etcd \
--data-dir '/data/k8s/etcd/data' \
--wal-dir '/data/k8s/etcd/wal' \
--name 'master-01' \                                                                                           ###根据etcd节点命名
--cert-file '/etc/etcd/cert/etcd.pem' \
--key-file '/etc/etcd/cert/etcd-key.pem' \
--trusted-ca-file '/etc/kubernetes/cert/ca.pem' \
--peer-cert-file '/etc/etcd/cert/etcd.pem' \
--peer-key-file '/etc/etcd/cert/etcd-key.pem' \
--peer-trusted-ca-file '/etc/kubernetes/cert/ca.pem' \
--peer-client-cert-auth  \
--client-cert-auth  \
--listen-peer-urls 'https://10.13.33.38:2380' \
--initial-advertise-peer-urls 'https://10.13.33.38:2380' \
--listen-client-urls 'https://10.13.33.38:2379,http://127.0.0.1:2379' \
--advertise-client-urls 'https://10.13.33.38:2379' \
--initial-cluster-token 'etcd-cluster-0' \
--initial-cluster 'master-03=https://10.13.33.40:2380,master-01=https://10.13.33.38:2380,master-02=https://10.13.33.29:2380' \                         ###各个etcd节点hostname及地址
--initial-cluster-state 'new' \
--auto-compaction-mode 'periodic' \
--auto-compaction-retention '1' \
--max-request-bytes '33554432' \
--quota-backend-bytes '6442450944' \
--heartbeat-interval '250' \
--election-timeout '2000'
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF

3、启动etcd

systemctl daemon-reload && systemctl enable etcd && systemctl restart etcd && systemctl status etcd|grep Active
curl -L 127.0.0.1:2379/health          ###检查启动状态

4、确认集群健康

[root@master-01 work]# curl -L 127.0.0.1:2379/health
{"health":"true"}
[root@master-01 work]# ETCDCTL_API=3 /opt/k8s/bin/etcdctl   -w table --cacert=/etc/kubernetes/cert/ca.pem   --cert=/etc/etcd/cert/etcd.pem   --key=/etc/etcd/cert/etcd-key.pem   --endpoints=https://10.13.33.29:2379,https://10.13.33.38:2379,https://10.13.33.40:2379  endpoint status
+--------------------------+------------------+---------+---------+-----------+-----------+------------+
|         ENDPOINT         |        ID        | VERSION | DB SIZE | IS LEADER | RAFT TERM | RAFT INDEX |
+--------------------------+------------------+---------+---------+-----------+-----------+------------+
| https://10.13.33.29:2379 | 3d57cc849d2a16** |  3.3.13 |   33 MB |      true |     36971 |    6201481 |
| https://10.13.33.38:2379 | bff9394ca77d77** |  3.3.13 |   33 MB |     false |     36971 |    6201481 |
| https://10.13.33.40:2379 | 2ce7325ad513f9** |  3.3.13 |   33 MB |     false |     36971 |    6201481 |
+--------------------------+------------------+---------+---------+-----------+-----------+------------+
[root@master-01 work]# ETCDCTL_API=3 /opt/k8s/bin/etcdctl   -w table --cacert=/etc/kubernetes/cert/ca.pem   --cert=/etc/etcd/cert/etcd.pem   --key=/etc/etcd/cert/etcd-key.pem   --endpoints=https://10.13.33.29:2379,https://10.13.33.38:2379,https://10.13.33.40:2379  member list
+------------------+---------+-----------+--------------------------+--------------------------+
|        ID        | STATUS  |   NAME    |        PEER ADDRS        |       CLIENT ADDRS       |
+------------------+---------+-----------+--------------------------+--------------------------+
| 2ce7325ad513f9** | started | master-03 | https://10.13.33.40:2380 | https://10.13.33.40:2379 |
| 3d57cc849d2a16** | started | master-02 | https://10.13.33.29:2380 | https://10.13.33.29:2379 |
| bff9394ca77d77** | started | master-01 | https://10.13.33.38:2380 | https://10.13.33.38:2379 |
+------------------+---------+-----------+--------------------------+--------------------------+
{"health":"true"}[root@ETCDCTL_API=3 /opt/k8s/bin/etcdctl   -w table --cacert=/etc/kubernetes/cert/ca.pem   --cert=/etc/etcd/cert/etcd.pem   --key=/etc/etcd/cert/etcd-key.pem   --endpoints=https://10.13.33.29:2379,https://10.13.33.38:2379,https://10.13.33.40:2379  endpoint health
https://10.13.33.38:2379 is healthy: successfully committed proposal: took = 2.174634ms
https://10.13.
3f22
33.29:2379 is healthy: successfully committed proposal: took = 1.465878ms
https://10.13.33.40:2379 is healthy: successfully committed proposal: took = 2.36525ms
安装kubectl后可以通过kubectl查看
[root@master-01 ~]# kubectl get cs
NAME                 STATUS    MESSAGE             ERROR
controller-manager   Healthy   ok
scheduler            Healthy   ok
etcd-0               Healthy   {"health":"true"}
etcd-1               Healthy   {"health":"true"}
etcd-2               Healthy   {"health":"true"}

5、向etcd写入POD网段信息

etcdctl \
--endpoints=https://10.13.33.29:2379,https://10.13.33.38:2379,https://10.13.33.40:2379 \
--ca-file=/opt/k8s/work/ca.pem \
--cert-file=/opt/k8s/work/flanneld.pem \
--key-file=/opt/k8s/work/flanneld-key.pem \
mk /kubernetes/network/config '{"Network":"'172.30.0.0/16'", "SubnetLen": 21, "Backend": {"Type": "vxlan"}}'

6、重启flannel、docker

systemctl restart flanneld docker
systemctl status flanneld docker | grep Active
ip addr show| grep flannel.1

7、检查flannel、docker是否获取到正确的子网

[root@master-01 work]# ip addr show|grep flannel.1
5: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default
inet 172.30.48.0/32 scope global flannel.1
[root@master-01 work]# ip addr show|grep docker
6: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
inet 172.30.48.1/21 brd 172.30.55.255 scope global docker0
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: