《kubernetes-1.8.0》04-master搭建
2017-11-22 20:31
483 查看
《kubernetes-1.8.0》04-master搭建
《kubernetes 1.8.0 测试环境安装部署》时间:2017-11-21
一、api-HA介绍
部署三台master:目前所谓的 Kubernetes HA 其实主要的就是 API Server 的 HA,master 上其他组件比如 controller-manager 等都是可以通过 Etcd 做选举;而 API Server 只是提供一个请求接收服务,所以对于 API Server 一般有两种方式做 HA;一种是对多个 API Server 做 vip(HAproxy或者keepalive),另一种使用 nginx 反向代理,本文采用 nginx 方式。
master 之间除 api server 以外其他组件通过 etcd 选举,api server 默认不作处理;在每个 node 上启动一个 nginx,每个 nginx 反向代理所有 api server,node 上 kubelet、kube-proxy 连接本地的 nginx 代理端口,当 nginx 发现无法连接后端时会自动踢掉出问题的 api server,从而实现 api server 的 HA。
源自:mritd.me/部署-ha-master
一、config 通用配置(/etc/kubernetes/config)
三台master上证书与 rpm 都安装完成后,只需要修改配置(配置位于 /etc/kubernetes 目录)后启动相关组件即可### # kubernetes system config # # The following values are used to configure various aspects of all # kubernetes services, including # # kube-apiserver.service # kube-controller-manager.service # kube-scheduler.service # kubelet.service # kube-proxy.service # logging to stderr means we get it in the systemd journal KUBE_LOGTOSTDERR="--logtostderr=true" # journal message level, 0 is debug KUBE_LOG_LEVEL="--v=2" # Should this cluster be allowed to run privileged docker containers KUBE_ALLOW_PRIV="--allow-privileged=true" # How the controller-manager, scheduler, and proxy find the apiserver KUBE_MASTER="--master=http://127.0.0.1:8080"
KUBE_MASTER:用于controller-manager, scheduler, and proxy find the apiserver;
二、apiserver配置(/etc/kubernetes/apiserver)
### # kubernetes system config # # The following values are used to configure the kube-apiserver # # The address on the local server to listen to. KUBE_API_ADDRESS="--advertise-address=172.18.169.131 --insecure-bind-address=127.0.0.1 --bind-address=172.18.169.131" # The port on the local server to listen on. KUBE_API_PORT="--insecure-port=8080 --secure-port=6443" # Port minions listen on # KUBELET_PORT="--kubelet-port=10250" # Comma separated list of nodes in the etcd cluster KUBE_ETCD_SERVERS="--etcd-servers=https://172.18.169.131:2379,https://172.18.169.132:2379,https://172.18.169.133:2379" # Address range to use for services KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" # default admission control policies KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota,NodeRestriction" # Add your own! KUBE_API_ARGS="--authorization-mode=RBAC,Node \ --runtime-config=batch/v2alpha1=true \ --anonymous-auth=false \ --kubelet-https=true \ --enable-bootstrap-token-auth \ --token-auth-file=/etc/kubernetes/token.csv \ --service-node-port-range=30000-50000 \ --tls-cert-file=/etc/kubernetes/ssl/kubernetes.pem \ --tls-private-key-file=/etc/kubernetes/ssl/kubernetes-key.pem \ --client-ca-file=/etc/kubernetes/ssl/k8s-root-ca.pem \ --service-account-key-file=/etc/kubernetes/ssl/k8s-root-ca.pem \ --etcd-quorum-read=true \ --storage-backend=etcd3 \ --etcd-cafile=/etc/etcd/ssl/etcd-root-ca.pem \ --etcd-certfile=/etc/etcd/ssl/etcd.pem \ --etcd-keyfile=/etc/etcd/ssl/etcd-key.pem \ --enable-swagger-ui=true \ --apiserver-count=3 \ --audit-policy-file=/etc/kubernetes/audit-policy.yaml \ --audit-log-maxage=30 \ --audit-log-maxbackup=3 \ --audit-log-maxsize=100 \ --audit-log-path=/var/log/kube-audit/audit.log \ --event-ttl=1h"
KUBE_API_ADDRESS:制定apiserver监听的IP,http监听127.0.0.1(不对外),https监听本机网卡地址。
--authorization-mode=RBAC,Node:授权模型增加了 Node 参数,因为 1.8 后默认 system:node role 不会自动授予 system:nodes 组,具体请参看 CHANGELOG(before-upgrading 段最后一条说明)
由于以上原因,–admission-control 同时增加了 NodeRestriction 参数,关于关于节点授权器请参考 Using Node Authorization
--enable-bootstrap-token-auth:用于开启apiserver token认证,支持kubelet通过token的方式进行注册。
--token-auth-file=/etc/kubernetes/token.csv:对应记录token的文件位置,后续需创建。
增加
--audit-policy-file参数用于指定高级审计配置,具体可参考 CHANGELOG(before-upgrading 第四条)、Advanced audit,后续创建对应的audit yaml文件。
增加
--runtime-config=batch/v2alpha1=true参数用于cron job定时任务的支持。
创建对应的token文件、kubelet TLS相关配置文件、kube-proxy TLS相关配置文件以及audit-prolicy.yaml文件
##设置环境变量,生成token随机数 export KUBE_APISERVER="https://127.0.0.1:6443" export BOOTSTRAP_TOKEN=$(head -c 16 /dev/urandom | od -An -t x | tr -d ' ') echo "Tokne: ${BOOTSTRAP_TOKEN}" ##创建对应的token文件 $ cat > /etc/kubernetes/token.csv <<EOF ${BOOTSTRAP_TOKEN},kubelet-bootstrap,10001,"system:kubelet-bootstrap" EOF ##创建kubelet以及kube-proxy的配置文件 ##kubelet配置文件 kubectl config set-cluster kubernetes \ --certificate-authority=k8s-root-ca.pem \ --embed-certs=true \ --server=${KUBE_APISERVER} \ --kubeconfig=bootstrap.kubeconfig kubectl config set-credentials kubelet-bootstrap \ --token=${BOOTSTRAP_TOKEN} \ --kubeconfig=bootstrap.kubeconfig kubectl config set-context default \ --cluster=kubernetes \ --user=kubelet-bootstrap \ --kubeconfig=bootstrap.kubeconfig kubectl config use-context default --kubeconfig=bootstrap.kubeconfig ##kube-proxy配置文件 kubectl config set-cluster kubernetes \ --certificate-authority=k8s-root-ca.pem \ --embed-certs=true \ --server=${KUBE_APISERVER} \ --kubeconfig=kube-proxy.kubeconfig kubectl config set-credentials kube-proxy \ --client-certificate=kube-proxy.pem \ --client-key=kube-proxy-key.pem \ --embed-certs=true \ --kubeconfig=kube-proxy.kubeconfig kubectl config set-context default \ --cluster=kubernetes \ --user=kube-proxy \ --kubeconfig=kube-proxy.kubeconfig kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig ##生成高级审计配置 cat >> audit-policy.yaml <<EOF # Log all requests at the Metadata level. apiVersion: audit.k8s.io/v1beta1 kind: Policy rules: - level: Metadata EOF
分发token文件、kubelet TLS相关配置文件、kube-proxy TLS相关配置文件以及audit-prolicy.yaml文件至三台master对应目录
for IP in `seq 131 133`;do scp *.kubeconfig /etc/kubernetes/token.csv audit-policy.yaml root@172.18.169.$IP:/etc/kubernetes ssh root@172.18.169.$IP chown -R kube:kube /etc/kubernetes/ssl done
设置 log 目录权限
for IP in `seq 131 133`;do ssh root@172.18.169.$IP mkdir -p /var/log/kube-audit /usr/libexec/kubernetes ssh root@172.18.169.$IP chown -R kube:kube /var/log/kube-audit /usr/libexec/kubernetes ssh root@172.18.169.$IP chmod -R 755 /var/log/kube-audit /usr/libexec/kubernetes done
三、controller-manager 配置(/etc/kubernetes/controller-manager)
### # The following values are used to configure the kubernetes controller-manager # defaults from config and apiserver should be adequate # Add your own! KUBE_CONTROLLER_MANAGER_ARGS="--address=0.0.0.0 \ --service-cluster-ip-range=10.254.0.0/16 \ --cluster-name=kubernetes \ --cluster-signing-cert-file=/etc/kubernetes/ssl/k8s-root-ca.pem \ --cluster-signing-key-file=/etc/kubernetes/ssl/k8s-root-ca-key.pem \ --service-account-private-key-file=/etc/kubernetes/ssl/k8s-root-ca-key.pem \ --root-ca-file=/etc/kubernetes/ssl/k8s-root-ca.pem \ --leader-elect=true \ --node-monitor-grace-period=40s \ --node-monitor-period=5s \ --pod-eviction-timeout=5m0s"
四、scheduler 配置(/etc/kubernetes/scheduler)
### # kubernetes scheduler config # default config should be adequate # Add your own! KUBE_SCHEDULER_ARGS="--leader-elect=true --address=0.0.0.0"
五、启动服务并查看群集组件状态
$ sudo systemctl daemon-reload $ sudo systemctl start kube-apiserver $ sudo systemctl start kube-controller-manager $ sudo systemctl start kube-scheduler $ sudo systemctl enable kube-apiserver $ sudo systemctl enable kube-controller-manager $ sudo systemctl enable kube-scheduler $ sudo kubectl get cs NAME STATUS MESSAGE ERROR scheduler Healthy ok controller-manager Healthy ok etcd-1 Healthy {"health": "true"} etcd-2 Healthy {"health": "true"} etcd-0 Healthy {"health": "true"}
至此master节点基本部署完成
本系列其他内容:
01-环境准备
02-etcd群集搭建
03-kubectl管理工具
04-master搭建
05-node节点搭建
06-addon-calico
07-addon-kubedns
08-addon-dashboard
09-addon-kube-prometheus
10-addon-EFK
11-addon-Harbor
12-addon-ingress-nginx
13-addon-traefik
参考链接:
https://mritd.me/2017/10/09/set-up-kubernetes-1.8-ha-cluster/
https://github.com/opsnull/follow-me-install-kubernetes-cluster
https://kubernetes.io/docs/reference/generated/kube-apiserver/
相关文章推荐
- Kubernetes多master集群搭建
- 手动搭建Kubernetes1.8高可用集群(4)Master
- 《kubernetes-1.8.0》02-etcd群集搭建
- ubuntu搭建hadoop伪分布式环境:jdk1.8.0+hadoop2.8.0+eclipse-jee-neon-3
- kubernetes的service的网络类型ingress的搭建
- Ubuntu学习总结-04 搭建JAVA开发环境
- Kubernetes高级实践:Master高可用方案设计和踩过的那些坑
- Ubuntu 16.04下搭建kubernetes集群环境
- jenkins 服务(分布式master+slave) 搭建
- Kubernetes集群搭建与测试
- kubernetes 简单集群搭建(2台虚拟机)
- kubernetes可视化搭建集群,web界面一键操作
- kubernetes环境搭建
- 04Mybatis_搭建Mybatis的开发环境
- 10分钟搭建Kubernetes容器集群平台【转】
- redis 主从master-slave搭建及测试
- cloudfoundry 中hbase_master启动不成功 vsphere搭建的ubuntu1004