Kubernetes多master集群搭建

官方社区 
 1、创建步骤：

Creating the reliable constituent nodes that collectively form our HA master implementation.
Setting up a redundant, reliable storage layer with clustered etcd.
Starting replicated, load balanced Kubernetes API servers
Setting up master-elected Kubernetes scheduler and controller-manager daemons
  创建高可用的节点来组件HA master架构
  通过etcd集群创建高可用、冗余的数据存储
  启动备份、负载均衡的APIservers
  创建支持master选举的scheduler 和controller-manage daemons



                                                                                      架构图---

2、初始化工作节点

通过kubelet来运行必要的工作节点.通过systemd、upstart等工具来保证节点的failover。然后通过在kubelet的/etc/kubernetes/manifests/目录下创建各个组件的yaml文件来生成运行容器（static
pod，有kubelet来保证各个pod的存货已经failover一旦pod停止或者配置文件更新，kubelet都会）根据重启策略来启动相关节点。

3、创建数据高可用、做好备份副本机制的存储层

replicated
etcd cluster is likely reliable enough. You can add additional reliability by increasing the size of the cluster from three to five nodes. 

目前replicated etcd 集群已经相当可靠，你可以通过添加节点来提供集群的可靠性。

4、Cluster etcd

创建etcd集群（etcd，类似于zookeeper的一个分布式key/value 存储服务。）

5、Replicated API Server

安装配置文件

安装一大堆的证书、配置文件stuff。

First you need to create the
initial log file, so that Docker mounts a file instead of a directory:

touch /var/log/kube-apiserver.log

Next, you need to create a 

/srv/kubernetes/

 directory
on each node. This directory includes:
basic_auth.csv - basic auth user and password
ca.crt - Certificate Authority cert
known_tokens.csv - tokens that entities (e.g. the kubelet) can use to talk to the apiserver
kubecfg.crt - Client certificate, public key
kubecfg.key - Client certificate, private key
server.cert - Server certificate, public key
server.key - Server certificate, private key

The easiest way to create this directory, may be to copy it from the master node of a working cluster, or you can manually generate these files yourself
配置完成上面的一大堆之后，将 kube-apiserver.yaml into 

/etc/kubernetes/manifests/

 on
each master node，来启动apiserver。kubelet会自动启动apiserver服务。

至此所有服务已经部署完毕，但是单单部署组件并不会是master elect生效，还需要在controller-manager和scheduler上配置一个--leader-elect的配置项，这将是controller-manager和scheduler选主。

Installing configuration files

First, create empty log files on each node, so that Docker will mount the files not make new directories:

touch /var/log/kube-scheduler.log
touch /var/log/kube-controller-manager.log

Next, set up the descriptions of the scheduler and controller manager pods on each node. by copying kube-scheduler.yaml and kube-controller-manager.yaml into
the 

/etc/kubernetes/manifests/

 directory.

kubernets

http://www.tuicool.com/articles/iy2ueaN

ubuntu安装k8s：
http://www.cnblogs.com/puyangsky/p/5635751.html

Docker
  采用registry镜像
   1、搭建本地registry（docker有提供具体镜像）
   2、配置docker daemon 启动参数 
   3、重启dockerdaemon

Kubernetes
   cAdvisor
    1、kubernetes内置cAdvisor作为单个node的监控。默认http ui端口 4194

    2、Heapster作为Pod运行在kubernetes集群中，通过kubelet发此案所有运行在集群中的节点
    3、kubelet通过cAdvisor获取所在node及容器的数据，heapster通过labels管理pod分组信息，并被推送到
          一个可配置后端，用于存储和可视化。当前支持的backend（InfluxDB，GCM）
-----Yaml--------------------------

Kubernetes资源创建yml语法 : http://blog.coocla.org/kubernetes-resource-yaml.html

------------------------------------
   Service Controller
---------------service---------------
Name:                   mongo-3
Namespace:              fanjun
Labels:                 app=mongo-3
Selector:               app=mongo-3
Type:                   NodePort
IP:                     10.254.150.87                  //虚拟集群ip
Port:                   0       27017/TCP            //service提供的 虚拟端口，对应如上IP
NodePort:               0       31700/TCP       // 映射为node端口可供外部访问
Endpoints:              172.16.58.5:27017     //pod id and port
Session Affinity:       None
No events.
------kubernetes 中deploy heapster & influxdb----------------------------

   https://github.com/kubernetes/heapster  目录下有一个deploy的目录，里面有安装的shell和 yaml文件。
   command ：$ kubectl create -f deploy/kube-config/influxdb/ https://github.com/kubernetes/heapster/blob/master/docs/influxdb.md
------influx db 中存储的监控数据 schema -------
measurement                    description

cpu/limit	CPU hard limit in millicores.
cpu/node_capacity	Cpu capacity of a node.

               .....                                    ...... https://github.com/kubernetes/heapster/blob/master/docs/storage-schema.md
----------------------------kubernetes +influxDb 中查询measurement 的语法----
select * from k8s."default"."cpu/limit" where labels="" and namespace_name="" and type=""

-查询语句--http://anyof.me/articles/390