Keystone 高可靠性部署与性能测试

Goal

Keystone Region 为跨地域的 Openstack 集群提供了统一的认证和用户租户管理。目前公司在国内外部署了数十套 Openstack 集群，其中既有集群在内网，又有集群在公网；既有 Havana 集群，也有 Icehouse 集群；既有 nova-network 集群，又有 Neutron 集群，如下图：

为了集中管理，全局共享一个 Keystone Server, 因此对 Keystone
Server 的安全性、兼容性以及性能，都有特殊的要求。

安全性通过 SSL 实现和防止 DDOS 实现， tempest 测试表明 Keystone 具有很高的向后兼容性，可靠性通过 Apache、Haproxy、mysqlcluster 实现（关于
openstack 整体 HA 的实现，可以参考 http://blog.csdn.net/wsfdl/article/details/41386155），如下图：

Deployment

物理主机信息

Host Name IP VIP/DNS CPU Memory

keystone01 internal_ip01 public_ip/keystone-server E5-2620(24 Processor) 64G

keystone02 internal_ip02 public_ip/keystone-server E5-2620(24 Processor) 64G

说明：若无注明，keystone01 和 keystone02 的部署与配置相同
# yum -y install mysql mysql-server MySQL-python

# yum -y install openstack-keystone python-keystoneclient

# yum -y install haproxy

# yum -y install httpd

# yum -y install keepalived

# yum -y install haproxy

# yum -y install httpd

# yum -y install keepalived

Configuration

/etc/keystone/keystone.conf

[DEFAULT] public_endpoint=https://keystone-server/main/ admin_endpoint=https://keystone-server/admin/ [database] connection=mysql://keystone:keystonepass@mysqlserver/keystone max_pool_size=500 [signing] token_format=UUID [ssl] cert_subject=/C=US/ST=Unset/L=Unset/O=Unset/CN=keystone-server [token] provider=keystone.token.providers.uuid.Provider

/etc/httpd/conf.d/wsgi-keystone.conf

NameVirtualHost *:5000 Listen internal_ip0x:5000 <VirtualHost *:5000> ServerName keystone-main WSGIScriptAlias /main /var/www/cgi-bin/keystone/main ErrorLog /var/log/keystone/apache2-main-error.log LogLevel debug CustomLog /var/log/keystone/apache2-main-access.log common </VirtualHost> NameVirtualHost *:35357 Listen internal_ip0x:35357 <VirtualHost *:35357> ServerName keystone-admin WSGIScriptAlias /admin /var/www/cgi-bin/keystone/admin ErrorLog /var/log/keystone/apache2-admin-error.log LogLevel debug CustomLog /var/log/keystone/apache2-admin-access.log common </VirtualHost>

/etc/haproxy/haproxy.cfg

global daemon log 127.0.0.1 local3 defaults maxconn 4000 log global timeout server 10s timeout connect 10s timeout client 10s mode http option forwardfor option http-server-close log global listen stats mode http bind public_ip:8000 stats enable stats hide-version stats uri / stats realm Haproxy\ Statistics stats auth lecloud:openstack stats admin if TRUE frontend keystone_frontend bind public_ip:443 ssl crt /etc/haproxy/keystone_https.pem reqadd X-Forwarded-Proto:\ https acl admin_path path_beg /admin acl main_path path_beg /main use_backend admin_backend if admin_path use_backend main_backend if main_path backend admin_backend balance roundrobin redirect scheme https if !{ ssl_fc } server keystone-server-01 internal_ip01:35357 check inter 10s server keystone-server-02 internal_ip02:35357 check inter 10s backend main_backend balance roundrobin redirect scheme https if !{ ssl_fc } server keystone-server-01 internal_ip01:5000 check inter 10s server keystone-server-02 internal_ip02:5000 check inter 10s

/etc/keepalived/keepalived.conf

vrrp_script haproxy-check { script "killall -0 haproxy" interval 2 weight 10 } vrrp_instance openstack-vip { state MASTER # 注：keystone01 为 MASTER, keystone02 为 BACKUP priority 102 interface eth0 virtual_router_id 108 advert_int 3 virtual_ipaddress { public_ip } track_script { haproxy-check } }

# mkdir /var/www/cgi-bin/keystone/

# cp /usr/share/keystone/keystone.wsgi /var/www/cgi-bin/keystone/

# ln -s /var/www/cgi-bin/keystone/keystone.wsgi /var/www/cgi-bin/keystone/admin

# ln -s /var/www/cgi-bin/keystone/keystone.wsgi /var/www/cgi-bin/keystone/main

# service httpd start

# chkconfig httpd on

# keystone-manage ssl_setup --keystone-user keystone --keystone-group keystone 注：keystone01

# cat /etc/keystone/ssl/certs/keystone.pem /etc/keystone/ssl/private/keystonekey.pem > /etc/haproxy/keystone_https.pem 注：keystone01，同时把 keystone_https.pem
拷贝至 keystone02 /etc/haproxy/ 目录下

# (crontab -l -u keystone 2>&1 | grep -q token_flush) || echo '@dayly /usr/bin/keystone-manage token_flush >/var/log/keystone/keystone-tokenflush.log 2>&1' >>
/var/spool/cron/keystone

# echo "net.ipv4.ip_nonlocal_bind = 1" >> /etc/sysctl.conf

# sysctl -p

# service haproxy start

# chkconfig haproxy on

# service keepalived start

# chkconfig keepalived on

Benchmark

Configure Rally

关于 Rally，详情请参见 Openstack 性能测试 http://blog.csdn.net/wsfdl/article/details/41654373
# git clone https://git.openstack.org/stackforge/rally && cd rally

# ./rally/install_rally.sh -v

# source /opt/rally/bin/activate

# rally deployment create --filename=existing.json --name=existing

# rally -v task start create-user.json

(rally)[root@controller rally]# cat existing.json
{
    "type": "ExistingCloud",
    "auth_url": "https://keystone-server/admin/v2.0",
    "admin": {
        "username": "test",
        "password": "test",
        "tenant_name": "test"
    }
}

create-user.json

{
    "KeystoneBasic.create_user": [
        {
            "args": {
                "name_length": 10
            },
            "runner": {
                "type": "constant",
                "times": 10000,
                "concurrency": 900
            }
        }
    ]
} 

Result

注：以创建用户为例，一个并发数(Concurrency)，包含两个 HTTPS 请求（一个为申请 token，另一个为创建用户）。此处仅给出 mysql(单点) 数据库下keystone server 的并发性能。