ZABBIX监控一台服务器上多实例memcache落地经验步骤

上一次给大家分享了工作中监控多实例tomcat的自发现zabbix做法，今天带给大家的就是一台服务器上监控多memcache的一个落地经验。
首先我带大家看一下多实例memcache做法，话不多说，咱们开工：
由于是多实例，所以就需要用到zabbix的自动发现功能（LLD）基本思路如下：

1、写自动发现脚本。2、写状态取值脚本。3、添加配置文件。4、添加权限。5、配置zabbix web。

zabbix我采用yum安装所以路径是/etc/zabbix 首先创建一个目录为: mkdir -p /etc/zabbix/scripts
这里我们要用到两个脚本：
cd /etc/zabbix/scripts

第一个脚本
vim memcached_discovery.py
#!/usr/bin/env python
import os
import json
t=os.popen("""sudo netstat -nltp|awk -F: '/memcached/&&/LISTEN/{print $2}'|awk '{print $1}'| grep -v grep | grep -v '^$' """)
ports = []
for port in t.readlines():
r = os.path.basename(port.strip())
ports += [{'{#MCPORT}':r}]
print json.dumps({'data':ports},sort_keys=True,indent=4,separators=(',',':'))

第二个脚本
vim memcached_status.sh
#!/bin/bash
# 1)yum install nc
# 2)python memcached_discovery.py
#########################

IP=127.0.0.1
PORT="$1"
METRIC="$2"

if [ $# -lt 2 ];then
echo "please set argument"
exit 1
fi

STATUS=`echo "stats" | nc $IP $PORT | grep -w "$METRIC" | awk '{print $3}'`
case $METRIC in
'version')
echo $STATUS
;;
'uptime')
echo $STATUS
;;
'curr_connections')
echo $STATUS
;;
'total_connections')
echo $STATUS
;;
'cmd_get')
echo $STATUS
;;
'cmd_set')
echo $STATUS
;;
'get_hits')
echo $STATUS
;;
'get_misses')
echo $STATUS
;;
'bytes_read')
echo $STATUS
;;
'bytes_written')
echo $STATUS
;;
'curr_items')
echo $STATUS
;;
'total_items')
echo $STATUS
;;
'expired_unfetched')
echo $STATUS
;;
'evicted_unfetched')
echo $STATUS
;;
*)
echo "Not selected metric"
exit 0
;;
esac

ok，两个脚本解决因为我zabbix配置文件里指定的.conf位置是zabbix_agentd.d下
所以cd /etc/zabbix/zabbix_agnetd.d
下边创建key
vim userparameter_memcached.conf

UserParameter=memcached.discovery[*],python /etc/zabbix/scripts/memcached_discovery.py
UserParameter=memcached[*],/bin/bash /etc/zabbix/scripts/memcached_status.sh $1 $2

保存退出！
添加zabbix权限
vim /etc/sudoers.d/zabbix
Defaults:zabbix !requiretty
zabbix ALL=(ALL) NOPASSWD: SUPERVISORCTLZB
Cmnd_Alias SUPERVISORCTLZB = /sbin/ss,/usr/sbin/ss,/sbin/dmidecode,/usr/sbin/dmidecode,/sbin/service,/usr/sbin/service,/bin/netstat

ok，记得要么关闭防火墙要么添加规则并添加zabbix权限 否则会监控时出现json格式的问题

下边就在web上导入模板添加模板就好了，这些都是基础的不在复述，下边给大家模板：

<?xml version="1.0" encoding="UTF-8"?>
<zabbix_export>
<version>3.0</version>
<date>2017-04-05T09:25:04Z</date>
<groups>
<group>
<name>Template For Base</name>
</group>
</groups>
<templates>
<template>
<template>Template Linux Memcached Discovery</template>
<name>Template Linux Memcached Discovery</name>
<description/>
<groups>
<group>
<name>Template For Base</name>
</group>
</groups>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<items/>
<discovery_rules>
<discovery_rule>
<name>Linux Memcached Discovery</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>memcached.discovery</key>
<delay>60</delay>
<status>0</status>
<allowed_hosts/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<delay_flex/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<filter>
<evaltype>0</evaltype>
<formula/>
<conditions/>
</filter>
<lifetime>30</lifetime>
<description>Discovery of redis ports on Linux</description>
<item_prototypes>
<item_prototype>
<name>Memcached-$1-bytes_read</name>
<type>0</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>memcached[{#MCPORT},bytes_read]</key>
<delay>60</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>1</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor/>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
<item_prototype>
<name>Memcached-$1-bytes_written</name>
<type>0</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>memcached[{#MCPORT},bytes_written]</key>
<delay>60</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>1</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor/>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
<item_prototype>
<name>Memcached-$1-cmd_get</name>
<type>0</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>memcached[{#MCPORT},cmd_get]</key>
<delay>60</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>1</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor/>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
<item_prototype>
<name>Memcached-$1-cmd_set</name>
<type>0</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>memcached[{#MCPORT},cmd_set]</key>
<delay>60</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>1</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor/>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
<item_prototype>
<name>Memcached-$1-curr_connections</name>
<type>0</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>memcached[{#MCPORT},curr_connections]</key>
<delay>60</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>0</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor/>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
<item_prototype>
<name>Memcached-$1-evicted_unfetched</name>
<type>0</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>memcached[{#MCPORT},evicted_unfetched]</key>
<delay>60</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>0</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor/>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
<item_prototype>
<name>Memcached-$1-expired_unfetched</name>
<type>0</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>memcached[{#MCPORT},expired_unfetched]</key>
<delay>60</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>0</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor/>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
<item_prototype>
<name>Memcached-$1-get_hits</name>
<type>0</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>memcached[{#MCPORT},get_hits]</key>
<delay>60</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>1</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor/>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
<item_prototype>
<name>Memcached-$1-get_misses</name>
<type>0</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>memcached[{#MCPORT},get_misses]</key>
<delay>60</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>1</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor/>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
<item_prototype>
<name>Memcached-$1-total_connections</name>
<type>0</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>memcached[{#MCPORT},total_connections]</key>
<delay>60</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>0</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor/>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
<item_prototype>
<name>Memcached-$1-total_items</name>
<type>0</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>memcached[{#MCPORT},total_items]</key>
<delay>60</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>0</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor/>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
<item_prototype>
<name>Memcached-$1-uptime</name>
<type>0</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>memcached[{#MCPORT},uptime]</key>
<delay>3600</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>0</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor/>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
<item_prototype>
<name>Memcached-$1-version</name>
<type>0</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>memcached[{#MCPORT},version]</key>
<delay>3600</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>0</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor/>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
<item_prototype>
<name>Memcached-$1-shutdown</name>
<type>0</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>net.tcp.listen[{#MCPORT}]</key>
<delay>60</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>0</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor/>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Memcached_Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
</item_prototypes>
<trigger_prototypes>
<trigger_prototype>
<expression>{Template Linux Memcached Discovery:net.tcp.listen[{#MCPORT}].last()}<>1</expression>
<name>Memcached-{HOST.NAME}-{#MCPORT} shutdown</name>
<url/>
<status>0</status>
<priority>4</priority>
<description/>
<type>0</type>
<dependencies/>
</trigger_prototype>
</trigger_prototypes>
<graph_prototypes/>
<host_prototypes/>
</discovery_rule>
</discovery_rules>
<macros/>
<templates/>
<screens/>
</template>
</templates>
</zabbix_export>

至此，zabbix监控多实例memcache已经完成！ 那个监控单点的怎么监控呢？

1：环境准备：
[root@agent scripts]# cat /etc/redhat-release
CentOS release 6.7 (Final)
[root@agent scripts]# sestatus
SELinux status: disabled
[root@agent scripts]# /etc/init.d/iptables stop

这里我的memcached服务器的ip是192.168.56.99
2：安装nc工具
本文的监控脚本中需要使用nc工具
yum install -y nc
3：配置自定义监控项
上传脚本
[root@agent ~]# mkdir /etc/zabbix/scripts/ -p
[root@agent ~]# cd /etc/zabbix/scripts/
[root@agent scripts]# ls
memcache_monitor.sh
脚本需要给执行权限！
脚本内容如下：
[root@agent scripts]# cat memcache_monitor.sh
#!/bin/bash
Status=$1
CACHEFILE="/tmp/memcache_monitor.txt"
# Check and run the script
TIMEFLM=`stat -c %Y $CACHEFILE`
TIMENOW=`date +%s`
if [ `expr $TIMENOW - $TIMEFLM` -gt 60 ]; then
rm -f $CACHEFILE
fi
if [ ! -f $CACHEFILE ];then
echo -e "stats\nquit"|nc 127.0.0.1 11211|grep STAT >$CACHEFILE 2>/dev/null
fi
awk '/ '$Status' /{print $NF}' $CACHEFILE
创建zabbix agent的自定义监控项配置文件
[root@agent scripts]# cd /etc/zabbix/zabbix_agentd.d/
[root@agent zabbix_agentd.d]# cat memcache_monitor.conf
UserParameter=memcached_stats[*],/etc/zabbix/scripts/memcache_monitor.sh $1
重启zabbix-agent使刚才的配置文件生效
[root@agent scripts]# /etc/init.d/zabbix-agent restart
Shutting down Zabbix agent: [ OK ]
Starting Zabbix agent: [ OK ]

4：在zabbix-server上测试
[root@zabbix-server01 ~]# zabbix_get -s 192.168.56.99 -k memcached_stats[uptime]
10518
[root@zabbix-server01 ~]# zabbix_get -s 192.168.56.99 -k memcached_stats[bytes]
74
5：web上配置
导入模板
链接模板到主机

在Latest data中看到最新的数据，恭喜你，成功了！
系统运维工程师：李超