通过UDFS实现Memcached与MySQL的自动更新

本文出自：http://www.mpyun.com/
UDFs是User Defined Functions的缩写，表示MYSQL的用户定义函数，应用程序可以利用这些函数从MYSQL5.0以上版本的数据库中访问Memcached写入或者获取的数据。此外，MYSQL从5.1版本开始支持触发器，从而可以在触发器中使用UDFs直接更新Memcached的内容，这种方式降低了应用程序设计和编写的复杂性。
安装UDFs需要在数据库上安装两个包，分别是libmemcached和memcached_functions_mysql，并且保证Mysql为5.1及以上版本，可通过mysql --version命令查看当前 Mysql版本。
libmemcached下载可以访问其官方站： http://download.tangent.org/ memcached_functions_mysql下载地址：
http://patg.net/downloads/
第一次安装用的是libmemcached-0.53.tar.gz，编译memcached_functions_mysql-0.9时遇到了下面的问题：

servers.c: In function 'memc_servers_set':
servers.c:122: error: 'memcached_st' has no member named 'hosts'
servers.c:123: error: 'memcached_st' has no member named 'hosts'
servers.c:124: error: 'memcached_st' has no member named 'hosts'
最后用libmemcached-0.34.tar.gz，就没问题了，可能是软件的兼容性问题吧。

[root@bogon home]# tar -zxvf libmemcached-0.34.tar.gz
[root@bogon home]# cd libmemcached-0.34
[root@localhost libmemcached-0.34]# ./configure --prefix=/usr/local/libmemcached34 --with-memcached=/usr/local/bin/memcached
[root@localhost libmemcached-0.34]# make && make install
[root@bogon home]# tar zxvf memcached_functions_mysql-0.9.tar.gz
[root@localhost memcached_functions_mysql-0.9]# ./configure --prefix=/usr/local/memcache_mysql --with-mysql=/usr/local/mysql/bin/mysql_config --with-libmemcached=/usr/local/libmemcached34 #/usr/local/mysql为MySQL安装目录
加上--with-libmemcached=/usr/local/libmemcached53不然会报以下错误
checking for libmemcached >= 0.17... configure: error: libmemcached not found
[root@localhost memcached_functions_mysql-0.9]# make && make install
安装完毕。

首次需要使用CREATE FUNCTION来初始化用户定义函数。有两种方法可以初始化所有Mysql提供的用户定义函数：
1),在Mysql的SQL命令行中执行memcached_functions_mysql源码目录下的sql/install_functions.sql。
2), 运行memcached_functions_mysql源码目录下的utils/install.pl这个Perl脚本，把memcache function作为UDFs加入MySQL。
这里采用第一种办法：
mysql> use mysql;
mysql> source /home/memcached_functions_mysql-1.1/sql/install_functions.sql
出现如下错误：
ERROR 1126 (HY000): Can't open shared library 'libmemcached_functions_mysql.so' (errno: 22 /usr/local/mysql/lib/mysql/plugin/libmemcached_functions_mysql.so: cannot open shared object file: No such file or directory)
解决办法手动拷贝libmemcached_functions_mysql.so库到 Mysql安装目录：
[root@localhost lib]# cp /usr/local/memcache_mysql/lib/libmemcached_functions_mysql.so* /usr/local/mysql/lib/mysql/plugin/

2.查看安装是否成功：

mysql> select name,dl from mysql.func;
+------------------------------+---------------------------------+
| name | dl |
+------------------------------+---------------------------------+
| memc_add | libmemcached_functions_mysql.so |
| memc_add_by_key | libmemcached_functions_mysql.so |
| memc_servers_set | libmemcached_functions_mysql.so |
| memc_server_count | libmemcached_functions_mysql.so |
| memc_set | libmemcached_functions_mysql.so |
| memc_set_by_key | libmemcached_functions_mysql.so |
| memc_cas | libmemcached_functions_mysql.so |
| memc_cas_by_key | libmemcached_functions_mysql.so |
| memc_get | libmemcached_functions_mysql.so |
| memc_get_by_key | libmemcached_functions_mysql.so |
| memc_delete | libmemcached_functions_mysql.so |
| memc_delete_by_key | libmemcached_functions_mysql.so |
| memc_append | libmemcached_functions_mysql.so |
| memc_append_by_key | libmemcached_functions_mysql.so |
| memc_prepend | libmemcached_functions_mysql.so |
| memc_prepend_by_key | libmemcached_functions_mysql.so |
| memc_increment | libmemcached_functions_mysql.so |
| memc_decrement | libmemcached_functions_mysql.so |
| memc_replace | libmemcached_functions_mysql.so |
| memc_replace_by_key | libmemcached_functions_mysql.so |
| memc_servers_behavior_set | libmemcached_functions_mysql.so |
| memc_servers_behavior_get | libmemcached_functions_mysql.so |
| memc_behavior_set | libmemcached_functions_mysql.so |
| memc_behavior_get | libmemcached_functions_mysql.so |
| memc_list_behaviors | libmemcached_functions_mysql.so |
| memc_list_hash_types | libmemcached_functions_mysql.so |
| memc_list_distribution_types | libmemcached_functions_mysql.so |
| memc_udf_version | libmemcached_functions_mysql.so |
| memc_libmemcached_version | libmemcached_functions_mysql.so |
| memc_stats | libmemcached_functions_mysql.so |
| memc_stat_get_keys | libmemcached_functions_mysql.so |
| memc_stat_get_value | libmemcached_functions_mysql.so |
+------------------------------+---------------------------------+
32 rows in set (0.08 sec)

3.memcached_functions_mysql应用实例：

1).新建两张表：urls和results,更新urls表中的内容，使系统自动更新Memcached的内容。results用来就更新Memcached失败的记录。

SQL代码：
use test;
drop table if exists urls;
CREATE TABLE urls(
id int(10) NOT NULL,
url varchar(255) NOT NULL DEFAULT '',
PRIMARY KEY (id)
);

drop table if exists results;
CREATE TABLE results(
id int(10) NOT NULL,
results varchar(255) NOT NULL DEFAULT 'error',
time timestamp NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (id)
);

2).建立3个trigger

当向urls表中插入数据时，对Memcached执行set操作。trigger的代码如下：
insert操作：
DELIMITER //
DROP TRIGGER IF EXISTS url_mem_insert;
CREATE TRIGGER url_mem_insert
BEFORE INSERT ON urls
FOR EACH ROW BEGIN
set @mm = memc_set(NEW.id,NEW.url);
if @mm <> 0 then
insert into results(id) values(NEW.id);
end if;
END //
DELIMITER;

更新时对Memcached执行replace操作：
DELIMITER //
DROP TRIGGER IF EXISTS url_mem_update;
CREATE TRIGGER url_mem_update
BEFORE UPDATE ON urls
FOR EACH ROW BEGIN
set @mm = memc_replace(OLD.id,NEW.url);
if @mm <> 0 then
insert into results(id) values(OLD.id);
end if;
END //
DELIMITER;

删除对Memcached执行delete操作：
DELIMITER //
DROP TRIGGER IF EXISTS url_mem_delete;
CREATE TRIGGER url_mem_delete
BEFORE DELETE ON urls
FOR EACH ROW BEGIN
set @mm = memc_delete(OLD.id);
if @mm <> 0 then
insert into results(id) values(OLD.id);
end if;
END //
DELIMITER;

3).设置Memcached相关参数
设置UDFs操作Memcached服务器的IP地址和端口:
mysql> select memc_servers_set('192.168.56.2:11211');
+----------------------------------------+
| memc_servers_set('192.168.56.2:11211') |
+----------------------------------------+
| 0 |
+----------------------------------------+
1 row in set (0.01 sec)

mysql> select memc_server_count();
+---------------------+
| memc_server_count() |
+---------------------+
| 1 |
+---------------------+
1 row in set (0.01 sec)

在MySQL命令行中列出可以修改Memcache参数的行为，如下：
mysql> select memc_list_behaviors()\G
*************************** 1. row ***************************
memc_list_behaviors():
MEMCACHED SERVER BEHAVIORS
MEMCACHED_BEHAVIOR_SUPPORT_CAS
MEMCACHED_BEHAVIOR_NO_BLOCK
MEMCACHED_BEHAVIOR_TCP_NODELAY
MEMCACHED_BEHAVIOR_HASH
MEMCACHED_BEHAVIOR_CACHE_LOOKUPS
MEMCACHED_BEHAVIOR_SOCKET_SEND_SIZE
MEMCACHED_BEHAVIOR_SOCKET_RECV_SIZE
MEMCACHED_BEHAVIOR_BUFFER_REQUESTS
MEMCACHED_BEHAVIOR_KETAMA
MEMCACHED_BEHAVIOR_POLL_TIMEOUT
MEMCACHED_BEHAVIOR_RETRY_TIMEOUT
MEMCACHED_BEHAVIOR_DISTRIBUTION
MEMCACHED_BEHAVIOR_BUFFER_REQUESTS
MEMCACHED_BEHAVIOR_USER_DATA
MEMCACHED_BEHAVIOR_SORT_HOSTS
MEMCACHED_BEHAVIOR_VERIFY_KEY
MEMCACHED_BEHAVIOR_CONNECT_TIMEOUT
MEMCACHED_BEHAVIOR_KETAMA_WEIGHTED
MEMCACHED_BEHAVIOR_KETAMA_HASH
MEMCACHED_BEHAVIOR_BINARY_PROTOCOL
MEMCACHED_BEHAVIOR_SND_TIMEOUT
MEMCACHED_BEHAVIOR_RCV_TIMEOUT
MEMCACHED_BEHAVIOR_SERVER_FAILURE_LIMIT
MEMCACHED_BEHAVIOR_IO_MSG_WATERMARK
MEMCACHED_BEHAVIOR_IO_BYTES_WATERMARK

1 row in set (0.00 sec)
设置MEMCACHED_BEHAVIOR_NO_BLOCK为打开状态，这样在Memcached出现问题（不能连接）时，数据可以继续插入MySQL中，但有报错提示：如果不设置此值，那么Memcached失败时，数据要等到Memcached失败超时后才能插入到表中。
通过下面的设置，可以避免这种情况的发生。
mysql> select memc_servers_behavior_set('MEMCACHED_BEHAVIOR_NO_BLOCK','1');
+--------------------------------------------------------------+
| memc_servers_behavior_set('MEMCACHED_BEHAVIOR_NO_BLOCK','1') |
+--------------------------------------------------------------+
| 0 |
+--------------------------------------------------------------+
1 row in set (0.01 sec)

mysql> select memc_servers_behavior_set('MEMCACHED_BEHAVIOR_TCP_NODELAY','1');
+-----------------------------------------------------------------+
| memc_servers_behavior_set('MEMCACHED_BEHAVIOR_TCP_NODELAY','1') |
+-----------------------------------------------------------------+
| 0 |
+-----------------------------------------------------------------+
1 row in set (0.01 sec)

4).对memcached_functions_mysql的功能进行测试
向urls表中插入数据，然后查看Memcached是否对数据执行set操作:
mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> insert into urls(id,url) values (1,'http://www.test.com.cn');
Query OK, 1 row affected (0.09 sec)
mysql> select memc_get('1');
+------------------------+
| memc_get('1') |
+------------------------+
| http://www.test.com.cn |
+------------------------+
1 row in set (0.01 sec)

[root@localhost ~]# telnet 192.168.56.2 11211
Trying 192.168.56.2...
Connected to 192.168.56.2 (192.168.56.2).
Escape character is '^]'.
get 1
VALUE 1 0 22 http://www.test.com.cn END

更新测试：
mysql> update test.urls set url='http://blog.test.com.cn' where id=1;
Query OK, 1 row affected (0.05 sec)
Rows matched: 1 Changed: 1 Warnings: 0

mysql> select memc_get('1');
+-------------------------+
| memc_get('1') |
+-------------------------+
| http://blog.test.com.cn |
+-------------------------+
1 row in set (0.01 sec)

[root@localhost ~]# telnet 192.168.56.2 11211
Trying 192.168.56.2...
Connected to 192.168.56.2 (192.168.56.2).
Escape character is '^]'.
get 1
VALUE 1 0 23 http://blog.test.com.cn END

删除测试：
mysql> delete from test.urls where id=1;
Query OK, 1 row affected (0.00 sec)

mysql> select memc_get('1');
+---------------+
| memc_get('1') |
+---------------+
| NULL |
+---------------+
1 row in set (0.01 sec)

[root@localhost ~]# telnet 192.168.56.2 11211
Trying 192.168.56.2...
Connected to 192.168.56.2 (192.168.56.2).
Escape character is '^]'.
get 1
END