如何给Sphinx做增量索引更新

在利用 Sphinx 做搜索引擎的时候，一般他的索引建立构成有如下几个部分：

1. 固定不变的主索引

2. 增量索引重建

3. 索引数据合并

在实际操作中，需要需要为增量索引的建立创建辅助表，这样才可以记住最后建立索引的记录ID,做实际的增量部分的索引建立。

CREATE TABLE search_counter

(

counterid INTEGER PRIMARY KEY NOT NULL,

max_doc_id INTEGER NOT NULL

);

在主索引的数据源中作如下方式的取数据设置

sql_query_pre = SET NAMES utf8

sql_query_pre = SET SESSION query_cache_type=OFF

sql_query_pre = REPLACE INTO search_counter SELECT 1,MAX(pid) FROM cdb_posts #创建主索引前更改标识位置

sql_query =

SELECT pid, fid,tid,authorid, dateline,subject, message

FROM cdb_posts

WHERE pid > $start AND pid <= $end

sql_query_range = SELECT 1, max_doc_id FROM search_counter WHERE counterid = 1

sql_range_step = 1000

sql_ranged_throttle = 1000

sql_query_info = SELECT * FROM cdb_posts WHERE pid=$id

在增量索引的数据源中作如下方式的取数据设置

sql_query_pre = SET NAMES utf8

sql_query_pre = SET SESSION query_cache_type=OFF

sql_query =

SELECT pid, fid,tid,authorid, dateline,

subject, message

FROM cdb_posts

WHERE pid >

(SELECT max_doc_id FROM search_counter WHERE counterid=1)

#增量索引是id大于标识位置的部分

在建立好配置后首先对sphinx中配置的全部索引做初始化

/usr/local/sphinx/bin/indexer –config –all /usr/local/sphinx/etc/sphinx.conf

为创建2个shell脚本，一个用来创建主索引、一个用来创建增量索引（此步可以省略）

1.创建主索引脚本build_main_index.sh

#!/bin/sh

/usr/local/sphinx/bin/searchd –stop >> /var/log/sphinx/searchdlog

/usr/local/sphinx/bin/indexer discuz –config /usr/local/sphinx/etc/sphinx.conf >> /var/log/sphinx/mainindexlog

/usr/local/sphinx/bin/searchd >> /var/log/sphinx/searchdlog

2.创建增量索引脚本build_delta_index.sh

#!/bin/sh

/usr/local/sphinx/bin/searchd –stop >> /var/log/sphinx/searchdlog

/usr/local/sphinx/bin/indexer discuz_delta –config /usr/local/sphinx/etc/sphinx.conf >> /var/log/sphinx/deltaindexlog

#/usr/local/sphinx/bin/indexer –merge discuz discuz_delta –config /usr/local/sphinx/etc/sphinx.conf >> /var/log/sphinx/deltaindexlog

/usr/local/sphinx/bin/searchd >> /var/log/sphinx/searchdlog

在crontab 中添加定时脚本，按照自己期望的策略定期执行重建索引的操作。

比如可以每天凌晨2点执行主索引重建，其他每10分钟建立一次增量索引重建。

转载自 http://th9988.blog.163.com/blog/static/4888243220113207210871/