【原创】用coreseek快速搭建sphinx中文分词搜索引擎
- WBOYオリジナル
- 2016-06-07 16:30:221079ブラウズ
以下内容基于linux 系统。 yum -y install glibc-common libtool autoconf automake mysql-devel expat-devel#如果不安装这个 可能下面 sh buildconf.sh会报错！！！cd /data/srctar -xjf ../software/autoconf-2.64.tar.bz2cd autoconf-2.64/./configuremak
以下内容基于linux 系统。
yum -y install glibc-common libtool autoconf automake mysql-devel expat-devel #如果不安装这个 可能下面 sh buildconf.sh会报错！！！ cd /data/src tar -xjf ../software/autoconf-2.64.tar.bz2 cd autoconf-2.64/ ./configure make && make install cd ../ cd /data/software wget http://www.coreseek.cn/uploads/csft/4.0/coreseek-4.1-beta.tar.gz cd /data/src tar zxf ../software/coreseek-4.1-beta.tar.gz cd coreseek-4.1-beta/mmseg-3.2.14 ./bootstrap ./configure --prefix=/usr/local/mmseg3 make && make install cd ../ cd /data/src/coreseek-4.1-beta/csft-4.1/ sh buildconf.sh ./configure --prefix=/usr/local/coreseek --without-unixodbc --with-mmseg --with-mmseg-includes=/usr/local/mmseg3/include/mmseg/ --with-mmseg-libs=/usr/local/mmseg3/lib/ --without-mysql make && make install cd ../ ##测试mmseg分词，coreseek搜索（需要预先设置好字符集为zh_CN.UTF-8，确保正确显示中文） cd testpack cat var/test/test.xml #此时应该正确显示中文 /usr/local/mmseg3/bin/mmseg -d /usr/local/mmseg3/etc var/test/test.xml /usr/local/coreseek/bin/indexer -c etc/csft.conf --all /usr/local/coreseek/bin/search -c etc/csft.conf 网络搜索 #创建sphinx创建索引的脚本： mkdir -p /data/sh/other
vi /data/sh/other/sphinx_update_index.sh
#!/bin/bash
CONFFILE=/usr/local/coreseek/etc/sphinx_index.conf
/bin/sed s#var\/data\/#var\/data2\/#g ${CONFFILE} > ${CONFFILE}.2
mkdir -p /usr/local/coreseek/var/data2
#/usr/local/coreseek/bin/indexer --config ${CONFFILE}.2 --all --rotate
/usr/local/coreseek/bin/indexer --config ${CONFFILE}.2 --all
pkill -9 searchd
sleep 4
/bin/rm -rf /usr/local/coreseek/var/data/
/bin/mv /usr/local/coreseek/var/data2/ /usr/local/coreseek/var/data/
sleep 2
/usr/local/coreseek/bin/searchd --config ${CONFFILE}
chmod 755 /data/sh/other/sphinx_update_index.sh
#配置sphinx索引参数配置
vi /usr/local/coreseek/etc/sphinx_index.conf
################################### PHPCMS ############################################
source cc_phpcms
{
type = mysql
sql_host = 172.26.11.75 #此处请改成您的真实配置
sql_user = phpcms #此处请改成您的真实配置
sql_pass = 123456 #此处请改成您的真实配置
sql_db = phpcms #此处请改成您的真实配置
sql_port= 3306 #此处请改成您的真实配置
sql_query_pre = SET SESSION query_cache_type=OFF
sql_query_pre = SET character_set_client = 'gbk'
sql_query_pre = SET character_set_connection ='gbk'
sql_query_pre = SET character_set_results ='utf8'
sql_query = SELECT `id`,`catid`,`typeid`,`title`,`status`,`updatetime` from `i_news` #此处请改成您的真实配置
sql_range_step = 1000
sql_attr_timestamp = updatetime
sql_attr_uint = catid
sql_attr_uint = typeid
sql_attr_uint = status
sql_query_post =
sql_ranged_throttle= 0
}
index cc_phpcms
{
source = cc_phpcms
path = /dev/shm/cc_phpcms #放这里比较好，因为这里是linux的内存区！
docinfo = extern
mlock = 0
enable_star = 1
morphology = none
stopwords =
min_word_len = 1
charset_dictpath = /usr/local/mmseg3/etc/ #注意此处
charset_type = zh_cn.utf-8 #注意此处
html_strip = 1
html_remove_elements = style, script
html_index_attrs = img=alt,title; a=title;
}
#################################### SETTING ############################################
indexer
{
mem_limit = 300M
}
searchd
{
# address = 0.0.0.0
#listen = 3312
#listen = 9312
#listen = 9306:mysql41
port = 3312
log = /usr/local/coreseek/var/log/searchd.log
query_log = /usr/local/coreseek/var/log/query.log
read_timeout = 5
max_children = 30
pid_file = /usr/local/coreseek/var/log/searchd.pid
max_matches = 1000
seamless_rotate = 1
}
#接下来实现数据源支持：让sphinx支持MySQL数据源
yum -y install mysql-devel libxml2-devel expat-devel cd /data/src/coreseek-4.1-beta/csft-4.1/ make clean sh buildconf.sh ./configure --prefix=/usr/local/coreseek --without-unixodbc --with-mmseg --with-mmseg-includes=/usr/local/mmseg3/include/mmseg/ --with-mmseg-libs=/usr/local/mmseg3/lib/ --with-mysql make && make install cd ../
##如果出现错误提示：“ERROR: cannot find MySQL include files…….To disable MySQL support, use –without-mysql option.“，可按照如下方法处理：
##请找到头文件mysql.h所在的目录，一般是/usr/local/mysql/include，请替换为实际的
##请找到库文件libmysqlclient.a所在的目录，一般是/usr/local/mysql/lib，请替换为实际的
##configure参数加上：–with-mysql-includes=/usr/local/mysql/include –with-mysql-libs=/usr/local/mysql/lib，执行后，重新编译安装
#跑sphinx服务脚本
/data/sh/other/sphinx_update_index.sh
好了，如果一切正常，将会顺利看到创建索引的信息如下：
下面写一段php代码进行测试（基于sphinx php 的api方式）：
$page = (int)$_GET['page'];
$page = ($page==0)?1:$page;
$perpage = 200;
$start = ($page -1) * $perpage;
$keyword = urldecode($_GET['key']);
require_once (S_ROOT . './api/sphinxapi.php');//请改成您的真实路径
$groupby = "";
$groupsort = "@group desc";
$filter = "fieldid";
$filtervals = array ();
$distinct = "";
$sortby = "";
$cl = new SphinxClient();
$cl->SetServer("localhost", 3312);
$cl->SetWeights(array (
100,
1
));
$cl->SetMatchMode(SPH_MATCH_ANY);
if (count($filtervals)) {
$cl->SetFilter($filter, $filtervals);
}
if ($groupby) {
$cl->SetGroupBy($groupby, SPH_GROUPBY_ATTR, $groupsort);
}
$order = 1;
if ($order == 0) { //按时间倒序
$cl->SetSortMode(SPH_SORT_ATTR_DESC, "inputtime");
}
elseif ($order == 1) { //按相关度排序
$cl->SetSortMode(SPH_SORT_RELEVANCE);
}
if ($distinct) {
$cl->SetGroupDistinct($distinct);
}
$cl->SetLimits($start, $perpage, ($limit > 1000) ? $limit : 1000);
$cl->SetRankingMode(SPH_RANK_PROXIMITY_BM25);
$cl->SetArrayResult(true);
$res = $cl->Query($keyword, 'cc_phpcms');
print_r($res);die;
上面的php代码没有做输入的字符过滤，这个请按自己的需要加上。
另外，
/data/sh/other/sphinx_update_index.sh 跑了一次后，
请
vi /data/sh/other/sphinx_update_index.sh
将
#/usr/local/coreseek/bin/indexer --config ${CONFFILE}.2 --all --rotate
/usr/local/coreseek/bin/indexer --config ${CONFFILE}.2 --all
变成
/usr/local/coreseek/bin/indexer --config ${CONFFILE}.2 --all --rotate
#/usr/local/coreseek/bin/indexer --config ${CONFFILE}.2 --all
也就是将注释调换，这样以后就可以设定个定时计划跑/data/sh/other/sphinx_update_index.sh 脚本了，
跑了/sphinx_update_index.sh 脚本后，自动会用–rotate的方式重建索引，也就是说新增加的内容也将会被索引到了。
当然，最好的方法还是做个实时索引的配置，下一篇将会重点介绍sphinx的实时索引功能！
原文地址：【原创】用coreseek快速搭建sphinx中文分词搜索引擎, 感谢原作者分享。