ElasticSearch 安装 go-mysql-elasticsearch 同步mysql的数据

一、首先在Centos6.5上安装 go 语言环境

　　下载Golang语言包：https://studygolang.com/dl

[hoojjack@localhost src]$ ls
apache-maven-3.5.0  go  go1.6.2.linux-amd64.tar.gz

二、解压.tar.gz安装包。

三、配置安装环境。

[hoojjack@localhost etc]$ vim profile
export GOPATH=/usr/local/src/go/work
export GOROOT=/usr/local/src/go
export PATH=$GOPATH/bin:$PATH

GOARCH（指定系统环境，i386表示x86，amd64表示x64）：amd64

GOROOT：/usr/local/src/go（go的解压路径）

GOBIN：%GOROOT%/bin（exe执行文件路径）

GOOS：（go运行的系统）

GOPATH：/usr/local/src/go/work（用于存放Go语言Package的目录，这个目录不能在Go的安装目录中，放了好像也没什么事，至少我目前没事 ^_^）

我们需要添加的环境变量分别是go的安装路径GOROOT：/usr/local/src/go，PATH：$GOROOT/bin:$PATH，go的工作路径GOPATH:/usr/local/src/go/work

四、配置go环境后配置go-mysql-elasticsearch

首先确保环境中安装了git

安装依赖包：

sudo yum install gettext-devel openssl-devel perl-CPAN perl-devel zlib-devel

也可以直接用yum命令安装：

yum install git

然后：

获取安装包：go get github.com/siddontang/go-mysql-elasticsearch

安装：

cd $GOPATH/src/github.com/siddontang/go-mysql-elasticsearch
make

[hoojjack@localhost etc]$ cd /usr/local/src/go/src/github.com/siddontang/go-mysql-elasticsearch
[hoojjack@localhost go-mysql-elasticsearch]$ ls
bin  cmd  elastic  etc  Godeps  LICENSE  Makefile  README.md  river  var
[hoojjack@localhost go-mysql-elasticsearch]$

五、修改配置文件

# MySQL address, user and password
# user must have replication privilege in MySQL.
my_addr = "127.0.0.1:3306"     //需要同步的mysql基本设置
my_user = "root"
my_pass = "root"

# Elasticsearch address
es_addr = "127.0.0.1:9200"     //本地elasticsearch配置

# Path to store data, like master.info, and dump MySQL data
data_dir = "./var"             //数据存储的url
//以下配置保存默认不变
# Inner Http status address
stat_addr = "127.0.0.1:12800"

# pseudo server id like a slave
server_id = 1001

# mysql or mariadb
flavor = "mysql"
# mysqldump execution path
mysqldump = "mysqldump"

# MySQL data source
[[source]]
schema = "hoojjack"             //elasticsearch 与 mysql 同步时对应的数据库名称

# Only below tables will be synced into Elasticsearch.
# "test_river_[0-9]{4}" is a wildcard table format, you can use it if you have many sub tables, like table_0000 - table_1023
# I don't think it is necessary to sync all tables in a database.
tables = ["test_river", "test_river_[0-9]{4}"]    //支持通配符，可以指定只复制hoojjack数据库中指定的表数据

# Below is for special rule mapping
[[rule]]
schema = "hoojjack"    //数据库名称
table = "test_river"   //表名称
index = "river"        //对应的索引名称
# title is MySQL test_river field name, es_title is the customized name in Elasticsearch
[rule.field]
# This will map column title to elastic search my_title
title="es_title"     //将可以将mysql的某个属性对应指向elasticsearch的某个field, 如test_river的titile属性对应es_title
# This will map column tags to elastic search my_tags and use array type
tags="my_tags,list"
# This will map column keywords to elastic search keywords and use array type
keywords=",list"

# wildcard table rule, the wildcard table must be in source tables
[[rule]]
schema = "hoojjack"
table = "test_river_[0-9]{4}"
index = "river"
type = "river"

# title is MySQL test_river field name, es_title is the customized name in Elasticsearch
[[rule.fields]]
mysql = "title"
elastic = "es_title"

六、执行go-mysql-elasticsearch同步，首先进入 go-mysql-elasticsearch 路径，然后执行 ： ./bin/go-mysql-elasticsearch -config=./etc/river.toml

cd $GOPATH/src/github.com/siddontang/go-mysql-elasticsearch
./bin/go-mysql-elasticsearch -config=./etc/river.toml

效果如下：

mysql> select * from test_river;
+----+-----------+-----------------+--------+
| id | name      | title           | number |
+----+-----------+-----------------+--------+
|  1 | hoojjack  | wo shi yi bing  |      1 |
|  2 | xunchuan  | ni shi er ren   |      2 |

七、出现的问题：

1、mysqldump: command not found

mysql的mysqldump命令没有找到，解决办法：创建软连接，查看mysqldump是在哪个路径下：find / -name mysqldump 会获得路径，然后创建软连接：ln -s /usr/local/mysql/bin/mysqldump /usr/bin

这是由于系统默认会查找/usr/bin下的命令，如果这个命令不在这个目录下，当然会找不到命令

2、make build go-mysql-elasticsearch项目的时候出现问题

【Reference】

[1] http://blog.csdn.net/laoyang360/article/details/51771483