您的位置：首页 > 其它

ES技术团队划重点 | ES5.X，你必须知道的API和相关技巧

2017-12-21 15:51 681 查看


题记

Elasticsearch5.X相关核心知识点必知必会（如下）。


0. ES相关推荐

首先，不要再使用curl，请安装sense（kibana5.x中默认包含sense） 

1）ES官方向导 
https://www.elastic.co/guide/en/elasticsearch/guide/master/index.html

2）ES官方文档（API相关） 
https://www.elastic.co/guide/en/elasticsearch/reference/master/index.html

3）ES资源清单（全） 
https://github.com/dzharii/awesome-elasticsearch

4）ES云 
https://www.found.no/play

5）ES官方论坛（英文） 
https://discuss.elastic.co/

6）ES|stackOverflow论坛地址 
https://stackoverflow.com/questions/tagged/elasticsearch

7）ES 性能测试相关（NB） 
https://www.datadoghq.com/blog/monitor-elasticsearch-performance-metrics/


1. 查询操作

基本查询有两种语法：左边是一个简单的语法，你不能使用任何选项， 

右边是一个扩展。 大多数初学者头痛的DSL来自于：
GET _search
{
"query": {
"match": {
"FIELD": "TEXT"
}
}
}
GET _search
{
"query": {
"match": {
"FIELD": {
"query": "TEXT",
"OPTION": "VALUE"
}
}
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19


1.1 包含高亮、聚合、过滤器的完整的检索示例

GET /_search
{
"query": {
"bool": {
"must": [
{
"match": {
"title": "smith"
}
}
],
"must_not": [
{
"match_phrase": {
"title": "granny smith"

4000
}
}
],
"filter": [
{
"exists": {
"field": "title"
}
}
]
}
},
"aggs": {
"my_agg": {
"terms": {
"field": "user",
"size": 10
}
}
},
"highlight": {
"pre_tags": [
"<em>"
],
"post_tags": [
"</em>"
],
"fields": {
"body": {
"number_of_fragments": 1,
"fragment_size": 20
},
"title": {}
}
},
"size": 20,
"from": 100,
"_source": [
"title",
"id"
],
"sort": [
{
"_id": {
"order": "desc"
}
}
]
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64


1.2 普通检索

多字段检索
"multi_match": {
"query": "Elastic",
"fields": ["user.*", "title^3"],
"type": "best_fields"
}
1
2
3
4
5

bool检索
"bool": {
"must": [],
"must_not": [],
"filter": [],
"should": [],
"minimum_should_match" : 1
}
1
2
3
4
5
6
7

范围检索
"range": {
"age": {
"gte": 10,
"lte": 20,
"boost": 2
}
}
1
2
3
4
5
6
7


1.3 QueryString语法概述


1.3.1 检索所有的_all字段

GET /_search?q=pony
1


1.3.2 包含运算符和包含boost精确检索的复杂检索

GET /_search?q=title:(joli OR code) AND author:"Damien Alexandre"^2
1


1.3.3 使用通配符和特殊查询进行检索

GET /_search?q=_exists_:title OR title:singl? noneOrAnyChar*cter
1


1.3.4 模糊搜素和范围检索

GET /_search?q=title:elastichurch~3 AND date:[2016-01-01 TO 2018-12-31]
1


1.3.5 使用 DSL检索（不推荐用于用户搜索）：

GET /_search
{
"query": {
"query_string": {
"default_field": "content",
"query": "elastic AND (title:lucene OR title:solr)"
}
}
}
1
2
3
4
5
6
7
8
9


2. 索引操作


2.1 创建包含设置和mapping的索引

PUT /my_index_name
{
"settings": {
"number_of_replicas": 1,
"number_of_shards": 3,
"analysis": {},
"refresh_interval": "1s"
},
"mappings": {
"my_type_name": {
"properties": {
"title": {
"type": "text",
"analyzer": "english"
}
}
}
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19


2.2 动态的更新设置

PUT /my_index_name/_settings
{
"index": {
"refresh_interval": "-1",
"number_of_replicas": 0
}
}
1
2
3
4
5
6
7


2.3 通过向类型添加字段更新索引

PUT /my_index_name/_mapping/my_type_name
{
"my_type_name": {
"properties": {
"tag": {
"type": "keyword"
}
}
}
}
1
2
3
4
5
6
7
8
9
10


2.4 获取Mapping和设置

GET /my_index_name/_mapping
GET /my_index_name/_settings
1
2


2.5 创建document

POST /my_index_name/my_type_name
{
"title": "Elastic is funny",
"tag": [
"lucene"
]
}
1
2
3
4
5
6
7


2.6 创建或更新document

PUT /my_index_name/my_type_name/12abc
{
"title": "Elastic is funny",
"tag": [
"lucene"
]
}
1
2
3
4
5
6
7


2.7 删除文档

DELETE /my_index_name/my_type_name/12abc
1


2.8 打开或关闭索引已节约内存和CPU

POST /my_index_name/_close
POST /my_index_name/_open
1
2


2.9 移除和创建别名

POST /_aliases
{
"actions": [
{
"remove": {
"index": "my_index_name",
"alias": "foo"
}
},
{
"add": {
"index": "my_index_name",
"alias": "bar",
"filter" : { "term" : { "user" : "damien" } }
}
}
]
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18


2.10 列举别名

GET /_aliases
GET /my_index_name/_alias/*
GET /*/_alias/*
GET /*/_alias/foo
1
2
3
4


2.11 索引监控和信息

GET /my_index_name/_stats
GET /my_index_name/_segments
GET /my_index_name/_recovery?pretty&human
1
2
3


2.12 索引状态和管理

POST /my_index_name/_cache/clear
POST /my_index_name/_refresh
POST /my_index_name/_flush
POST /my_index_name/_forcemerge
POST /my_index_name/_upgrade
GET /my_index_name/_upgrade?pretty&human
1
2
3
4
5
6


3. 调试和部署


3.1 检索调试


3.1.1 获取query操作到底做了什么？

GET /blog/post/_validate/query?explain
{
"query": {
"match": {
"title": "Smith"
}
}
}
1
2
3
4
5
6
7
8


3.1.2 获取文档是否匹配？

GET /blog/post/1/_explain
{
"query": {
"match": {
"title": "Smith"
}
}
}
1
2
3
4
5
6
7
8


3.2 分析


3.2.1 测试内容如何在文档中被标记？

GET /blog/_analyze?field=title&text=powerful
1


3.2.2 测试分析器输出？

GET /_analyze?analyzer=english&text=powerful
1


3.3 集群管理和插件管理


3.3.1 集群和节点信息

GET /_cluster/health?pretty
GET /_cluster/health?wait_for_status=yellow&timeout=50s
GET /_cluster/state
GET /_cluster/stats?human&pretty
GET /_cluster/pending_tasks
GET /_nodes
GET /_nodes/stats
GET /_nodes/nodeId1,nodeId2/stats
1
2
3
4
5
6
7
8


3.3.2 手动移动分片

索引1的分片移动到索引2
POST /_cluster/reroute
{
"commands": [
{
"move": {
"index": "my_index_name",
"shard": 0,
"from_node": "node1",
"to_node": "node2"
}
},
{
"allocate": {
"index": "my_index_name",
"shard": 1,
"node": "node3"
}
}
]
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20


3.3.3 更新设置

动态更新最小节点数。
PUT /_cluster/settings
{
"persistent": {
"discovery.zen.minimum_master_nodes": 3
}
}

PUT /_cluster/settings
{
"transient": {
"discovery.zen.minimum_master_nodes": 2
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13

使分片失效，在rolling重启前有效。
PUT /_cluster/settings
{
"transient" : {
"cluster.routing.allocation.enable" : "none"
}
}

PUT /_cluster/settings
{
"transient" : {
"cluster.routing.allocation.enable" : "all"
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13


4.最有用的插件

ES5.X版本已不再支持站点插件，请查看kibana应用或类似Cerebro等其他独立的app做管理。 
Cerebro地址： https://github.com/lmenezes/cerebro

Analysis ICU 

从Unicode ICU库中添加有用的tokenizer和令牌过滤器。
bin/elasticsearch-plugin install analysis-icu
1

AWS Cloud 

允许在Amazon云（EC2和S3）中发现和存储。
bin / elasticsearch-plugin install discovery-ec2 bin / elasticsearch-plugin install repository-s3
1

Azure Cloud 

允许在微软云（EC2和S3）中发现和存储。
bin/elasticsearch-plugin install discovery-azure-classicbin/elasticsearch-plugin install repository-azure
1

插件管理：
bin/elasticsearch-plugin install file:///path/to/plugin
bin/elasticsearch-plugin list
bin/elasticsearch-plugin remove [pluginname]
1
2
3


5.其他信息


5.1 如何发现其他插件

RPM: /usr/share/elasticsearch/bin
Debian: /usr/share/elasticsearch/bin
1
2


5.2 缺省端口

Kibana: http://localhost:5601/. 
Elasticsearch: http://localhost:9200/.[/code]1 2
3

5.3 如何设置堆大小
单一Elasticsearch 服务器最大可用内存小于总内存的50%，且小于32GB； 

假定你使用Ubuntu/Debian 服务器，你可以通过如下配置修改：
/etc/security/limits.conf

elasticsearch - nofile 65535
elasticsearch - memlock unlimited
1
2
3
4

对于Centos服务器，修改配置如下：
/etc/default/elasticsearch (on CentOS/RH: /etc/sysconfig/elasticsearch)

ES_HEAP_SIZE=20g
MAX_OPEN_FILES=65535
MAX_LOCKED_MEMORY=unlimited
1
2
3
4
5

5.4 elasticsearch.yml中有用的需要改变的配置
cluster.name: jolicluster
node.name: ${HOSTNAME}
discovery.zen.ping.unicast.hosts: ["front01", "front02"]
discovery.zen.minimum_master_nodes: 2
network.host: _site_
network.bind_host: [_site_, _local_]
plugin.mandatory: analysis-icu
node.data: true
node.master: true
bootstrap.memory_lock: true
action.auto_create_index: +aaa*,-bbb*,+ccc*,-*
1
2
3
4
5
6
7
8
9
10
11

6. 小结
本文是翻译的 http://elasticsearch-cheatsheet.jolicode.com/#es5。

Cheat sheet是作弊表的意思。可见源作者对文章寄托的意义就是最核心、最关键的ES知识点“小抄”。

我把它理解成ES学习中待翻看的字典。

尤其第0章节的相关链接，都是上乘ES佳作，待进一步深度剖析。
内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理 点击这里给我发消息
标签： 
相关文章推荐
新的分享
章节导航