elasticsearch api client使用

一、基本概念

1、Node 与 Cluster

Elastic 本质上是一个分布式数据库，允许多台服务器协同工作，每台服务器可以运行多个 Elastic 实例。

单个 Elastic 实例称为一个节点（node）。一组节点构成一个集群（cluster）。

2、Index

Elastic 会索引所有字段，经过处理后写入一个反向索引（Inverted Index）。查找数据的时候，直接查找该索引。

所以，Elastic 数据管理的顶层单位就叫做 Index（索引）。它是单个数据库的同义词。每个 Index （即数据库）的名字必须是小写。

下面的命令可以查看当前节点的所有 Index。

$ curl -X GET 'http://localhost:9200/_cat/indices?v'

3、Document

Index 里面单条的记录称为 Document（文档）。许多条 Document 构成了一个 Index。

Document 使用 JSON 格式表示，下面是一个例子。

{
"user": "张三",
"title": "工程师",
"desc": "数据库管理"
}

同一个 Index 里面的 Document，不要求有相同的结构（scheme），但是最好保持相同，这样有利于提高搜索效率。

4、Type

Document 可以分组，比如weather这个 Index 里面，可以按城市分组（北京和上海），也可以按气候分组（晴天和雨天）。这种分组就叫做 Type，它是虚拟的逻辑分组，用来过滤 Document。

不同的 Type 应该有相似的结构（schema），举例来说，id字段不能在这个组是字符串，在另一个组是数值。这是与关系型数据库的表的一个区别。性质完全不同的数据（比如products和logs）应该存成两个 Index，而不是一个 Index 里面的两个 Type（虽然可以做到）。

下面的命令可以列出每个 Index 所包含的 Type。

$ curl 'localhost:9200/_mapping?pretty=true'

根据规划，Elastic 6.x 版只允许每个 Index 包含一个 Type，7.x 版将会彻底移除 Type。

以上部分摘自：http://www.ruanyifeng.com/blog/2017/08/elasticsearch.html

二、通过Http请求排序

1、数据格式

{
"_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAATaBwFklfYTRhdy0wVHJxQUNpcm5sWVBHeHcAAAAAAEvhqhYwNTgtVi1xT1FUNlkxMl9CVldWM1lnAAAAAACXzBgWVlhBRnRfd2xRd09HdlduY2tRNXpmQQ==",
"took": 3,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"failed": 0
},
"hits": {
"total": 9564,
"max_score": 1,
"hits": [
{
"_index": "alert-201712s",
"_type": "HISTORY",
"_id": "000E94E15DA381A680F9C0E0C14F1E7F-1513323398",
"_score": 1,
"_source": {
"duration": 120,
"times": 2,
"status": "resolve",
"level": "warning",
"project": "AAAA"
}
},
{
"_index": "alert-201712s",
"_type": "HISTORY",
"_id": "00A70A194DCF6DE937BC97610715DDCE-1513320277",
"_score": 1,
"_source": {
"duration": 120,
"times": 54,
"level": "critical",
"project": "BBBB"
}
},
..........
]
}
}

想要先按照project聚合，再按照level聚合，再把聚合后的各个项目、各个level的duration求和（类似与sql中的select sum(duration) ….group by project,level）

2、聚合排序

通过postman请求：

请求方式：Post

url：

ip:9200/index名称/Type名称/_search

此处应该是：

localhost:9200/alert-201712s/HISTORY/_search

body参数：

{
"size": 0,
"query": {
"bool": {
"filter": {
"terms": {
"project": ["AAAA",
"BBBB"]
}
}
}
},
"aggs": {
"projects": {
"terms": {
"field": "project",
"size": 10000
},
"aggs": {
"levels": {
"terms": {
"field": "level"
},
"aggs": {
"durations": {
"sum": {
"field": "duration"
}
}
}
}
}
}
}
}

body参数注意aggs的嵌套结构（层级）

查询结果：

{
"took": 3,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"failed": 0
},
"hits": {
"total": 8768,
"max_score": 0,
"hits": []
},
"aggregations": {
"types_count": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "AAA",
"doc_count": 2077,
"types_count": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "serious",
"doc_count": 789,
"durations": {
"value": 18720
}
},
{
"key": "null",
"doc_count": 456,
"durations": {
"value": 23
}
},
{
"key": "warning",
"doc_count": 401,
"durations": {
"value": 234
}
},
{
"key": "critical",
"doc_count": 4,
"durations": {
"value": 78
}
}
]
}
},
{
"key": "BBB",
"doc_count": 1225,
"types_count": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "serious",
"doc_count": 966,
"durations": {
"value": 56
}
},
{
"key": "null",
"doc_count": 258,
"durations": {
"value": 34
}
},
{
"key": "critical",
"doc_count": 1,
"durations": {
"value": 2343
}
}
]
}
}
}
}

三、java http 请求

1、pom依赖

<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>5.6.4</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
<version>5.1.1</version>
</dependency>

2、代码

import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.apache.http.Header;
import org.apache.http.HttpHost;
import org.apache.http.entity.StringEntity;
import org.apache.http.message.BasicHeader;
import org.apache.http.util.EntityUtils;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.client.Response;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.transport.TransportClient;

import java.util.HashMap;
import java.util.Map;

public class Test {

private static String es_url = "localhost:9200";
private TransportClient client;
private IndexRequest source;

//将postman中参数直接复制到idea中自动转义的
private static String str = "{\n" +
"\t\"size\": 0,\n" +
"\t\"query\": {\n" +
"\t\t\"bool\": {\n" +
"\t\t\t\"filter\": {\n" +
"\t\t\t\t\"terms\": {\n" +
"\t\t\t\t\t\"project\": [\"AA\",\n" +
"\t\t\t\t\t\"BB\"]\n" +
"\t\t\t\t}\n" +
"\t\t\t}\n" +
"\t\t}\n" +
"\t},\n" +
"\t\"aggs\": {\n" +
"\t\t\"projects\": {\n" +
"\t\t\t\"terms\": {\n" +
"\t\t\t\t\"field\": \"project\",\n" +
"\t\t\t\t\"size\": 10000\n" +
"\t\t\t},\n" +
"\t\t\t\"aggs\": {\n" +
"\t\t\t\t\"levels\": {\n" +
"\t\t\t\t\t\"terms\": {\n" +
"\t\t\t\t\t\t\"field\": \"level\",\n" +
"\t\t\t\t\t\t\"size\": 10000\n" +
"\t\t\t\t\t},\n" +
"\t\t\t\t\t\"aggs\": {\n" +
"\t\t\t\t\t\t\"durations\": {\n" +
"\t\t\t\t\t\t\t\"sum\": {\n" +
"\t\t\t\t\t\t\t\t\"field\": \"duration\"\n" +
"\t\t\t\t\t\t\t}\n" +
"\t\t\t\t\t\t}\n" +
"\t\t\t\t\t}\n" +
"\t\t\t\t}\n" +
"\t\t\t}\n" +
"\t\t}\n" +
"\t}\n" +
"}";

public static void main(String[] args) throws Exception {
HttpHost[] hosts = new HttpHost[1];
hosts[0] = HttpHost.create(es_url);
//创建ES请求客户端
RestClient restClient = RestClient.builder(hosts).build();
String index = "alert-201712s";
String type = "HISTORY";
String endpoint = "/" + index + "/" + type + "/_search";
Map params = new HashMap();
StringEntity queryBody = new StringEntity(str, "UTF-8");
Header header = new BasicHeader("content-type", "application/json");

Response response = restClient.performRequest("GET", endpoint, params, queryBody, header);
//System.out.println(response);

String resultJson = EntityUtils.toString(response.getEntity());
Gson gson = new Gson();
//获取到返回的数据
JsonObject resultObj = gson.fromJson(resultJson, JsonObject.class);

}

}

个人微信公众号：



作者：jiankunking 出处：http://blog.csdn.net/jiankunking