您的位置：首页 > 编程语言 > Java开发

从零学Elasticsearch系列——JAVA API操作

2019-01-09 11:18 330 查看

系列文章：

参考资料：https://spring.io/projects/spring-data-elasticsearch#overview

测试代码GitHub地址：https://github.com/gaozhy520/es_springdata_demo

查询方式

Restful API

基于http协议，使用JSON为数据交换格式，通过9200端口的与Elasticsearch进行通信，您可使用
```
HttpClient
```
类库通过9200端口操作Elasticsearch。
JAVA API(Spring Data ElasticSearch)

Spring Data ElasticSearch封装了与ES交互的实现细节，可以使系统开发者以Spring Data Repository 风格实现与ES的数据交互。Elasticsearch为Java用户提供了两种内置客户端：

节点客户端(node client)：
节点客户端以无数据节点(none data node)身份加入集群，换言之，它自己不存储任何数据，但是它知道数据在集群中的具体位置，并且能够直接转发请求到对应的节点上。

传输客户端(Transport client)：
这个更轻量的传输客户端能够发送请求到远程集群。它自己不加入集群，只是简单转发请求给集群中的节点。两个Java客户端都通过9300端口与集群交互，使用Elasticsearch传输协议(Elasticsearch Transport Protocol)。集群中的节点之间也通过9300端口进行通信。如果此端口未开放，你的节点将不能组成集群。

Spring Data ElasticSearch实践

Maven依赖

<dependencies>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-elasticsearch</artifactId>
<version>3.1.3.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<version>5.1.3.RELEASE</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
</dependencies>

准备配置文件

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:elasticesearch="http://www.springframework.org/schema/data/elasticsearch"
xmlns:elasticsearch="http://www.springframework.org/schema/data/elasticsearch"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/data/elasticsearch http://www.springframework.org/schema/data/elasticsearch/spring-elasticsearch.xsd">

<!-- Spring data 自动扫描es repository接口，生成实现类 -->
<elasticsearch:repositories base-package="com.baizhi.es.dao"></elasticsearch:repositories>

<!-- ip:port换成具体的ip和端口，多个以逗号分隔 -->
<elasticesearch:transport-client id="client" cluster-name="elasticsearch"
cluster-nodes="192.168.23.143:9300"></elasticesearch:transport-client>

<!-- es操作对象-->
<bean id="elasticsearchTemplate" class="org.springframework.data.elasticsearch.core.ElasticsearchTemplate">
<constructor-arg name="client" ref="client"></constructor-arg>
</bean>

<bean id="customUserRepository" class="com.baizhi.es.dao.CustomUserRepositoryImpl">

</bean>
</beans>

准备映射实体类

package com.baizhi.entity;

import org.springframework.data.annotation.Id;
import org.springframework.data.annotation.PersistenceConstructor;
import org.springframework.data.elasticsearch.annotations.Document;

import java.util.Date;

/**
* @author gaozhy
* @date 2018/12/28.17:05
*/
// 文档注解 用于描述索引及其相关信息
@Document(indexName = "zpark",type = "user")
public class User {

// 主键
@Id
private String id;

private String name;

private String realname;

private Integer age;

private Double salary;

private Date birthday;

// 指定address域的类型 并明确索引和检索使用的分词器（需安装IK分词器）
@Field(type = FieldType.Text,searchAnalyzer = "ik_max_word",analyzer = "ik_max_word")
private String address;

// 省略get/set toString方法 ......
}

spring data repository

spring data elsaticsearch提供了三种构建查询模块的方式：

基本的增删改查：继承spring data提供的接口就默认提供
接口中声明方法：无需实现类，spring data根据方法名，自动生成实现类，方法名必须符合一定的规则（这里还扩展出一种忽略方法名，根据注解的方式查询）
自定义repository：在实现类中注入elasticsearchTemplate，实现上面两种方式不易实现的查询（例如：聚合、分组、深度翻页等）

上面的第一点和第二点只需要声明接口，无需实现类，spring data会扫描并生成实现类

基础的repository接口：提供基本的增删改查和根据方法名的查询

package com.baizhi.es.dao;

import com.baizhi.entity.User;
import org.springframework.data.elasticsearch.annotations.Query;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;

import java.util.List;

/**
* 基础操作的es repository接口（定义的有通用的增删改查方法）
*
* @author gaozhy
* @date 2018/12/29.9:26
*/
public interface UserRepository extends ElasticsearchRepository<User,String> {

/**
* 根据年龄区间查询数据 并根据年龄降序排列
*/
public List<User> findByAgeBetweenOrderByAgeDesc(int start,int end);

/**
* 查询真实姓名已“王”开头的数据
*/
public List<User> findByRealnameStartingWith(String startStr);

/**
* 通过Query注解自定义查询表达式
*/
@Query("{\"bool\" : {\"must\" : {\"fuzzy\" : {\"name\" : \"?0\"}}}}")
public List<User> findByNameLike(String name);
}

测试代码如下：

package com.baizhi.es.test;

import com.baizhi.entity.User;
import com.baizhi.es.dao.UserRepository;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;

import org.springframework.data.domain.Sort;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

import java.util.Date;
import java.util.List;
import java.util.Optional;

/**
* @author gaozhy
* @date 2018/12/29.9:30
*/
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("classpath:applicationContext-es.xml")
public class UserRepositoryTest {
//=============================ElasticsearchRepository接口方法测试====================
@Autowired
private UserRepository userRepository;

/**
* 查所有
*/
@Test
public void testQueryAll(){
Iterable<User> users = userRepository.findAll();
for (User user : users) {
System.out.println(user);
}
}

/**
* 查询所有 并根据年龄倒序排列
*/
@Test
public void testQueryBySort(){
Iterable<User> users = userRepository.findAll(Sort.by(Sort.Direction.DESC, "age"));
for (User user : users) {
System.out.println(user);
}
}

/**
* 根据id查询
*/
@Test
public void testQueryById(){
Optional<User> user = userRepository.findById("1");
System.out.println(user.get());
}

/**
* 新增或者修改数据
*/
@Test
public void testAdd(){
User user = userRepository.save(new User("6", "wb", "王八", 26, 10000D, new Date(), "河南省郑州市二七区德化街南路33号"));
System.out.println(user);
}

//================================自定义方法==================================

/**
* 接口中声明方法查询：
*    根据年龄区间查询数据 并根据年龄降序排列
*/
@Test
public void testQueryByRange(){
List<User> users = userRepository.findByAgeBetweenOrderByAgeDesc(20, 28);
users.forEach(user -> System.out.println(user));
}

/**
* 接口中声明方法查询：
*    查询真实姓名已“王”开头的数据
*
*    响应结果：
*    User{id='6', name='wb', realname='王八', age=26, salary=10000.0, birthday=Sat Dec 29 14:38:39 CST 2018, address='河南省郑州市二七区德化街南路33号'}
User{id='3', name='ww', realname='王五', age=25, salary=4300.0, birthday=Tue Mar 15 08:00:00 CST 2016, address='北京市海淀区中关村大街新中关商城2楼511室'}
*/
@Test
public void testQueryByPrefix(){
List<User> users = userRepository.findByRealnameStartingWith("王");
users.forEach(user -> System.out.println(user));
}

//==================================================================
/**
* 通过Query注解自定义查询表达式
*/
@Test
public void testQueryByNameLike(){
List<User> users = userRepository.findByNameLike("zs");
users.forEach(user -> System.out.println(user));
}
}

自定义Repository接口：使用

elasticsearchTemplate

实现复杂查询

自定义

CustomUserRepository

接口

package com.baizhi.es.dao;

import com.baizhi.entity.User;

import java.util.List;
import java.util.Map;

/**
* @author gaozhy
* @date 2019/1/1.23:10
*/
public interface CustomUserRepository {

public List<User> findByPageable(int nowPage,int pageSize);

public List<User> findByFieldDesc(String field);

public List<User> findByRealNameLikeAndHighLight(String realName);

public List<User> findByNameWithTermFilter(String ...terms);

public List<User> findByAgeWithRangeFilter(int start,int end);

public Map findByNameStartingWithAndAggregations(String prefixName);

/**
* 嵌套查询：
*
* 先按年龄直方图（桶聚合）统计
* 然后再统计区间内员工的最高工资（度量聚合）
*/
public Map aggregationsWithHistogramAndMax();

/**
* 日期直方图（桶聚合）
*/
public Map aggregationsWithDateHistogram();

}

自定义

CustomUserRepositoryImpl

实现类

package com.baizhi.es.dao;

import com.baizhi.entity.User;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.aggregations.Aggregation;
import org.elasticsearch.search.aggregations.AggregationBuilders;
import org.elasticsearch.search.aggregations.Aggregations;
import org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramInterval;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.sort.SortBuilders;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.domain.PageRequest;
import org.springframework.data.domain.Pageable;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import org.springframework.data.elasticsearch.core.ResultsExtractor;
import org.springframework.data.elasticsearch.core.SearchResultMapper;
import org.springframework.data.elasticsearch.core.aggregation.AggregatedPage;
import org.springframework.data.elasticsearch.core.aggregation.impl.AggregatedPageImpl;
import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder;
import org.springframework.data.elasticsearch.core.query.SearchQuery;

import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.Map;

import static org.elasticsearch.index.query.QueryBuilders.*;

/**
* @author gaozhy
* @date 2019/1/1.23:11
*/
public class CustomUserRepositoryImpl implements CustomUserRepository {

@Autowired
private ElasticsearchTemplate template;

/**
* ====================================
* {
* "query": {
* "match_all": {}
* },
* "from":1,     //从第几条开始    (从0开始)
* "size":1      //大小
* }
* ====================================
*
* @param nowPage
* @param pageSize
* @return
*/
@Override
public List<User> findByPageable(int nowPage, int pageSize) {
SearchQuery query = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery())
.withPageable(new PageRequest(nowPage - 1, pageSize))
.build();

return template.queryForList(query, User.class);
}

/**
* @param field
* @return
*/
@Override
public List<User> findByFieldDesc(String field) {
SearchQuery query = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery())
.withSort(SortBuilders.fieldSort(field).order(SortOrder.DESC))
.build();
return template.queryForList(query, User.class);
}

/**
* 高亮
*
* @param realName
* @return
*/
@Override
public List<User> findByRealNameLikeAndHighLight(String realName) {
SearchQuery query = new NativeSearchQueryBuilder()
.withQuery(matchQuery("realname", realName)
)
.withHighlightFields(new HighlightBuilder.Field("realname"))
.build();
AggregatedPage<User> users = template.queryForPage(query, User.class, new SearchResultMapper() {
@Override
public <T> AggregatedPage<T> mapResults(SearchResponse searchResponse, Class<T> aClass, Pageable pageable) {
ArrayList<User> users = new ArrayList<>();
SearchHits searchHits = searchResponse.getHits();
for (SearchHit searchHit : searchHits) {
if (searchHits.getHits().length <= 0) {
return null;
}
User user = new User();
user.setId(searchHit.getId());
// searchHit.getSourceAsMap().forEach((k, v) -> System.out.println(k + " " + v));
user.setName(searchHit.getSourceAsMap().get("name").toString());
user.setAddress(searchHit.getSourceAsMap().get("address").toString());
user.setAge(Integer.parseInt(searchHit.getSourceAsMap().get("age").toString()));
user.setBirthday(new Date(Long.parseLong(searchHit.getSourceAsMap().get("birthday").toString())));
user.setSalary(Double.parseDouble(searchHit.getSourceAsMap().get("salary").toString()));
String realname = searchHit.getHighlightFields().get("realname").fragments()[0].toString();
user.setRealname(realname);

users.add(user);
}
return new AggregatedPageImpl<T>((List<T>) users);
}
});
return users.getContent();
}

@Override
public List<User> findByNameWithTermFilter(String... terms) {
SearchQuery query = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery())
.withFilter(termsQuery("name",terms))
.build();
System.out.println(query.getFilter());
return template.queryForList(query,User.class);
}

@Override
public List<User> findByAgeWithRangeFilter(int start, int end) {
SearchQuery query = new NativeSearchQueryBuilder()
.withFilter(rangeQuery("age").gte(start).lte(end))
.build();
System.out.println(query.getQuery());
System.out.println(query.getFilter());
return template.queryForList(query,User.class);
}

@Override
public Map<String, Aggregation> findByNameStartingWithAndAggregations(String prefixName) {
SearchQuery query = new NativeSearchQueryBuilder()
.withQuery(prefixQuery("name",prefixName))
// result为度量聚合结果的别名
.addAggregation(AggregationBuilders.avg("result").field("age"))
.build();
Aggregations aggregations = template.query(query, new ResultsExtractor<Aggregations>() {
@Override
public Aggregations extract(SearchResponse searchResponse) {
Aggregations aggregations = searchResponse.getAggregations();
return aggregations;
}
});
Map<String, Aggregation> map = aggregations.getAsMap();
return map;
}

@Override
public Map aggregationsWithHistogramAndMax() {
SearchQuery query = new NativeSearchQueryBuilder()
.addAggregation(AggregationBuilders.histogram("result").field("age").interval(5)
.subAggregation(AggregationBuilders.max("max_salary").field("salary")))
.build();
Aggregations aggregations = template.query(query, new ResultsExtractor<Aggregations>() {

@Override
public Aggregations extract(SearchResponse searchResponse) {
return searchResponse.getAggregations();
}
});
return aggregations.getAsMap();
}

@Override
public Map aggregationsWithDateHistogram() {
SearchQuery query = new NativeSearchQueryBuilder()
.addAggregation(AggregationBuilders.dateHistogram("result").field("birthday").format("yyyy-MM-dd").dateHistogramInterval(DateHistogramInterval.YEAR))
.build();
Aggregations aggregations = template.query(query, new ResultsExtractor<Aggregations>() {

@Override
public Aggregations extract(SearchResponse searchResponse) {
return searchResponse.getAggregations();
}
});
return aggregations.getAsMap();
}
}

自定义

CustomUserRepositoryTest

测试类

package com.baizhi.es.test;

import com.baizhi.entity.User;
import com.baizhi.es.dao.CustomUserRepository;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

import java.util.List;
import java.util.Map;

/**
* @author gaozhy
* @date 2019/1/1.23:26
*/
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("classpath:applicationContext-es.xml")
public class CustomUserRepositoryTest {

@Autowired
private CustomUserRepository repository;

@Test
public void testQueryByPage(){
List<User> users = repository.findByPageable(0, 2);
users.forEach(user -> {
System.out.println(user);
});
}
@Test
public void testQueryBySort(){
List<User> users = repository.findByFieldDesc("_id");
users.forEach(user -> {
System.out.println(user);
});
}
@Test
public void testQueryByHighLight(){
List<User> users = repository.findByRealNameLikeAndHighLight("王八");
users.forEach(user -> {
System.out.println(user);
});
}
@Test
public void testQueryByNameWithTermFilter(){
List<User> users = repository.findByNameWithTermFilter("zs","ls");
users.forEach(user -> {
System.out.println(user);
});
}

@Test
public void testQueryByAgeWithRangeFilter(){
List<User> users = repository.findByAgeWithRangeFilter(21,30);
users.forEach(user -> {
System.out.println(user);
});
}

@Test
public void testQueryByNameStartingWithAndAggregations(){
Map map = repository.findByNameStartingWithAndAggregations("z");
System.out.println(map.get("result"));
}

@Test
public void testAggregationsWithHistogramAndMax(){
Map map = repository.aggregationsWithHistogramAndMax();
System.out.println(map.get("result"));
}

@Test
public void testAggregationsWithDateHistogram(){
Map map = repository.aggregationsWithDateHistogram();
System.out.println(map.get("result"));
}
}

内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理

标签：

相关文章推荐

新的分享

章节导航

从零学Elasticsearch系列——JAVA API操作

查询方式

Spring Data ElasticSearch实践

Maven依赖

准备配置文件

准备映射实体类

spring data repository

自定义CustomUserRepository接口

自定义CustomUserRepositoryImpl实现类

自定义CustomUserRepositoryTest测试类

自定义
CustomUserRepository
接口

自定义
CustomUserRepositoryImpl
实现类

自定义
CustomUserRepositoryTest
测试类