Elasticsearch:通过 Spring boot 创建 REST APIs 来访问 Elasticsearch
2021年10月18日 | by mebius
在我之前的文章 “Elasticsearch:Java 运用示例” 我讲述了在客户端如何使用 Elasticsearch 所提供的 API 来访问 Elasticsearch 的数据。在很多的应用场景中,这是非常有效的一种方法。但是,如果我们的 Elasticsearch 由于升级的缘故,那么 API 的使用可能有所变化。这对于一些场景来说,并不是一种很好的方案。在实际的使用中,我们可以使用一个 API gateway 来提供一个标准的接口。这样如果 Elasticsearch 有升级而造成的 API 的变化,那么我们直接升级这个 API gateway 即可:
如上所示,我们使用 Spring Boot 来创建一个 API gateway 来访问 Elasticsearch。特别指出的是:我们也可以使用其它的语言框架来完成这个操作,并不一定要局限于 Spring Boot。
为了方便大家理解下面的内容,我把最后的代码放到 github 上:
创建 REST API 接口
创建最基本的 Spring Boot 框架
我们接下来使用我们喜欢的 IDE 来创建一个最基本的 Spring Boot 应用。为了能够使得应用不和本地的 8080 端口想冲突,我们重新在 application.properties 里定义了如下的一个端口 9999。
application.properties
server.servlet.context-path=/hr
server.port = 9999
上面,我们定义了访问端口 9999。同时我们也定义了访问的路径 http://localhost:9999/hr。在这里,employees 是我们将要在 Elasticsearch 中定义的索引名称。
我们可以在 Kibana 中打入如的命令来创建一个叫做 employees 的索引:
POST employees/_bulk
{"index":{"_id":"1"}}
{"name":"张三","sex":"male","salary":"10000","occupation":"software developer"}
{"index":{"_id":"2"}}
{"name":"李四","sex":"female","salary":"20000","occupation":"manager"}
{"index":{"_id":"3"}}
{"name":"王五","sex":"male","salary":"90000","occupation":"test engineer"}
这样我们就创建了一个叫做 employees 的索引。这个索引将在我们如下的 Spring Boot REST 接口中将要用到。
我们使用自己喜欢的工具首先来创建一个最为基本的 Spring Boot 框架:
pom.xml
4.0.0
org.springframework.boot
spring-boot-starter-parent
2.5.5
com.liuxg
SpringBootElasticsearch
0.0.1-SNAPSHOT
SpringBootElasticsearch
SpringBootElasticsearch
1.8
7.10.0
org.springframework.boot
spring-boot-starter-thymeleaf
org.springframework.boot
spring-boot-starter-web
org.springframework.boot
spring-boot-devtools
runtime
true
org.springframework.boot
spring-boot-starter-test
test
org.elasticsearch.client
elasticsearch-rest-high-level-client
${elastic.version}
org.elasticsearch.client
elasticsearch-rest-client
${elastic.version}
org.elasticsearch
elasticsearch
${elastic.version}
org.springframework.boot
spring-boot-maven-plugin
employee.java
package com.liuxg.springbootelasticsearch.entity;
public class Employee {
private String name;
private String sex;
private String occupation;
private int salary;
public Employee(String name, String sex, String occupation, int salary) {
this.name = name;
this.sex = sex;
this.occupation = occupation;
this.salary = salary;
}
public String getName() {
return name;
}
public int getSalary() {
return salary;
}
public String getSex() {
return sex;
}
public String getOccupation() {
return occupation;
}
public void setName(String name) {
this.name = name;
}
public void setOccupation(String occupation) {
this.occupation = occupation;
}
public void setSalary(int salary) {
this.salary = salary;
}
public void setSex(String sex) {
this.sex = sex;
}
}
EmployeeRepository.java
package com.liuxg.springbootelasticsearch.repository;
import com.liuxg.springbootelasticsearch.entity.Employee;
import org.springframework.stereotype.Repository;
import java.util.List;
@Repository
public interface EmployeeRepository {
List findAllEmployeeDetailsFromES();
List findAllUserDataByNameFromES(String name);
List findAllUserDataByNameAndOccupationFromES(String name, String occupation);
}
EmployeeRepositoryImpl.java
package com.liuxg.springbootelasticsearch.repository;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.liuxg.springbootelasticsearch.entity.Employee;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import java.util.ArrayList;
import java.util.List;
@Component
public class EmployeeRepositoryImpl implements EmployeeRepository {
@Autowired
private ObjectMapper objectMapper;
@Override
public List findAllEmployeeDetailsFromES() {
Employee employee = new Employee("liuxg", "male", "engineer", 10000);
List list = new ArrayList();
list.add(employee);
return list;
}
@Override
public Ltgcodeist findAllUserDataByNameFromES(String name) {
return null;
}
@Override
public List findAllUserDataByNameAndOccupationFromES(String name, String occupation) {
return null;
}
}
EmployeeController.java
package com.liuxg.springbootelasticsearch.controller;
import com.liuxg.springbootelasticsearch.entity.Employee;
import com.liuxg.springbootelasticsearch.service.EmployeeService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.bind.annotation.PathVariable;
import java.util.ArrayList;
import java.util.List;
@RestController
@RequestMapping(value = "employee")
public class EmployeeController {
@Autowired
private EmployeeService employeeService;
@GetMapping(value ="/allemployees", produces = MediaType.APPLICATION_JSON_VALUE)
public List getAllEmployees() {
return employeeService.getAllEmployeeInfo();
}
@GetMapping(value ="/allemployees/{name}", produces = MediaType.APPLICATION_JSON_VALUE)
public List getUserByName(@PathVariable String name){
return employeeService.getEmployeesByName(name);
}
@GetMapping(value ="/allemployees/{name}/{address}", produces = MediaType.APPLICATION_JSON_VALUE)
public List getUserByNameAndAddress(@PathVariable String name, @PathVariable String address){
return employeeService.getEmployeesByNameAndOccupation(name, address);
}
}
EmployeeService.java
package com.liuxg.springbootelasticsearch.service;
import com.liuxg.springbootelasticsearch.entity.Employee;
import com.liuxg.springbootelasticsearch.repository.EmployeeRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.util.List;
@Service
public class EmployeeService {
@Autowired
private EmployeeRepository employeeRepository;
public List getAllEmployeeInfo() {
return employeeRepository.findAllEmployeeDetailsFromES();
}
public List getEmployeesByName(String name) {
return employeeRepository.findAllUserDataByNameFromES(name);
}
public List getEmployeesByNameAndOccupation(String name, String occupation) {
return employeeRepository.findAllUserDataByNameAndOccupationFromES(name, occupation);
}
}
SpringBootElasticsearchApplication.java
package com.liuxg.springbootelasticsearch;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class SpringBootElasticsearchApplication {
public static void main(String[] args) {
SpringApplication.run(SpringBootElasticsearchApplication.class, args);
}
}
整个项目的结构如下:
$ tree
.
├── README.md
├── pom.xml
└── src
├── main
│ ├── java
│ │ └── com
│ │ └── liuxg
│ │ └── springbootelasticsearch
│ │ ├── SpringBootElasticsearchApplication.java
│ │ ├── controller
│ │ │ └── EmployeeController.java
│ │ ├── entity
│ │ │ └── Employee.java
│ │ ├── repository
│ │ │ ├── EmployeeRepository.java
│ │ │ └── EmployeeRepositoryImpl.java
│ │ └── service
│ │ └── EmployeeService.java
│ └── resources
│ ├── application.properties
│ ├── static
│ └── templates
└── test
└── java
└── com
└── liuxg
└── springbootelasticsearch
└── SpringBootElasticsearchApplicationTests.java
我们可以运行上面的最为基本的 Spring Boot 应用,并使用浏览器来查看我们的接口是否正确:
我们也可以使用喜欢的测试 REST j接口的工具比如 PostMan 来进行查看。
为了方便大家阅读,我已经把这个最基本的应用上传到 github:GitHub – liu-xiao-guo/SpringBootElasticsearch。我们可以通过如下的方式来得到代码:
git clone https://github.com/liu-xiao-guo/SpringBootElasticsearch
我们在接下来的练习中继续使用这个向下进行。
在接口中访问 Elasticsearch 进行查询
在上面的设计中,我们的需要实现访问的部分代码是这个:
public class EmployeeRepositoryImpl implements EmployeeRepository {
@Autowired
private ObjectMapper objectMapper;
@Override
public List findAllEmployeeDetailsFromES() {
Employee employee = new Employee("liuxg", "male", "engineer", 10000);
List list = new ArrayList();
list.add(employee);
return list;
}
@Override
public List findAllUserDataByNameFromES(String name) {
return null;
}
@Override
public List findAllUserDataByNameAndOccupationFromES(String name, String occupation) {
return null;
}
}
我们需要实现上面的三个 method 来完成对 Elasticsearch 的访问。我们首先来实现
public List findAllEmployeeDetailsFromES()
我们首先在 Kibana 中针对 employees 来做一个如下的查询。我们查询所有的 employee:
我们可以看到返回的数据的结构。
由于在我的 Elasticsearch 中,我配置有安全,所以,我重新编写EmployeeRepositoryImpl.java 文件如下:
EmployeeRepositoryImpl.java
package com.liuxg.springbootelasticsearch.repository;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.liuxg.springbootelasticsearch.entity.Employee;
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;tgcode
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
@Component
public class EmtgcodeployeeRepositoryImpl implements EmployeeRepository {
@Autowired
private ObjectMapper objectMapper;
public EmployeeRepositoryImpl() {
makeConnection();
}
private static RestHighLevelClient client = null;
private static synchronized RestHighLevelClient makeConnection() {
final BasicCredentialsProvider basicCredentialsProvider = new BasicCredentialsProvider();
basicCredentialsProvider
.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("elastic", "password"));
if (client == null) {
client = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost", 9200, "http"))
.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
@Override
public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
httpClientBuilder.disableAuthCaching();
return httpClientBuilder.setDefaultCredentialsProvider(basicCredentialsProvider);
}
})
);
}
return client;
}
private static synchronized void closeConnection() throws IOException {
client.close();
client = null;
}
@Override
public List findAllEmployeeDetailsFromES() {
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("employees");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.matchAllQuery());
searchRequest.source(searchSourceBuilder);
List list = new ArrayList();
SearchResponse searchResponse = null;
try {
searchResponse =client.search(searchRequest, RequestOptions.DEFAULT);
if (searchResponse.getHits().getTotalHits().value > 0) {
SearchHit[] searchHit = searchResponse.getHits().getHits();
for (SearchHit hit : searchHit) {
Map map = hit.getSourceAsMap();
list.add(objectMapper.convertValue(map, Employee.class));
}
}
} catch (IOException e) {
e.printStackTrace();
}
return list;
}
@Override
public List findAllUserDataByNameFromES(String name) {
return null;
}
@Override
public List findAllUserDataByNameAndOccupationFromES(String name, String occupation) {
return null;
}
}
请注意在我的 Elasticsearch 中,我设置超级用户 elastic 用户的密码为 password。
我们重新运行 Spring Boot 应用,并在浏览器中发送请求:
http://localhost:9999/hr/employee/allemployees
从上面的显示中,我们可以看出来它从 Elasticsearch 中获得所有的 3 个文档。
接下来,我们来完成如下的方法:
public List findAllUserDataByNameFromES(String name)
@Override
public List findAllUserDataByNameFromES(String name) {
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("employees");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.boolQuery().must(QueryBuilders.termQuery("name.keyword", name)));
searchRequest.source(searchSourceBuilder);
List list = new ArrayList();
try {
SearchResponse searchResponse = null;
searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
if (searchResponse.getHits().getTotalHits().value > 0) {
SearchHit[] searchHit = searchResponse.getHits().getHits();
for (SearchHit hit : searchHit) {
Map map = hit.getSourceAsMap();
list.add(objectMapper.convertValue(map, Employee.class));
}
}
} catch (IOException e) {
e.printStackTrace();
}
return list;
}
重新运行 Spring Boot 应用,并在浏览器中输入:
http://localhost:9999/hr/employee/allemployees/%e5%bc%a0%e4%b8%89
上面的那些不可以认识的字符串是 “张三”。我们可以使用工具进行查看:
上面的请求的结果为:
从上面的请求中,我们可以看到搜索到张三的结果。这个是一个精确的匹配。
再接下来,我们来搜索 name 为 “张三” 并且 occupation 为 developer 的文档。我们来完成
public List findAllUserDataByNameAndOccupationFromES(String name, String occupation)
@Override
public List findAllUserDataByNameAndOccupationFromES(String name, String occupation) {
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("employees");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.boolQuery().must(QueryBuilders.termQuery("name.keyword", name))
.must(QueryBuilders.matchQuery("occupation", occupation)));
searchRequest.source(searchSourceBuilder);
List list = new ArrayList();
try {
SearchResponse searchResponse = null;
searchResponse =client.search(searchRequest, RequestOptions.DEFAULT);
if (searchResponse.getHits().getTotalHits().value > 0) {
SearchHit[] searchHit = searchResponse.getHits().getHits();
for (SearchHit hit : searchHit) {
Map map = hit.getSourceAsMap();
list.add(objectMapper.convertValue(map, Employee.class));
}
}
} catch (IOException e) {
e.printStackTrace();
}
return list;
}
重新运行 Sprint Boot 应用,并在浏览器中打入如下的 URL 请求:
http://localhost:9999/hr/employee/allemployees/%E5%BC%A0%E4%B8%89/developer
我们可以在浏览器中看到如下的结果。
从上面,我们可以看到有一个文档被搜索到了。上面的请求类似于 Kibana 中这样的搜索:
GET employees/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"name.keyword": "张三"
}
},
{
"match": {
"occupation": "developer"
}
}
]
}
}
}
它显示的结果为:
如果我们打入如下的请求:
我们看到一个空的返回。这是因为在我们的文档中,没有一个 name 叫做 “张三” 而且 occupation 是 engineer 的文档。
为了方便阅读,我把最终的代码放入如下的地址里:
git clone https://github.com/liu-xiao-guo/SpringBootElasticsearch
cd SpringBootElasticsearch
git checkout final
然后,你就可以看到整个项目的代码了。
文章来源于互联网:Elasticsearch:通过 Spring boot 创建 REST APIs 来访问 Elasticsearch
相关推荐: Elasticsearch:使用 Python 进行 Bulk insert 及 Scan
在本文章中,我将展示如何使用 Python 来对索引进行 Bulk写入。在单个 BulkAPI 调用中执行多个索引或删除操作。 这减少了开销并且可以大大提高索引速度。 在今天的展示中,我将使用 Jupyter 来进行演示。如果你对 Jupyter 还不是很熟的…