Elasticsearch:通过 Spring boot 创建 REST APIs 来访问 Elasticsearch

2021年10月18日   |   by mebius

在我之前的文章 “Elasticsearch:Java 运用示例” 我讲述了在客户端如何使用 Elasticsearch 所提供的 API 来访问 Elasticsearch 的数据。在很多的应用场景中,这是非常有效的一种方法。但是,如果我们的 Elasticsearch 由于升级的缘故,那么 API 的使用可能有所变化。这对于一些场景来说,并不是一种很好的方案。在实际的使用中,我们可以使用一个 API gateway 来提供一个标准的接口。这样如果 Elasticsearch 有升级而造成的 API 的变化,那么我们直接升级这个 API gateway 即可:

%title插图%num

如上所示,我们使用 Spring Boot 来创建一个 API gateway 来访问 Elasticsearch。特别指出的是:我们也可以使用其它的语言框架来完成这个操作,并不一定要局限于 Spring Boot。

为了方便大家理解下面的内容,我把最后的代码放到 github 上:

创建 REST API 接口

创建最基本的 Spring Boot 框架

我们接下来使用我们喜欢的 IDE 来创建一个最基本的 Spring Boot 应用。为了能够使得应用不和本地的 8080 端口想冲突,我们重新在 application.properties 里定义了如下的一个端口 9999。

application.properties

server.servlet.context-path=/hr
server.port = 9999

上面,我们定义了访问端口 9999。同时我们也定义了访问的路径 http://localhost:9999/hr。在这里,employees 是我们将要在 Elasticsearch 中定义的索引名称。

我们可以在 Kibana 中打入如的命令来创建一个叫做 employees 的索引:

POST employees/_bulk
{"index":{"_id":"1"}}
{"name":"张三","sex":"male","salary":"10000","occupation":"software developer"}
{"index":{"_id":"2"}}
{"name":"李四","sex":"female","salary":"20000","occupation":"manager"}
{"index":{"_id":"3"}}
{"name":"王五","sex":"male","salary":"90000","occupation":"test engineer"}

这样我们就创建了一个叫做 employees 的索引。这个索引将在我们如下的 Spring Boot REST 接口中将要用到。

我们使用自己喜欢的工具首先来创建一个最为基本的 Spring Boot 框架:

pom.xml



    4.0.0
    
        org.springframework.boot
        spring-boot-starter-parent
        2.5.5
         
    
    com.liuxg
    SpringBootElasticsearch
    0.0.1-SNAPSHOT
    SpringBootElasticsearch
    SpringBootElasticsearch
    
        1.8
        7.10.0
    
    
        
            org.springframework.boot
            spring-boot-starter-thymeleaf
        
        
            org.springframework.boot
            spring-boot-starter-web
        

        
            org.springframework.boot
            spring-boot-devtools
            runtime
            true
        
        
            org.springframework.boot
            spring-boot-starter-test
            test
        

        
            org.elasticsearch.client
            elasticsearch-rest-high-level-client
            ${elastic.version}
        
        
            org.elasticsearch.client
            elasticsearch-rest-client
            ${elastic.version}
        
        
            org.elasticsearch
            elasticsearch
            ${elastic.version}
        
    

    
        
            
                org.springframework.boot
                spring-boot-maven-plugin
            
        
    

employee.java

package com.liuxg.springbootelasticsearch.entity;

public class Employee {
    private String name;
    private String sex;
    private String occupation;
    private int salary;

    public Employee(String name, String sex, String occupation, int salary) {
        this.name = name;
        this.sex = sex;
        this.occupation = occupation;
        this.salary = salary;
    }

    public String getName() {
        return name;
    }

    public int getSalary() {
        return salary;
    }

    public String getSex() {
        return sex;
    }

    public String getOccupation() {
        return occupation;
    }

    public void setName(String name) {
        this.name = name;
    }

    public void setOccupation(String occupation) {
        this.occupation = occupation;
    }

    public void setSalary(int salary) {
        this.salary = salary;
    }

    public void setSex(String sex) {
        this.sex = sex;
    }
}

EmployeeRepository.java

package com.liuxg.springbootelasticsearch.repository;

import com.liuxg.springbootelasticsearch.entity.Employee;
import org.springframework.stereotype.Repository;

import java.util.List;

@Repository
public interface EmployeeRepository {

    List findAllEmployeeDetailsFromES();
    List findAllUserDataByNameFromES(String name);
    List findAllUserDataByNameAndOccupationFromES(String name, String occupation);

}

EmployeeRepositoryImpl.java

package com.liuxg.springbootelasticsearch.repository;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.liuxg.springbootelasticsearch.entity.Employee;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import java.util.ArrayList;
import java.util.List;

@Component
public class EmployeeRepositoryImpl implements EmployeeRepository {

    @Autowired
    private ObjectMapper objectMapper;

    @Override
    public List findAllEmployeeDetailsFromES() {
        Employee employee = new Employee("liuxg", "male", "engineer", 10000);
        List list = new ArrayList();
        list.add(employee);
        return list;
    }

    @Override
    public Ltgcodeist findAllUserDataByNameFromES(String name) {
        return null;
    }

    @Override
    public List findAllUserDataByNameAndOccupationFromES(String name, String occupation) {
        return null;
    }
}

EmployeeController.java

package com.liuxg.springbootelasticsearch.controller;

import com.liuxg.springbootelasticsearch.entity.Employee;
import com.liuxg.springbootelasticsearch.service.EmployeeService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.bind.annotation.PathVariable;

import java.util.ArrayList;
import java.util.List;

@RestController
@RequestMapping(value = "employee")
public class EmployeeController {

    @Autowired
    private EmployeeService employeeService;

    @GetMapping(value ="/allemployees", produces = MediaType.APPLICATION_JSON_VALUE)
    public List getAllEmployees() {
        return employeeService.getAllEmployeeInfo();
    }

    @GetMapping(value ="/allemployees/{name}", produces = MediaType.APPLICATION_JSON_VALUE)
    public List getUserByName(@PathVariable String name){
        return employeeService.getEmployeesByName(name);
    }

    @GetMapping(value ="/allemployees/{name}/{address}", produces = MediaType.APPLICATION_JSON_VALUE)
    public List getUserByNameAndAddress(@PathVariable String name, @PathVariable String address){
        return employeeService.getEmployeesByNameAndOccupation(name, address);
    }

}

EmployeeService.java

package com.liuxg.springbootelasticsearch.service;

import com.liuxg.springbootelasticsearch.entity.Employee;
import com.liuxg.springbootelasticsearch.repository.EmployeeRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.util.List;

@Service
public class EmployeeService {

    @Autowired
    private EmployeeRepository employeeRepository;

    public List getAllEmployeeInfo() {
        return employeeRepository.findAllEmployeeDetailsFromES();
    }

    public List getEmployeesByName(String name) {
        return employeeRepository.findAllUserDataByNameFromES(name);
    }

    public List getEmployeesByNameAndOccupation(String name, String occupation) {
        return employeeRepository.findAllUserDataByNameAndOccupationFromES(name, occupation);
    }
}

SpringBootElasticsearchApplication.java

package com.liuxg.springbootelasticsearch;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class SpringBootElasticsearchApplication {

    public static void main(String[] args) {
        SpringApplication.run(SpringBootElasticsearchApplication.class, args);
    }

}

整个项目的结构如下:

$ tree
.
├── README.md
├── pom.xml
└── src
    ├── main
    │ ├── java
    │ │ └── com
    │ │     └── liuxg
    │ │         └── springbootelasticsearch
    │ │             ├── SpringBootElasticsearchApplication.java
    │ │             ├── controller
    │ │             │ └── EmployeeController.java
    │ │             ├── entity
    │ │             │ └── Employee.java
    │ │             ├── repository
    │ │             │ ├── EmployeeRepository.java
    │ │             │ └── EmployeeRepositoryImpl.java
    │ │             └── service
    │ │                 └── EmployeeService.java
    │ └── resources
    │     ├── application.properties
    │     ├── static
    │     └── templates
    └── test
        └── java
            └── com
                └── liuxg
                    └── springbootelasticsearch
                        └── SpringBootElasticsearchApplicationTests.java

我们可以运行上面的最为基本的 Spring Boot 应用,并使用浏览器来查看我们的接口是否正确:

%title插图%num

我们也可以使用喜欢的测试 REST j接口的工具比如 PostMan 来进行查看。

为了方便大家阅读,我已经把这个最基本的应用上传到 github:GitHub – liu-xiao-guo/SpringBootElasticsearch。我们可以通过如下的方式来得到代码:

git clone https://github.com/liu-xiao-guo/SpringBootElasticsearch

我们在接下来的练习中继续使用这个向下进行。

在接口中访问 Elasticsearch 进行查询

在上面的设计中,我们的需要实现访问的部分代码是这个:

public class EmployeeRepositoryImpl implements EmployeeRepository {

    @Autowired
    private ObjectMapper objectMapper;

    @Override
    public List findAllEmployeeDetailsFromES() {
        Employee employee = new Employee("liuxg", "male", "engineer", 10000);
        List list = new ArrayList();
        list.add(employee);
        return list;
    }

    @Override
    public List findAllUserDataByNameFromES(String name) {
        return null;
    }

    @Override
    public List findAllUserDataByNameAndOccupationFromES(String name, String occupation) {
        return null;
    }
}

我们需要实现上面的三个 method 来完成对 Elasticsearch 的访问。我们首先来实现

public List findAllEmployeeDetailsFromES()

我们首先在 Kibana 中针对 employees 来做一个如下的查询。我们查询所有的 employee:

%title插图%num

我们可以看到返回的数据的结构。

由于在我的 Elasticsearch 中,我配置有安全,所以,我重新编写EmployeeRepositoryImpl.java 文件如下:

EmployeeRepositoryImpl.java

package com.liuxg.springbootelasticsearch.repository;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.liuxg.springbootelasticsearch.entity.Employee;
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;tgcode
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;

@Component
public class EmtgcodeployeeRepositoryImpl implements EmployeeRepository {
    @Autowired
    private ObjectMapper objectMapper;

    public EmployeeRepositoryImpl() {
        makeConnection();
    }

    private static RestHighLevelClient client = null;

    private static synchronized RestHighLevelClient makeConnection() {
        final BasicCredentialsProvider basicCredentialsProvider = new BasicCredentialsProvider();
        basicCredentialsProvider
                .setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("elastic", "password"));

        if (client == null) {
            client = new RestHighLevelClient(
                    RestClient.builder(new HttpHost("localhost", 9200, "http"))
                            .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                                @Override
                                public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                                    httpClientBuilder.disableAuthCaching();
                                    return httpClientBuilder.setDefaultCredentialsProvider(basicCredentialsProvider);
                                }
                            })
            );
        }

        return client;
    }

    private static synchronized void closeConnection() throws IOException {
        client.close();
        client = null;
    }

    @Override
    public List findAllEmployeeDetailsFromES() {
        SearchRequest searchRequest = new SearchRequest();
        searchRequest.indices("employees");
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(QueryBuilders.matchAllQuery());
        searchRequest.source(searchSourceBuilder);
        List list = new ArrayList();
        SearchResponse searchResponse = null;
        try {
            searchResponse =client.search(searchRequest, RequestOptions.DEFAULT);
            if (searchResponse.getHits().getTotalHits().value > 0) {
                SearchHit[] searchHit = searchResponse.getHits().getHits();
                for (SearchHit hit : searchHit) {
                    Map map = hit.getSourceAsMap();
                    list.add(objectMapper.convertValue(map, Employee.class));
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return list;
    }

    @Override
    public List findAllUserDataByNameFromES(String name) {
        return null;
    }

    @Override
    public List findAllUserDataByNameAndOccupationFromES(String name, String occupation) {
        return null;
    }
}

请注意在我的 Elasticsearch 中,我设置超级用户 elastic 用户的密码为 password。

我们重新运行 Spring Boot 应用,并在浏览器中发送请求:

http://localhost:9999/hr/employee/allemployees

%title插图%num

从上面的显示中,我们可以看出来它从 Elasticsearch 中获得所有的 3 个文档。

接下来,我们来完成如下的方法:

public List findAllUserDataByNameFromES(String name)
@Override
    public List findAllUserDataByNameFromES(String name) {
        SearchRequest searchRequest = new SearchRequest();
        searchRequest.indices("employees");
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(QueryBuilders.boolQuery().must(QueryBuilders.termQuery("name.keyword", name)));
        searchRequest.source(searchSourceBuilder);
        List list = new ArrayList();

        try {
            SearchResponse searchResponse = null;
            searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
            if (searchResponse.getHits().getTotalHits().value > 0) {
                SearchHit[] searchHit = searchResponse.getHits().getHits();
                for (SearchHit hit : searchHit) {
                    Map map = hit.getSourceAsMap();
                    list.add(objectMapper.convertValue(map, Employee.class));
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return list;
    }

重新运行 Spring Boot 应用,并在浏览器中输入:

http://localhost:9999/hr/employee/allemployees/%e5%bc%a0%e4%b8%89

上面的那些不可以认识的字符串是 “张三”。我们可以使用工具进行查看:

%title插图%num

上面的请求的结果为:

%title插图%num

从上面的请求中,我们可以看到搜索到张三的结果。这个是一个精确的匹配。

再接下来,我们来搜索 name 为 “张三” 并且 occupation 为 developer 的文档。我们来完成

public List findAllUserDataByNameAndOccupationFromES(String name, String occupation)
    @Override
    public List findAllUserDataByNameAndOccupationFromES(String name, String occupation) {
        SearchRequest searchRequest = new SearchRequest();
        searchRequest.indices("employees");
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(QueryBuilders.boolQuery().must(QueryBuilders.termQuery("name.keyword", name))
                .must(QueryBuilders.matchQuery("occupation", occupation)));
        searchRequest.source(searchSourceBuilder);
        List list = new ArrayList();

        try {
            SearchResponse searchResponse = null;
            searchResponse =client.search(searchRequest, RequestOptions.DEFAULT);
            if (searchResponse.getHits().getTotalHits().value > 0) {
                SearchHit[] searchHit = searchResponse.getHits().getHits();
                for (SearchHit hit : searchHit) {
                    Map map = hit.getSourceAsMap();
                    list.add(objectMapper.convertValue(map, Employee.class));
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return list;
    }

重新运行 Sprint Boot 应用,并在浏览器中打入如下的 URL 请求:

http://localhost:9999/hr/employee/allemployees/%E5%BC%A0%E4%B8%89/developer

我们可以在浏览器中看到如下的结果。

%title插图%num

从上面,我们可以看到有一个文档被搜索到了。上面的请求类似于 Kibana 中这样的搜索:

GET employees/_search 
{
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "name.keyword": "张三"
          }
        },
        {
          "match": {
            "occupation": "developer"
          }
        }
      ]
    }
  }
}

它显示的结果为:

%title插图%num

如果我们打入如下的请求:

%title插图%num我们看到一个空的返回。这是因为在我们的文档中,没有一个 name 叫做 “张三” 而且 occupation 是 engineer 的文档。

为了方便阅读,我把最终的代码放入如下的地址里:

git clone https://github.com/liu-xiao-guo/SpringBootElasticsearch
cd SpringBootElasticsearch
git checkout final

然后,你就可以看到整个项目的代码了。

文章来源于互联网:Elasticsearch:通过 Spring boot 创建 REST APIs 来访问 Elasticsearch

相关推荐: Elasticsearch:使用 Python 进行 Bulk insert 及 Scan

在本文章中,我将展示如何使用 Python 来对索引进行 Bulk写入。在单个 BulkAPI 调用中执行多个索引或删除操作。 这减少了开销并且可以大大提高索引速度。 在今天的展示中,我将使用 Jupyter 来进行演示。如果你对 Jupyter 还不是很熟的…

Tags: , , , ,