How to integrate ClickHouse database in SpringBoot2

1. Introduction to ClickHouse

1. Basic introduction

ClickHouse is an open source database by Yandex, which is particularly suitable for processing time series streaming data or batch data storage. ClickHouse should not be used as a general-purpose database, but as an ultra-high-performance distributed real-time processing platform for rapid query of massive data. In terms of data summary queries (such as GROUP BY), ClickHouse's query speed is very fast.

2. Data analysis capability

• OLAP scenario characteristics

· 大多数是读请求
· 数据总是以相当大的批(> 1000 rows)进行写入
· 不修改已添加的数据
· 每次查询都从数据库中读取大量的行，但是同时又仅需要少量的列
· 宽表，即每个表包含着大量的列
· 较少的查询(通常每台服务器每秒数百个查询或更少)
· 对于简单查询，允许延迟大约50毫秒
· 列中的数据相对较小： 数字和短字符串(例如，每个URL 60个字节)
· 处理单个查询时需要高吞吐量（每个服务器每秒高达数十亿行）
· 事务不是必须的
· 对数据一致性要求低
· 每一个查询除了一个大表外都很小
· 查询结果明显小于源数据，换句话说，数据被过滤或聚合后能够被盛放在单台服务器的内存中

• Column type Data storage

(1), row data

(2), column data

(3), comparative analysis

Analytical queries usually only need to read a small part of the columns of the table. In a columnar database, only the data you need can be read. Data is always packed into batch reads, so compression is very easy. At the same time, the data is stored separately in columns, which makes it easier to compress. This further reduces I/O size. This will help more data be cached by the system due to reduced I/O.

2. Integrate the SpringBoot framework

This case is based on the integration of Druid connection pool and mybatis. SQL parser Druid version 1.1.10 now supports ClickHouse.

1. Core dependency

<dependency>
    <groupId>ru.yandex.clickhouse</groupId>
    <artifactId>clickhouse-jdbc</artifactId>
    <version>0.1.53</version>
</dependency>

2. Attached data source

spring:
  datasource:
    type: com.alibaba.druid.pool.DruidDataSource
    click:
      driverClassName: ru.yandex.clickhouse.ClickHouseDriver
      url: jdbc:clickhouse://127.0.0.1:8123/default
      initialSize: 10
      maxActive: 100
      minIdle: 10
      maxWait: 6000

3. Druid connection pool configuration

@Configuration
public class DruidConfig {
    @Resource
    private JdbcParamConfig jdbcParamConfig ;
    @Bean
    public DataSource dataSource() {
        DruidDataSource datasource = new DruidDataSource();
        datasource.setUrl(jdbcParamConfig.getUrl());
        datasource.setDriverClassName(jdbcParamConfig.getDriverClassName());
        datasource.setInitialSize(jdbcParamConfig.getInitialSize());
        datasource.setMinIdle(jdbcParamConfig.getMinIdle());
        datasource.setMaxActive(jdbcParamConfig.getMaxActive());
        datasource.setMaxWait(jdbcParamConfig.getMaxWait());
        return datasource;
    }
}

4. Parameter configuration class

@Component
@ConfigurationProperties(prefix = "spring.datasource.click")
public class JdbcParamConfig {
    private String driverClassName ;
    private String url ;
    private Integer initialSize ;
    private Integer maxActive ;
    private Integer minIdle ;
    private Integer maxWait ;
    // 省略 GET 和 SET
}

In this way, the integration code is completed.

3. Operation case demonstration

1. Mapper interface

public interface UserInfoMapper {
    // 写入数据
    void saveData (UserInfo userInfo) ;
    // ID 查询
    UserInfo selectById (@Param("id") Integer id) ;
    // 查询全部
    List<UserInfo> selectList () ;
}

Here are three simple interfaces demonstrated.

2. Mapper.xml file

<mapper namespace="com.click.house.mapper.UserInfoMapper">
    <resultMap id="BaseResultMap" type="com.click.house.entity.UserInfo">
        <id column="id" jdbcType="INTEGER" property="id" />
        <result column="user_name" jdbcType="VARCHAR" property="userName" />
        <result column="pass_word" jdbcType="VARCHAR" property="passWord" />
        <result column="phone" jdbcType="VARCHAR" property="phone" />
        <result column="email" jdbcType="VARCHAR" property="email" />
        <result column="create_day" jdbcType="VARCHAR" property="createDay" />
    </resultMap>
    <sql id="Base_Column_List">
        id,user_name,pass_word,phone,email,create_day
    </sql>
    <insert id="saveData" parameterType="com.click.house.entity.UserInfo" >
        INSERT INTO cs_user_info
        (id,user_name,pass_word,phone,email,create_day)
        VALUES
        (#{id,jdbcType=INTEGER},#{userName,jdbcType=VARCHAR},#{passWord,jdbcType=VARCHAR},
        #{phone,jdbcType=VARCHAR},#{email,jdbcType=VARCHAR},#{createDay,jdbcType=VARCHAR})
    </insert>
    <select id="selectById" parameterType="java.lang.Integer" resultMap="BaseResultMap">
        select
        <include refid="Base_Column_List" />
        from cs_user_info
        where id = #{id,jdbcType=INTEGER}
    </select>
    <select id="selectList" resultMap="BaseResultMap" >
        select
        <include refid="Base_Column_List" />
        from cs_user_info
    </select>
</mapper>

Here create_day is converted as a string, so you need to pay attention here.

3. Control layer interface

@RestController
@RequestMapping("/user")
public class UserInfoController {
    @Resource
    private UserInfoService userInfoService ;
    @RequestMapping("/saveData")
    public String saveData (){
        UserInfo userInfo = new UserInfo () ;
        userInfo.setId(4);
        userInfo.setUserName("winter");
        userInfo.setPassWord("567");
        userInfo.setPhone("13977776789");
        userInfo.setEmail("winter");
        userInfo.setCreateDay("2020-02-20");
        userInfoService.saveData(userInfo);
        return "sus";
    }
    @RequestMapping("/selectById")
    public UserInfo selectById () {
        return userInfoService.selectById(1) ;
    }
    @RequestMapping("/selectList")
    public List<UserInfo> selectList () {
        return userInfoService.selectList() ;
    }
}

The above is the detailed content of How to integrate ClickHouse database in SpringBoot2. For more information, please follow other related articles on the PHP Chinese website!