Detailed steps to install Kafka on CentOS7

Introduction

Kafka is a high-throughput distributed publish-subscribe messaging system that can replace traditional message queues for decoupling data processing, caching unprocessed messages, etc. It also has higher throughput and supports partitioning, multiple copies, Redundant, so it is widely used in large-scale message data processing applications

Kafka supports Java and multiple other language clients, and can be used in conjunction with Hadoop, Storm, Spark and other big data tools.

This tutorial mainly introduces the installation and use of Kafka on Centos 7, including functional verification and simple configuration of the cluster.

Install JDK

Kafka uses Zookeeper to save relevant configuration information. Kafka and Zookeeper rely on the Java operating environment. Download the JDK installation package from the Oracle website, unzip and install:

$tar zxvf jdk-8u65-linux-x64.tar.gz
$mv jdk1.8.0_65 java

Set Java environment variables:

JAVA_HOME=/opt/java
PATH=$PATH:$JAVA_HOME/bin
export JAVA_HOME PATH

You can also choose yum install to install and set the environment variables accordingly.

Install Kafka

Download the Kafka installation package from the official website, unzip and install: Official website address: https://www.php.cn/link/dcf531edc9b229acfe0f4b87e1e278dd

tar zxvf kafka_2.11-0.8.2.2.tgz
mv kafka_2.11-0.8.2.2 kafka
cd kafka

Functional Verification

1. Start Zookeeper Use the script in the installation package to start a single-node Zookeeper instance:

bin/zookeeper-server-start.sh -daemon config/zookeeper.properties

2. Start the Kafka service. Use kafka-server-start.sh to start the kafka service:

bin/kafka-server-start.sh config/server.properties

3. Create topic Use kafka-topics.sh to create a topic test with a single partition and a single copy:

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

View topic:

bin/kafka-topics.sh --list --zookeeper localhost:2181
test

4. Generate messages Use kafka-console-producer.sh to send messages:

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test 
Hello world！

5. Consume messages Use kafka-console-consumer.sh to receive messages and print them on the terminal:

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning

The messages generated by the producer and the messages consumed by the consumer are synchronized.

The above is the detailed content of Detailed steps to install Kafka on CentOS7. For more information, please follow other related articles on the PHP Chinese website!