Kafka 高可用部署
· 阅读需 2 分钟
Kafka 高可用部署
- 下载连接：https://kafka.apache.org/downloads
- 部署 java 环境 （略）
解压并放到对应目录
tar xf kafka_2.12-2.8.0.tgz
mv kafka_2.12-2.8.0 /opt
zookeeper 相关配置
vim /opt/kafka_2.12-2.8.0/config/zookeeper.properties
dataDir=/data/zookeeper
clientPort=2181
maxClientCnxns=0
tickTime=2000
initLimit=20
syncLimit=10
server.1=192.168.8.204:2888:3888
server.2=192.168.8.205:2888:3888
server.3=192.168.8.206:2888:3888
kafka 相关配置
vim /opt/kafka_2.12-2.8.0/config/server.properties
## 此处的broker每个节点id不能相同
broker.id=1
host.name=192.168.8.204
listeners=PLAINTEXT://192.168.8.204:9092
log.dirs=/data/kafka-logs
log.retention.check.interval.ms=300000
log.retention.hours=1
log.segment.bytes=1073741824
num.io.threads=4
default.replication.factor=2
num.network.threads=3
num.partitions=30
num.recovery.threads.per.data.dir=1
port=9092
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
socket.send.buffer.bytes=102400
zookeeper.connect=192.168.8.204:2181,192.168.8.205:2181,192.168.8.206:2181
zookeeper.connection.timeout.ms=10000
fetch.message.max.bytes=52428700
replica.fetch.max.bytes=52428700
启动时加入到/etc/rc.local
vim /etc/rc.local
cd /var/tmp;nohup /opt/kafka_2.12-2.8.0/bin/zookeeper-server-start.sh /opt/kafka_2.12-2.8.0/config/zookeeper.properties &
cd /tmp;nohup /opt/kafka_2.12-2.8.0/bin/kafka-server-start.sh /opt/kafka_2.12-2.8.0/config/server.properties &
touch /var/lock/subsys/local
创建 zookeeper myid 文件
依次创建zookeeper的数据目录/data/zookeeper和文件/data/zookeeper/myid,此时我们需要对应的Zookeeper服务器上在dataDir的目录下面新建myid文件，同时把对应的1、2、3写进去。
mkdir -p /data/zookeeper
vim /data/zookeeper/myid
1
启动 zookeeper
依次启动集群的kafka1、kafka2、kafka3 的zookeeper
nohup /opt/kafka_2.12-2.8.0/bin/zookeeper-server-start.sh /opt/kafka_2.12-2.8.0/config/zookeeper.properties &
启动 kafka
依次启动集群的kafka1、kafka2、kafka3 的kafka服务
nohup /opt/kafka_2.12-2.8.0/bin/kafka-server-start.sh /opt/kafka_2.12-2.8.0/config/server.properties &
检查
使用 jps 还有 netstat 检查服务