Zookeeper安装记录

Zookeeper 单机和集群的安装记录

本地模式安装（单机）

安装步骤

  1. 服务器需要预先配置好 JDK

  2. 准备安装包：https://archive.apache.org/dist/zookeeper/zookeeper-3.5.7/

  3. 下载文件：apache-zookeeper-3.5.7-bin.tar.gz

  4. 将此压缩包上传至服务器上

    [root@hadoop102 software]# ll
总用量 529696
-rw-r--r--. 1 root root   9311744 4月  27 22:00 apache-zookeeper-3.5.7-bin.tar.gz
-rw-r--r--. 1 root root 338075860 4月  17 15:46 hadoop-3.1.3.tar.gz
-rw-r--r--. 1 root root 195013152 4月  17 15:46 jdk-8u212-linux-x64.tar.gz
[root@hadoop102 software]# pwd
/opt/software

  5. 解压此压缩包至指定目录：

    tar -zxvf apache-zookeeper-3.5.7-bin.tar.gz -C /opt/module/
# 进入到解压后的目录，把文件夹改个名
[root@hadoop102 ~]# cd /opt/module/
[root@hadoop102 module]# ll
总用量 0
drwxr-xr-x.  6 root  root  134 4月  27 22:03 apache-zookeeper-3.5.7-bin
drwxr-xr-x. 13 houge houge 204 4月  17 21:16 hadoop-3.1.3
drwxr-xr-x.  7    10   143 245 4月   2 2019 jdk1.8.0_212
[root@hadoop102 module]# mv apache-zookeeper-3.5.7-bin/ zookeeper-3.5.7
[root@hadoop102 module]# ll
总用量 0
drwxr-xr-x. 13 houge houge 204 4月  17 21:16 hadoop-3.1.3
drwxr-xr-x.  7    10   143 245 4月   2 2019 jdk1.8.0_212
drwxr-xr-x.  6 root  root  134 4月  27 22:03 zookeeper-3.5.7
[root@hadoop102 module]# pwd
/opt/module

  6. 进入解压缩后的文件目录

    [root@hadoop102 module]# cd zookeeper-3.5.7/
[root@hadoop102 zookeeper-3.5.7]# ll
总用量 32
drwxr-xr-x. 2  502 games   232 2月  10 2020 bin
drwxr-xr-x. 2  502 games    77 2月   7 2020 conf
drwxr-xr-x. 5  502 games  4096 2月  10 2020 docs
drwxr-xr-x. 2 root root   4096 4月  27 22:03 lib
-rw-r--r--. 1  502 games 11358 9月  13 2018 LICENSE.txt
-rw-r--r--. 1  502 games   432 2月  10 2020 NOTICE.txt
-rw-r--r--. 1  502 games  1560 2月   7 2020 README.md
-rw-r--r--. 1  502 games  1347 2月   7 2020 README_packaging.txt

  7. 关于目录的介绍：

    • bin: zookeeper 的相关脚本
    • conf: zookeeper 的配置文件
    • docs: zookeeper 相关文档
    • lib: zookeeper 相关依赖包

  8. 配置修改

    # 进入配置文件目录
cd conf/
# zk 提供了一个样例配置文件，我们把这个样例配置文件名改为 zoo.cfg 即可被 zk 检测到
mv zoo_sample.cfg zoo.cfg
# 编写配置文件
vim zoo.cfg
# 主要改一个地方，就是 zk 的数据存储目录 不能用 tmp，这个目录会被系统定时清除
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/opt/module/zookeeper-3.5.7/zkData
# the port at which the clients will connect
clientPort=2181
# 这个数据存储目录必须是存在的，所以创建这个目录
mkdir /opt/module/zookeeper-3.5.7/zkData

  9. 启动 zookeeper

    [root@hadoop102 zookeeper-3.5.7]# ./bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@hadoop102 zookeeper-3.5.7]# pwd
/opt/module/zookeeper-3.5.7
# 查看进程是否启动
[root@hadoop102 zookeeper-3.5.7]# jps
41593 QuorumPeerMain
41871 Jps

  10. 查看 zookeeper 状态

    [root@hadoop102 zookeeper-3.5.7]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: standalone

  11. 启动 / 退出客户端

    # 启动
[root@hadoop102 zookeeper-3.5.7]# ./bin/zkCli.sh
# 使用 quit 退出
[zk: localhost:2181(CONNECTED) 0] quit

WATCHER::

WatchedEvent state:Closed type:None path:null
2022-04-27 22:21:42,223 [myid:] - INFO  [main:ZooKeeper@1422] - Session: 0x100005c32360000 closed
2022-04-27 22:21:42,224 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@524] - EventThread shut down for session: 0x100005c32360000

  12. 停止 zookeeper

    [root@hadoop102 zookeeper-3.5.7]# ./bin/zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@hadoop102 zookeeper-3.5.7]# jps
44450 Jps

配置文件参数解释

  1. 配置文件内容

    # The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/opt/module/zookeeper-3.5.7/zkData
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

  2. 参数解读

    • tickTime：Zookeeper 服务端与客户端的心跳时间，单位毫秒

    • initLimit：Leader 和 Follower 初始连接的时候能容忍的最多心跳次数

    • syncLimit： Leader 和 Follower之间通信时间如果超过 syncLimit * tickTime，Leader认为Follwer死掉，从服务器列表中删除 Follwer

    • dataDir： 保存 Zookeeper 中的数据

    • clientPort： 客户端连接端口

集群安装

一共三台服务器 hadoop102、hadoop103、hadoop104

  1. 执行单机安装前八步，完成 zookeeper 安装，此时应该是已经配置了 zookeeper 的 dataDir，并且也同步创建了 zkDara 目录

  2. 现在进入 zkData 目录，创建 myid 文件

    注意：文件名只能是 myid，不能是其他的名字，zookeeper 靠这个来给机器进行标识，后续选举也会用到这个 myid

    [root@hadoop102 zkData]# pwd
/opt/module/zookeeper-3.5.7/zkData
[root@hadoop102 zkData]# vim myid

    然后向 myid 中写入一个数字，可以自定义，注意这个数字前后不要有空格，上下不要有空行，我这里是写入的是 2

  3. 分发配置好的 zookeeper 文件夹

    [root@hadoop102 module]# xsync zookeeper-3.5.7

  4. 然后上另外两台服务器上改 myid，分别改成 3 和 4

    [root@hadoop103 ~]# cd /opt/module/zookeeper-3.5.7/zkData/
[root@hadoop103 zkData]# ll
总用量 4
-rw-r--r--. 1 root root  2 5月   6 10:19 myid
drwxr-xr-x. 2 root root 37 4月  27 22:21 version-2
[root@hadoop103 zkData]# vim myid
# 104
[root@hadoop104 ~]# cd /opt/module/zookeeper-3.5.7/zkData/
[root@hadoop104 zkData]# ll
总用量 4
-rw-r--r--. 1 root root  2 5月   6 10:19 myid
drwxr-xr-x. 2 root root 37 4月  27 22:21 version-2
[root@hadoop104 zkData]# vim myid

  5. 配置 zoo.cfg

    [root@hadoop102 conf]# pwd
/opt/module/zookeeper-3.5.7/conf
[root@hadoop102 conf]# vim zoo.cfg

    在 zoo.cfg 中加入如下配置

    #######################cluster##########################
server.2=hadoop102:2888:3888
server.3=hadoop103:2888:3888
server.4=hadoop104:2888:3888

    这段配置啥意思？

    把里面的变量用 A B C D 来表示

    server.A=B:C:D
    • A：就是我们在 dataDir 下配置 myid 文件中写入的数字值；Zookeeper 启动时读取此文件，拿到里面的数据与 zoo.cfg 里面的配置信息比 较从而判断到底是哪个 server
    • B：这台服务器的地址
    • C：是这个服务器 Follower 与集群中的 Leader 服务器交换信息的端口
    • D：是万一集群中的 Leader 服务器挂了，需要一个端口来重新进行选举，选出一个新的 Leader，这个端口就是用来执行选举时服务器互相通信的端口

  6. 同步配置文件

    [root@hadoop102 conf]# xsync zoo.cfg

  7. 在三台机器上分别启动 zookeeper

    # 103
[root@hadoop103 zookeeper-3.5.7]# bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
# 102
[root@hadoop102 zookeeper-3.5.7]# bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
# 104
[root@hadoop104 zookeeper-3.5.7]# bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

  8. 三台机器上查看 zookeeper 的状态

    # 102
[root@hadoop102 zookeeper-3.5.7]# bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: follower
# 103
[root@hadoop103 zookeeper-3.5.7]# bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: leader
# 104
[root@hadoop104 zookeeper-3.5.7]# bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: follower

    可以看到 103 是 Leader，这个 Leader 咋选出来的呢（zookeeper 选举机制）？

  9. 可以发现每次启动停止三台机器都得操作，麻烦的不行，所以还是得写一个集群的启动停止脚本

    参考我们编写 xsync 脚本时候的操作：进入到 /usr/bin 在这里写的可执行文件可以全局使用 而且任何用户都可用

    [root@hadoop102 zookeeper-3.5.7]# cd /usr/bin/
[root@hadoop102 bin]# vim zk.sh

    写入以下内容：

    #!/bin/bash
case $1 in
"start"){
  for i in hadoop102 hadoop103 hadoop104
  do
	echo ---------- zookeeper $i 启动 ------------
    ssh $i "/opt/module/zookeeper-3.5.7/bin/zkServer.sh start"
  done
};;
"stop"){
  for i in hadoop102 hadoop103 hadoop104
  do
    echo ---------- zookeeper $i 停止 ------------ 
    ssh $i "/opt/module/zookeeper-3.5.7/bin/zkServer.sh stop"
  done
};;
"status"){
  for i in hadoop102 hadoop103 hadoop104
  do
    echo ---------- zookeeper $i 状态 ------------ 
    ssh $i "/opt/module/zookeeper-3.5.7/bin/zkServer.sh status"
  done
};;
esac

    脚本整体思路：case 脚本执行的时候传入的参数，然后在三台机器中做循环，分别 ssh 到对应机器上以绝对路径的方式执行 zookeeper 的启动、停止和查看状态命令

    编辑完成添加可执行权限

    [root@hadoop102 bin]# chmod +x zk.sh
# 可以把脚本分发到另外两台机器上，这样在任意一台机器都能跑这个脚本了
[root@hadoop102 bin]# xsync zk.sh

    验证一下：

    # 停止
[root@hadoop102 /]# zk.sh stop
---------- zookeeper hadoop102 停止 ------------
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
---------- zookeeper hadoop103 停止 ------------
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
---------- zookeeper hadoop104 停止 ------------
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
# 启动
[root@hadoop102 /]# zk.sh start
---------- zookeeper hadoop102 启动 ------------
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
---------- zookeeper hadoop103 启动 ------------
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
---------- zookeeper hadoop104 启动 ------------
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
# 查看状态
[root@hadoop102 /]# zk.sh status
---------- zookeeper hadoop102 状态 ------------
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: follower
---------- zookeeper hadoop103 状态 ------------
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: leader
---------- zookeeper hadoop104 状态 ------------
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: follower
大数据
#大数据 #中间件 #环境搭建 #zookeeper
Zookeeper安装记录
https://www.powercheng.fun/articles/99c80bc2/
作者
powercheng
发布于
2022年4月27日
许可协议