Zookeeper安装记录

Zookeeper 单机和集群的安装记录

本地模式安装（单机）

安装步骤

1. 服务器需要预先配置好 JDK

2. 准备安装包：https://archive.apache.org/dist/zookeeper/zookeeper-3.5.7/

3. 下载文件：apache-zookeeper-3.5.7-bin.tar.gz

4. 将此压缩包上传至服务器上

   [root@hadoop102 software]# ll
   总用量 529696
   -rw-r--r--. 1 root root   9311744 4月  27 22:00 apache-zookeeper-3.5.7-bin.tar.gz
   -rw-r--r--. 1 root root 338075860 4月  17 15:46 hadoop-3.1.3.tar.gz
   -rw-r--r--. 1 root root 195013152 4月  17 15:46 jdk-8u212-linux-x64.tar.gz
   [root@hadoop102 software]# pwd
   /opt/software

5. 解压此压缩包至指定目录：

   tar -zxvf apache-zookeeper-3.5.7-bin.tar.gz -C /opt/module/
   # 进入到解压后的目录，把文件夹改个名
   [root@hadoop102 ~]# cd /opt/module/
   [root@hadoop102 module]# ll
   总用量 0
   drwxr-xr-x.  6 root  root  134 4月  27 22:03 apache-zookeeper-3.5.7-bin
   drwxr-xr-x. 13 houge houge 204 4月  17 21:16 hadoop-3.1.3
   drwxr-xr-x.  7    10   143 245 4月   2 2019 jdk1.8.0_212
   [root@hadoop102 module]# mv apache-zookeeper-3.5.7-bin/ zookeeper-3.5.7
   [root@hadoop102 module]# ll
   总用量 0
   drwxr-xr-x. 13 houge houge 204 4月  17 21:16 hadoop-3.1.3
   drwxr-xr-x.  7    10   143 245 4月   2 2019 jdk1.8.0_212
   drwxr-xr-x.  6 root  root  134 4月  27 22:03 zookeeper-3.5.7
   [root@hadoop102 module]# pwd
   /opt/module

6. 进入解压缩后的文件目录

   [root@hadoop102 module]# cd zookeeper-3.5.7/
   [root@hadoop102 zookeeper-3.5.7]# ll
   总用量 32
   drwxr-xr-x. 2  502 games   232 2月  10 2020 bin
   drwxr-xr-x. 2  502 games    77 2月   7 2020 conf
   drwxr-xr-x. 5  502 games  4096 2月  10 2020 docs
   drwxr-xr-x. 2 root root   4096 4月  27 22:03 lib
   -rw-r--r--. 1  502 games 11358 9月  13 2018 LICENSE.txt
   -rw-r--r--. 1  502 games   432 2月  10 2020 NOTICE.txt
   -rw-r--r--. 1  502 games  1560 2月   7 2020 README.md
   -rw-r--r--. 1  502 games  1347 2月   7 2020 README_packaging.txt

7. 关于目录的介绍：

   • bin: zookeeper 的相关脚本
   • conf: zookeeper 的配置文件
   • docs: zookeeper 相关文档
   • lib: zookeeper 相关依赖包

8. 配置修改

   # 进入配置文件目录
   cd conf/
   # zk 提供了一个样例配置文件，我们把这个样例配置文件名改为 zoo.cfg 即可被 zk 检测到
   mv zoo_sample.cfg zoo.cfg
   # 编写配置文件
   vim zoo.cfg
   # 主要改一个地方，就是 zk 的数据存储目录 不能用 tmp，这个目录会被系统定时清除
   # The number of milliseconds of each tick
   tickTime=2000
   # The number of ticks that the initial
   # synchronization phase can take
   initLimit=10
   # The number of ticks that can pass between
   # sending a request and getting an acknowledgement
   syncLimit=5
   # the directory where the snapshot is stored.
   # do not use /tmp for storage, /tmp here is just
   # example sakes.
   dataDir=/opt/module/zookeeper-3.5.7/zkData
   # the port at which the clients will connect
   clientPort=2181
   # 这个数据存储目录必须是存在的，所以创建这个目录
   mkdir /opt/module/zookeeper-3.5.7/zkData

9. 启动 zookeeper

   [root@hadoop102 zookeeper-3.5.7]# ./bin/zkServer.sh start
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Starting zookeeper ... STARTED
   [root@hadoop102 zookeeper-3.5.7]# pwd
   /opt/module/zookeeper-3.5.7
   # 查看进程是否启动
   [root@hadoop102 zookeeper-3.5.7]# jps
   41593 QuorumPeerMain
   41871 Jps

10. 查看 zookeeper 状态

    [root@hadoop102 zookeeper-3.5.7]# ./bin/zkServer.sh status
    ZooKeeper JMX enabled by default
    Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
    Client port found: 2181. Client address: localhost.
    Mode: standalone

11. 启动 / 退出客户端

    # 启动
    [root@hadoop102 zookeeper-3.5.7]# ./bin/zkCli.sh
    # 使用 quit 退出
    [zk: localhost:2181(CONNECTED) 0] quit
    
    WATCHER::
    
    WatchedEvent state:Closed type:None path:null
    2022-04-27 22:21:42,223 [myid:] - INFO  [main:ZooKeeper@1422] - Session: 0x100005c32360000 closed
    2022-04-27 22:21:42,224 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@524] - EventThread shut down for session: 0x100005c32360000

12. 停止 zookeeper

    [root@hadoop102 zookeeper-3.5.7]# ./bin/zkServer.sh stop
    ZooKeeper JMX enabled by default
    Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
    Stopping zookeeper ... STOPPED
    [root@hadoop102 zookeeper-3.5.7]# jps
    44450 Jps

配置文件参数解释

1. 配置文件内容

   # The number of milliseconds of each tick
   tickTime=2000
   # The number of ticks that the initial
   # synchronization phase can take
   initLimit=10
   # The number of ticks that can pass between
   # sending a request and getting an acknowledgement
   syncLimit=5
   # the directory where the snapshot is stored.
   # do not use /tmp for storage, /tmp here is just
   # example sakes.
   dataDir=/opt/module/zookeeper-3.5.7/zkData
   # the port at which the clients will connect
   clientPort=2181
   # the maximum number of client connections.
   # increase this if you need to handle more clients
   #maxClientCnxns=60
   #
   # Be sure to read the maintenance section of the
   # administrator guide before turning on autopurge.
   #
   # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
   #
   # The number of snapshots to retain in dataDir
   #autopurge.snapRetainCount=3
   # Purge task interval in hours
   # Set to "0" to disable auto purge feature
   #autopurge.purgeInterval=1

2. 参数解读

   • tickTime：Zookeeper 服务端与客户端的心跳时间，单位毫秒

     

   • initLimit：Leader 和 Follower 初始连接的时候能容忍的最多心跳次数

   • syncLimit： Leader 和 Follower之间通信时间如果超过 syncLimit * tickTime，Leader认为Follwer死掉，从服务器列表中删除 Follwer

   • dataDir： 保存 Zookeeper 中的数据

   • clientPort： 客户端连接端口

集群安装

一共三台服务器 hadoop102、hadoop103、hadoop104

1. 执行单机安装前八步，完成 zookeeper 安装，此时应该是已经配置了 zookeeper 的 dataDir，并且也同步创建了 zkDara 目录

2. 现在进入 zkData 目录，创建 myid 文件

   注意：文件名只能是 myid，不能是其他的名字，zookeeper 靠这个来给机器进行标识，后续选举也会用到这个 myid

   [root@hadoop102 zkData]# pwd
   /opt/module/zookeeper-3.5.7/zkData
   [root@hadoop102 zkData]# vim myid

   然后向 myid 中写入一个数字，可以自定义，注意这个数字前后不要有空格，上下不要有空行，我这里是写入的是 2

3. 分发配置好的 zookeeper 文件夹

   [root@hadoop102 module]# xsync zookeeper-3.5.7

4. 然后上另外两台服务器上改 myid，分别改成 3 和 4

   [root@hadoop103 ~]# cd /opt/module/zookeeper-3.5.7/zkData/
   [root@hadoop103 zkData]# ll
   总用量 4
   -rw-r--r--. 1 root root  2 5月   6 10:19 myid
   drwxr-xr-x. 2 root root 37 4月  27 22:21 version-2
   [root@hadoop103 zkData]# vim myid
   # 104
   [root@hadoop104 ~]# cd /opt/module/zookeeper-3.5.7/zkData/
   [root@hadoop104 zkData]# ll
   总用量 4
   -rw-r--r--. 1 root root  2 5月   6 10:19 myid
   drwxr-xr-x. 2 root root 37 4月  27 22:21 version-2
   [root@hadoop104 zkData]# vim myid

5. 配置 zoo.cfg

   [root@hadoop102 conf]# pwd
   /opt/module/zookeeper-3.5.7/conf
   [root@hadoop102 conf]# vim zoo.cfg

   在 zoo.cfg 中加入如下配置

   #######################cluster##########################
   server.2=hadoop102:2888:3888
   server.3=hadoop103:2888:3888
   server.4=hadoop104:2888:3888

   这段配置啥意思？

   把里面的变量用 A B C D 来表示

   server.A=B:C:D

   • A：就是我们在 dataDir 下配置 myid 文件中写入的数字值；Zookeeper 启动时读取此文件，拿到里面的数据与 zoo.cfg 里面的配置信息比 较从而判断到底是哪个 server
   • B：这台服务器的地址
   • C：是这个服务器 Follower 与集群中的 Leader 服务器交换信息的端口
   • D：是万一集群中的 Leader 服务器挂了，需要一个端口来重新进行选举，选出一个新的 Leader，这个端口就是用来执行选举时服务器互相通信的端口

6. 同步配置文件

   [root@hadoop102 conf]# xsync zoo.cfg

7. 在三台机器上分别启动 zookeeper

   # 103
   [root@hadoop103 zookeeper-3.5.7]# bin/zkServer.sh start
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Starting zookeeper ... STARTED
   # 102
   [root@hadoop102 zookeeper-3.5.7]# bin/zkServer.sh start
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Starting zookeeper ... STARTED
   # 104
   [root@hadoop104 zookeeper-3.5.7]# bin/zkServer.sh start
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Starting zookeeper ... STARTED

8. 三台机器上查看 zookeeper 的状态

   # 102
   [root@hadoop102 zookeeper-3.5.7]# bin/zkServer.sh status
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Client port found: 2181. Client address: localhost.
   Mode: follower
   # 103
   [root@hadoop103 zookeeper-3.5.7]# bin/zkServer.sh status
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Client port found: 2181. Client address: localhost.
   Mode: leader
   # 104
   [root@hadoop104 zookeeper-3.5.7]# bin/zkServer.sh status
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Client port found: 2181. Client address: localhost.
   Mode: follower

   可以看到 103 是 Leader，这个 Leader 咋选出来的呢（zookeeper 选举机制）？

9. 可以发现每次启动停止三台机器都得操作，麻烦的不行，所以还是得写一个集群的启动停止脚本

   参考我们编写 xsync 脚本时候的操作：进入到 /usr/bin 在这里写的可执行文件可以全局使用 而且任何用户都可用

   [root@hadoop102 zookeeper-3.5.7]# cd /usr/bin/
   [root@hadoop102 bin]# vim zk.sh

   写入以下内容：

   #!/bin/bash
   case $1 in
   "start"){
     for i in hadoop102 hadoop103 hadoop104
     do
   	echo ---------- zookeeper $i 启动 ------------
       ssh $i "/opt/module/zookeeper-3.5.7/bin/zkServer.sh start"
     done
   };;
   "stop"){
     for i in hadoop102 hadoop103 hadoop104
     do
       echo ---------- zookeeper $i 停止 ------------ 
       ssh $i "/opt/module/zookeeper-3.5.7/bin/zkServer.sh stop"
     done
   };;
   "status"){
     for i in hadoop102 hadoop103 hadoop104
     do
       echo ---------- zookeeper $i 状态 ------------ 
       ssh $i "/opt/module/zookeeper-3.5.7/bin/zkServer.sh status"
     done
   };;
   esac

   脚本整体思路：case 脚本执行的时候传入的参数，然后在三台机器中做循环，分别 ssh 到对应机器上以绝对路径的方式执行 zookeeper 的启动、停止和查看状态命令

   编辑完成添加可执行权限

   [root@hadoop102 bin]# chmod +x zk.sh
   # 可以把脚本分发到另外两台机器上，这样在任意一台机器都能跑这个脚本了
   [root@hadoop102 bin]# xsync zk.sh

   验证一下：

   # 停止
   [root@hadoop102 /]# zk.sh stop
   ---------- zookeeper hadoop102 停止 ------------
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Stopping zookeeper ... STOPPED
   ---------- zookeeper hadoop103 停止 ------------
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Stopping zookeeper ... STOPPED
   ---------- zookeeper hadoop104 停止 ------------
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Stopping zookeeper ... STOPPED
   # 启动
   [root@hadoop102 /]# zk.sh start
   ---------- zookeeper hadoop102 启动 ------------
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Starting zookeeper ... STARTED
   ---------- zookeeper hadoop103 启动 ------------
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Starting zookeeper ... STARTED
   ---------- zookeeper hadoop104 启动 ------------
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Starting zookeeper ... STARTED
   # 查看状态
   [root@hadoop102 /]# zk.sh status
   ---------- zookeeper hadoop102 状态 ------------
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Client port found: 2181. Client address: localhost.
   Mode: follower
   ---------- zookeeper hadoop103 状态 ------------
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Client port found: 2181. Client address: localhost.
   Mode: leader
   ---------- zookeeper hadoop104 状态 ------------
   ZooKeeper JMX enabled by default
   Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
   Client port found: 2181. Client address: localhost.
   Mode: follower