kafka install (작성중)

0. 서버 기본 설정

0.1 hostname 변경하기 (모든서버에서 작업)

구성할 kafka 서버에 hostname들을 각각 변경한

[ws@localhost~]$ sudo hostnamectl set-hostname kserver01
[sudo] ws의 암호:
[ws@localhost~]$

[ws@localhost~]$ sudo hostnamectl set-hostname kserver02
[sudo] ws의 암호:
[ws@localhost~]$

[ws@localhost~]$ sudo hostnamectl set-hostname kserver03
[sudo] ws의 암호:
[ws@localhost~]$

0.2 hosts 변경하기 (모든서버에서 작업)

[ws@kserver01~]$ sudo vi /etc/hosts

172.30.2.11 kserver01
172.30.2.12 kserver02
172.30.2.13 kserver03
		    
172.30.2.21 zserver01
172.30.2.22 zserver02
172.30.2.23 zserver03

0.3 Java 설치 (모든서버에서 작업)

[ws@kserver01 ~]$ sudo yum install java-1.8.0-openjdk-devel.x86_64 -y
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: ftp.kaist.ac.kr
 * extras: ftp.kaist.ac.kr
 * updates: ftp.kaist.ac.kr
Resolving Dependencies
--> Running transaction check

Installed:
  java-1.8.0-openjdk-devel.x86_64 1:1.8.0.352.b08-2.el7_9

Dependency Installed:
  atk.x86_64 0:2.28.1-2.el7                     avahi-libs.x86_64 0:0.6.31-20.el7                cairo.x86_64 0:1.15.12-4.el7                        copy-jdk-configs.noarch 0:3.3-11.el7_9
  cups-libs.x86_64 1:1.6.3-51.el7               dejavu-fonts-common.noarch 0:2.33-6.el7          dejavu-sans-fonts.noarch 0:2.33-6.el7               fontconfig.x86_64 0:2.13.0-4.3.el7
  fontpackages-filesystem.noarch 0:1.44-8.el7   fribidi.x86_64 0:1.0.2-1.el7_7.1                 gdk-pixbuf2.x86_64 0:2.36.12-3.el7                  giflib.x86_64 0:4.1.6-9.el7
  graphite2.x86_64 0:1.3.10-1.el7_3             gtk-update-icon-cache.x86_64 0:3.22.30-8.el7_9   gtk2.x86_64 0:2.24.31-1.el7                         harfbuzz.x86_64 0:1.7.5-2.el7
  hicolor-icon-theme.noarch 0:0.12-7.el7        jasper-libs.x86_64 0:1.900.1-33.el7              java-1.8.0-openjdk.x86_64 1:1.8.0.352.b08-2.el7_9   java-1.8.0-openjdk-headless.x86_64 1:1.8.0.352.b08-2.el7_9
  javapackages-tools.noarch 0:3.4.1-11.el7      jbigkit-libs.x86_64 0:2.0-11.el7                 libICE.x86_64 0:1.0.9-9.el7                         libSM.x86_64 0:1.2.2-2.el7
  libX11.x86_64 0:1.6.7-4.el7_9                 libX11-common.noarch 0:1.6.7-4.el7_9             libXau.x86_64 0:1.0.8-2.1.el7                       libXcomposite.x86_64 0:0.4.4-4.1.el7
  libXcursor.x86_64 0:1.1.15-1.el7              libXdamage.x86_64 0:1.1.4-4.1.el7                libXext.x86_64 0:1.3.3-3.el7                        libXfixes.x86_64 0:5.0.3-1.el7
  libXft.x86_64 0:2.3.2-2.el7                   libXi.x86_64 0:1.7.9-1.el7                       libXinerama.x86_64 0:1.1.3-2.1.el7                  libXrandr.x86_64 0:1.5.1-2.el7
  libXrender.x86_64 0:0.9.10-1.el7              libXtst.x86_64 0:1.2.3-1.el7                     libXxf86vm.x86_64 0:1.1.4-1.el7                     libfontenc.x86_64 0:1.1.3-3.el7
  libglvnd.x86_64 1:1.0.1-0.8.git5baa1e5.el7    libglvnd-egl.x86_64 1:1.0.1-0.8.git5baa1e5.el7   libglvnd-glx.x86_64 1:1.0.1-0.8.git5baa1e5.el7      libjpeg-turbo.x86_64 0:1.2.90-8.el7
  libthai.x86_64 0:0.1.14-9.el7                 libtiff.x86_64 0:4.0.3-35.el7                    libwayland-client.x86_64 0:1.15.0-1.el7             libwayland-server.x86_64 0:1.15.0-1.el7
  libxcb.x86_64 0:1.13-1.el7                    libxshmfence.x86_64 0:1.2-1.el7                  libxslt.x86_64 0:1.1.28-6.el7                       lksctp-tools.x86_64 0:1.0.17-2.el7
  mesa-libEGL.x86_64 0:18.3.4-12.el7_9          mesa-libGL.x86_64 0:18.3.4-12.el7_9              mesa-libgbm.x86_64 0:18.3.4-12.el7_9                mesa-libglapi.x86_64 0:18.3.4-12.el7_9
  pango.x86_64 0:1.42.4-4.el7_7                 pcsc-lite-libs.x86_64 0:1.8.8-8.el7              pixman.x86_64 0:0.34.0-1.el7                        python-javapackages.noarch 0:3.4.1-11.el7
  python-lxml.x86_64 0:3.2.1-4.el7              ttmkfdir.x86_64 0:3.0.9-42.el7                   tzdata-java.noarch 0:2022e-1.el7                    xorg-x11-font-utils.x86_64 1:7.5-21.el7
  xorg-x11-fonts-Type1.noarch 0:7.5-9.el7

Complete!

[ws@kserver01 ~]$ java -version
openjdk version "1.8.0_352"
OpenJDK Runtime Environment (build 1.8.0_352-b08)
OpenJDK 64-Bit Server VM (build 25.352-b08, mixed mode)

0.4 방화벽 해제

## zookeeper
[ws@kserver01 ~]$ sudo firewall-cmd --permanent --zone=public --add-port=2181/tcp
success
[ws@kserver01 ~]$ sudo firewall-cmd --permanent --zone=public --add-port=2888/tcp
success
[ws@kserver01 ~]$ sudo firewall-cmd --permanent --zone=public --add-port=3888/tcp
success

## kafka
[ws@kserver01 ~]$ sudo firewall-cmd --permanent --zone=public --add-port=9092/tcp
success

[ws@kserver01 ~]$ sudo firewall-cmd --reload
success

1. Zookeeper

1.1 Zookeeper 다운로드 & 압축풀기

[ws@zserver03 ~]$ wget <https://mirror.navercorp.com/apache/zookeeper/zookeeper-3.6.3/apache-zookeeper-3.6.3-bin.tar.gz>
--2022-11-04 16:19:09--  <https://mirror.navercorp.com/apache/zookeeper/zookeeper-3.6.3/apache-zookeeper-3.6.3-bin.tar.gz>
Resolving mirror.navercorp.com (mirror.navercorp.com)... 125.209.216.167
Connecting to mirror.navercorp.com (mirror.navercorp.com)|125.209.216.167|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12516362 (12M) [application/octet-stream]
Saving to: ‘apache-zookeeper-3.6.3-bin.tar.gz’

100%[=======================================================================================================================================================================>] 12,516,362  2.85MB/s   in 4.2s

2022-11-04 16:19:14 (2.85 MB/s) - ‘apache-zookeeper-3.6.3-bin.tar.gz’ saved [12516362/12516362]
[ws@zserver03 ~]$ sudo mkdir /sw
[ws@zserver03 ~]$ sudo tar zxvf apache-zookeeper-3.6.3-bin.tar.gz -C /sw
[ws@zserver03 ~]$ ll /sw
합계 0
drwxr-xr-x. 6 root root 133 11월  4 16:20 apache-zookeeper-3.6.3-bin
[ws@zserver03 sw]$ sudo mv apache-zookeeper-3.6.3-bin/ zookeeper
[ws@zserver03 sw]$ ll
합계 0
drwxr-xr-x. 6 root root 133 11월  4 16:20 zookeeper
[ws@zserver03 sw]$

1.2 환경설정

[ws@zserver03 ~]$ sudo cp /sw/zookeeper/conf/zoo_sample.cfg /sw/zookeeper/conf/zoo.cfg
[ws@zserver03 ~]$ sudo vi /sw/zookeeper/conf/zoo.cfg

(수정)
dataDir=/data
(추가)
server.1=kserver01:2888:3888
server.2=kserver02:2888:3888
server.3=kserver03:2888:3888

[ws@zserver03 ~]$ cd /data
[ws@zserver03 data]$ ll
[ws@zserver03 data]$ sudo echo 3 > /data/myid
[ws@zserver03 data]$ cat myid
3

[ws@zserver02 data]$ sudo echo 2 > /data/myid
[ws@zserver02 data]$ cat myid
2

[ws@zserver01 data]$ sudo echo 1 > /data/myid
[ws@zserver01 data]$ cat myid
1

1.3 zookeeper 서비스 등록

[ws@zserver03 data]$ sudo vi /etc/systemd/system/zookeeper-server.service

[Unit]
Description=zookeeper-server
After=network.target

[Service]
Type=forking
User=zookeeper
Group=zookeeper
SyslogIdentifier=zookeeper-server
WorkingDirectory=/sw/zookeeper
Restart=always
RestartSec=0s
ExecStart=/sw/zookeeper/bin/zkServer.sh start
ExecStop=/sw/zookeeper/bin/zkServer.sh stop

[Install]
WantedBy=multi-user.target

[ws@zserver03 data]$ sudo systemctl daemon-reload
[ws@zserver03 data]$ sudo systemctl start zookeeper-server.service
[ws@zserver03 data]$ sudo systemctl status zookeeper-server.service 
[ws@zserver03 data]$ sudo systemctl enable zookeeper-server.service

kafka download & untar (모든서버에서 작업)

[ws@kserver01~]$ sudo wget <https://archive.apache.org/dist/kafka/3.1.0/kafka_2.13-3.1.0.tgz>
[ws@kserver01~]$ sudo mkdir /sw /data1 /data2
[ws@kserver01~]$ sudo tar zxvf kafka_2.13-3.1.0.tgz -C /sw/

[ws@kserver01~]$ tar zxvf kafka_2.13-3.1.0.tgz
[ws@kserver01~]$ ll
drwxr-xr-x. 7 ws ws      105  1월 12  2022 kafka_2.13-3.1.0
-rw-rw-r--. 1 ws ws 88130011  1월 21  2022 kafka_2.13-3.1.0.tgz
[ws@localhost ~]$ ll kafka_2.13-3.1.0
합계 64
-rw-r--r--. 1 ws ws 14536  1월 12  2022 LICENSE
-rw-r--r--. 1 ws ws 28184  1월 12  2022 NOTICE
drwxr-xr-x. 3 ws ws  4096  1월 12  2022 bin
drwxr-xr-x. 3 ws ws  4096  1월 12  2022 config
drwxr-xr-x. 2 ws ws  8192 11월  4 15:53 libs
drwxr-xr-x. 2 ws ws   262  1월 12  2022 licenses
drwxr-xr-x. 2 ws ws    44  1월 12  2022 site-docs