Category: kafka Introduction

bookmark_borderKafka 101

Posted on by Jay Jo
  1. Installation
    1. Move directory
    2. Unzip Apache kafka
    3. Add PATH for Apache Kafka
  2. Start Zookeeper and Kafka
    1. Check Zookeeper Data Directory
    2. Create data directory for Zookeeper Data
    3. Restart zookeeper
    4. check data/zookeeper
    5. create & edit Apache kafka directory
    6. Start Kafka

Installation

https://kafka.apache.org/

Move directory & unzip Apache kafka

$ mv Downloads/kafka_2.13-2.6.0.tgz
$ tar -xvf kafka_2.13-2.6.0.tgz

Add PATH for Apache Kafka

# Apache Kafka
export PATH="$PATH:/Users/jayjo/kafka_2.13-2.6.0/bin"

Start Zookeeper and Kafka In Apache kafka directory

$ kafka-topics.sh 
$ zookeeper-server-start.sh config/zookeeper.properties
[2020-09-10 17:16:44,026] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)

Check Zookeeper Data Directory

In Apache kafka directory

$ cat config/zookeeper.properties
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
# 
#    http://www.apache.org/licenses/LICENSE-2.0
# 
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# the directory where the snapshot is stored.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
# disable the per-ip limit on the number of connections since this is a non-production config
maxClientCnxns=0
# Disable the adminserver by default to avoid port conflicts.
# Set the port to something non-conflicting if choosing to enable this
admin.enableServer=false
# admin.serverPort=8080
  • dataDir is set up as /tmp/zookeeper

Create data directory for Zookeeper Data

$ mkdir data
$ mkdir data/zookeeper
$ open config/zookeeper.properties
dataDir=/Users/jayjo/kafka_2.13-2.6.0/data/zookeeper
  • dataDir to new directory

Restart zookeeper

$ zookeeper-server-start.sh config/zookeeper.properties

check data/zookeeper

$ ls data/zookeeper/
version-2

create & edit Apache kafka directory

$ mkdir data/kafka
$ open config/server.properties
log.dirs=/Users/jayjo/kafka_2.13-2.6.0/data/kafka

Start Kafka

$ bin/kafka-server-start.sh config/server.properties
[2020-09-10 18:13:01,674] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)
  • started message

https://www.udemy.com/course/apache-kafka/learn/lecture/11566926#announcements

ANOTE.DEV