Category: kafka Introduction

bookmark_borderWhy Apache Kafka?

Posted on by Jay Jo
  1. Monolith System vs Micro Service Architecture
  2. What is a problem?
  3. How to solve the problem?
  4. Apache Kafka Data
  5. Why Apache Kafka?

Apache Kafka is a popular tool for developers because it is easy to pick up and provides a powerful event streaming platform complete with 4 APIs: Producer, Consumer, Streams, and Connect.

Monolith System vs Micro Service Architecture

Monolith System

  • Easy to data exchange.

However, Monolith System to Micro Service Architecture?

  • Micro Service Architecture: it will be more and more complicated.

What is a problem?

if you have 5 Source Systems and 6 Target Systems, you need 24 integrations. many integrations increase time and cost.

  • Protocol – How the data is transported (TCP, HTTP, REST, FTP, JDBC…)
  • Data format – How the data is parsed (Binary, CSV, JSON, Avro ..)
  • Data Schema & Evolution – How the data is shaped and may change

Also,from each Source systems’s connections, occurs increased load.

How to solve the problem?

Apach kafka: data steams decoupling. Apach kafka is optimized for Decoupling.

Data of Source system send Apach kafka, and Target system got data from Apache Kafka

Apache Kafka Data

  • website events
  • pricing data
  • financial transactions
  • user interactions
  • more..

Moreover, any data in Apach Kafka. any Target Systems can use freely.

  • database
  • analytics systems
  • email system
  • audit

Why Apache Kafka?

  • Created in LinkedIn, now open source project, mostly management at Confluent
  • Distributed, resilient architecture, fault tolerant
  • Vertically Scale up
    • Can scale to 100s of brokers
    • Can scale to millions of message/second
  • High performance (latency of less than 10ms) – real time
  • 2000+ firms, 35% of the Fortune 500
    • LinkedIn, Airbnb, Netflix, Uber, Walmart, and more
ANOTE.DEV