Prepare for your Kafka interview with these 30 essential questions covering basic concepts, practical scenarios, and advanced topics. This guide progresses from beginner to advanced difficulty, helping freshers, 1-3 year professionals, and 3-6 year experts master Apache Kafka fundamentals and real-world applications.
Basic Kafka Interview Questions (1-10)
1. What is Apache Kafka?
Apache Kafka is a distributed streaming platform designed for high-throughput, fault-tolerant processing of real-time data feeds. It acts as a centralized hub for handling continuous data streams across systems[1][5].
2. What are the main components of Kafka architecture?
The core components include topics (data streams), partitions (data division within topics), brokers (servers storing data), producers (send data), and consumers (read data)[1][3].
3. What is a Kafka topic?
A topic is a category or feed name to which records are published. Topics are partitioned for scalability and can have multiple subscribers[1][5].
4. What is a partition in Kafka?
A partition is an ordered, immutable sequence of messages that serves as the unit of parallelism and scalability in Kafka. Partitions allow topics to spread across multiple brokers[1][3].
5. What are producers and consumers in Kafka?
Producers publish data to Kafka topics, while consumers subscribe to topics and process published messages. Consumers use group coordination for load balancing[2][5].
6. What are the four main Kafka APIs?
Kafka provides four core APIs: Producer API (send data), Consumer API (read data), Streams API (process streams), and Connector API (connect external systems)[2].
7. What is the role of brokers in Kafka?
Brokers are Kafka servers that store data, handle read/write requests, and manage replication. A Kafka cluster consists of multiple brokers working together[3][6].
8. What is an offset in Kafka?
An offset is a unique identifier for each message in a partition, representing its position in the log. Consumers track offsets to know which messages they’ve processed[1][2].
9. What are consumer groups?
Consumer groups enable parallel processing of topics. Each partition is consumed by exactly one consumer in a group, providing load balancing and fault tolerance[2][5].
10. How does Kafka ensure high throughput?
Kafka achieves high throughput through sequential disk I/O, batching, compression, and zero-copy techniques that minimize network and CPU overhead[3][4].
Intermediate Kafka Interview Questions (11-20)
11. Explain leader and follower in Kafka partitions.
Each partition has one leader broker handling all reads/writes and multiple followers that replicate data. If the leader fails, a follower becomes the new leader[1][6].
12. What are replicas and ISR in Kafka?
Replicas are copies of partitions across brokers for fault tolerance. ISR (In-Sync Replicas) are replicas keeping up with the leader’s log[1][6].
13. What are the acknowledgment modes (acks) in Kafka producers?
acks=0: No acknowledgment (fastest, least durable). acks=1: Leader acknowledgment. acks=all: All ISR acknowledgment (most durable)[6].
14. How does Kafka achieve fault tolerance?
Fault tolerance comes from replication across brokers, leader election from ISR, and configurable replication factors ensuring data availability despite failures[3][6].
15. What is Kafka retention policy?
Retention policy defines how long messages are kept based on time or size limits. Old messages are automatically deleted to manage storage[4].
16. Scenario: At Zoho, how would you configure a producer for maximum durability?
Set acks=all, retries=Integer.MAX_VALUE, enable.idempotence=true, and replication factor ≥3 to ensure messages survive broker failures[6].
props.put("acks", "all");
props.put("retries", Integer.MAX_VALUE);
props.put("enable.idempotence", true);17. What is exactly-once semantics (EOS) in Kafka?
EOS prevents duplicate processing using idempotent producers and transactional APIs, guaranteeing each message is processed exactly once[4][6].
18. How do you change retention time for a Kafka topic?
Use the retention.ms or retention.bytes config when creating/modifying topics via kafka-topics.sh tool[1].
19. What are Kafka Connect source and sink connectors?
Source connectors import data from external systems into Kafka. Sink connectors export data from Kafka to external systems[3].
20. Scenario: Paytm needs to process 1M events/sec. How would you scale Kafka?
Increase partitions per topic, add more brokers, use multiple consumer groups, and partition by event key for even distribution[3][5].
Advanced Kafka Interview Questions (21-30)
21. How does Kafka ensure data consistency?
Consistency is maintained through ISR tracking, atomic writes per partition, leader-only writes, and configurable acknowledgments[6].
22. What is the maximum message size Kafka can handle?
Default is 1MB per message, configurable via message.max.bytes. Larger messages require broker and client config changes[1].
23. Scenario: Salesforce sees consumer lag at Atlassian-scale volumes. How to fix?
Increase partitions/consumers, optimize deserialization, enable compression, use Streams API for processing, and monitor with JMX metrics[3].
24. Explain Kafka Streams API use case.
Streams API processes data in real-time within Kafka without external systems, ideal for aggregations, joins, and windowed computations[2].
25. How does Kafka handle backpressure?
Producers batch and retry based on buffer availability. Consumers control fetch rates. ISR prevents overload during failures[3].
26. What Kafka operations would you perform at Swiggy for cluster expansion?
Add brokers, rebalance partitions, increase replication factor, mirror data between clusters, and monitor ISR health[2].
27. Scenario: Flipkart broker failure – recovery steps?
Wait for automatic leader election from ISR, verify replication factor, check under-replicated partitions, and restore from backups if needed[3][6].
28. How does Kafka support multi-tenancy?
Multi-tenancy uses topic quotas, client ID quotas, and authentication to isolate tenants while sharing the same cluster[2].
29. Compare Kafka log compaction vs. delete retention.
Delete retention removes old messages by time/size. Log compaction keeps latest value per key, enabling key-based storage[5].
30. Design fault-tolerant Kafka for Adobe real-time analytics.
Use 3+ replication factor, acks=all, multiple AZs/datacenters, monitoring (Prometheus), idempotent producers, and regular ISR checks[3][6].
