Getting Started with Kafka

A distributed streaming platform

What is Kafka for?

Is a producer consumer system.

Terminology

1. Topics
2. Partition — Topics are partitioned for faster read and write of the data

Kafka in Spring

Annotation : @KafkaListener

Logic

import org.springframework.core.task.SimpleAsyncTaskExecutor;
import org.springframework.core.task.TaskExecutor;
import org.springframework.kafka.annotation.KafkaListener;
import org.apache.kafka.clients.admin.NewTopic;
private final TaskExecutor exec = new SimpleAsyncTaskExecutor();
@KafkaListener(id = "fooGroup", topics = "topic1") 
public void listen(Foo2 foo) {  
    this.exec.execute(() -> {} ); 
}
@Bean
public NewTopic topic() {  
    return new NewTopic("topic1", 1, (short) 1); 
}

Controller

@Autowired private KafkaTemplate<Object, Object> template; 
@PostMapping(path = "/send/foo/{what}") 
public void sendFoo(@PathVariable String what) {
      this.template.send("topic1", new Foo1(what)); 
}

Kafka APIs

Admin API (inspect topics), Producer API (write), Consumer API (read), Kafka Streams API, Kafka Connect API