A topic usually resides within the broker (server process) and has multiple partitions, but can it span across multiple brokers or nodes (machines)?
i.e, lets say a topic is divided into two partitions, partition-1 and partition-2. Can partition-1 belong to broker-1 and partition-2 of the same topic belong to a different broker-2 or a different node altogether?
A Kafka cluster consists of one or more servers (Kafka brokers).
Each Broker can have one or more Topics. Kafka topics are divided into a number of partitions, each partition can be placed on a single or separate machine to allow for multiple consumers to read from a topic in parallel.
Yes, different partitions of a topic can be stored in multiple machines. But, a single partition has to be stored entirely in one machine.
Th kafka topics are replicated across multiple brokers so that multiple consumers can receive the data parallely or in case the one broker goes down , the data can be retrieved from other broker with the same topic partition.