kafka.coordinator.assignors package¶
Submodules¶
kafka.coordinator.assignors.abstract module¶
-
class
kafka.coordinator.assignors.abstract.
AbstractPartitionAssignor
[source]¶ Bases:
object
Abstract assignor implementation which does some common grunt work (in particular collecting partition counts which are always needed in assignors).
-
assign
(cluster, members)[source]¶ Perform group assignment given cluster metadata and member subscriptions
- Arguments:
cluster (ClusterMetadata): metadata for use in assignment members (dict of {member_id: MemberMetadata}): decoded metadata for
each member in the group.- Returns:
- dict: {member_id: MemberAssignment}
-
metadata
(topics)[source]¶ Generate ProtocolMetadata to be submitted via JoinGroupRequest.
- Arguments:
- topics (set): a member’s subscribed topics
- Returns:
- MemberMetadata struct
-
name
¶ .name should be a string identifying the assignor
-
kafka.coordinator.assignors.roundrobin module¶
-
class
kafka.coordinator.assignors.roundrobin.
RoundRobinPartitionAssignor
[source]¶ Bases:
kafka.coordinator.assignors.abstract.AbstractPartitionAssignor
The roundrobin assignor lays out all the available partitions and all the available consumers. It then proceeds to do a roundrobin assignment from partition to consumer. If the subscriptions of all consumer instances are identical, then the partitions will be uniformly distributed. (i.e., the partition ownership counts will be within a delta of exactly one across all consumers.)
For example, suppose there are two consumers C0 and C1, two topics t0 and t1, and each topic has 3 partitions, resulting in partitions t0p0, t0p1, t0p2, t1p0, t1p1, and t1p2.
- The assignment will be:
- C0: [t0p0, t0p2, t1p1] C1: [t0p1, t1p0, t1p2]
When subscriptions differ across consumer instances, the assignment process still considers each consumer instance in round robin fashion but skips over an instance if it is not subscribed to the topic. Unlike the case when subscriptions are identical, this can result in imbalanced assignments.
For example, suppose we have three consumers C0, C1, C2, and three topics t0, t1, t2, with unbalanced partitions t0p0, t1p0, t1p1, t2p0, t2p1, t2p2, where C0 is subscribed to t0; C1 is subscribed to t0, t1; and C2 is subscribed to t0, t1, t2.
- The assignment will be:
- C0: [t0p0] C1: [t1p0] C2: [t1p1, t2p0, t2p1, t2p2]
-
name
= 'roundrobin'¶
-
version
= 0¶