You measure the throughout that you can achieve on a single partition for production (call it p) and consumption (call it c). Let’s say your target throughput is t.
t = throughput (QPS) p = single partition for production c = consumption
3 Matching Annotations
- Jun 2017
-
www.confluent.io www.confluent.io
-
-
cwiki.apache.org cwiki.apache.org
-
Designing a High Level Consumer
By far the most important thing you need to know to make SECOR operate with Kafkaf
-
- May 2017
-
stackoverflow.com stackoverflow.com
-
the number of partitions -- there's no real "formula" other than this: you can have no more parallelism than you have partitions.
This is an important thing to keep in mind. If we need massive parallelism we need to have more partitions.
-