Changing and Optimizing Partition Count

KSQL automatically creates Kafka topics to hold its results, but you may still have to think about topic sizing. In cases where you don’t want to use the cluster-wide default number of partitions, you can specify the number of partitions you want to use.

Environment 4.1 or higher


In any KSQL query that will create a new Kafka topic, simply add the PARTITIONS parameter to the WITH clause. The new topic will now have that number of partitions.

1. Consider a Kafka topic called clickstream-raw. We don’t even need to know how many partitions it has, but we do know we need a new topic that has five partitions.

2. Create a STREAM from that partition:

CREATE STREAM clickstream-raw WITH (KAFKA_TOPIC=’clickstream-raw’,VALUE_FORMAT=’JSON’);

3. Create a new stream with the desired number of partitions. The newly created topic will be partitioned the way we specify.

CREATE STREAM clickstream_new WITH (PARTITIONS=5) AS SELECT * from clickstream_raw;

Nous utilisons des cookies afin de comprendre comment vous utilisez notre site et améliorer votre expérience. Cliquez ici pour en apprendre davantage ou pour modifier vos paramètres de cookies. En poursuivant la navigation, vous consentez à ce que nous utilisions des cookies.