The Producer API

The idea in one sentence

A producer builds a message (topic, optional key, value), hands it to send() which returns a Promise and completes asynchronously, and Kafka appends it to a partition of the topic.

The producer and the message

You create a producer once from a Kafka client, connect() it once, and reuse it for the life of your application. You never write raw bytes to a broker yourself — you hand send() a topic and one or more messages.

A message carries three things you care about most:

topic — the named stream you are writing to (set on the send() call, required).
key — optional; records with the same key go to the same partition, which is how Kafka preserves per-key ordering.
value — the payload itself (the event data).

Because a broker only stores bytes, a message key and value must be a string, a Buffer, or null. KafkaJS has no pluggable serializer classes — you serialize the payload yourself, typically JSON.stringify(...) for a structured value and a plain string for the key. (Other clients, like the Java one, instead configure key.serializer and value.serializer classes.)

import { Kafka } from 'kafkajs';

// brokers: the initial broker list the client contacts to discover the whole cluster
const kafka = new Kafka({ clientId: 'orders-app', brokers: ['localhost:9092'] });
const producer = kafka.producer();

send() is asynchronous — await the Promise

send() does not block until the record reaches the broker. KafkaJS buffers and batches internally, then returns a Promise. To learn the outcome — the partition and offset Kafka assigned — await the promise (or use .then()); on failure it throws, so wrap it in try/catch.

import { Kafka } from 'kafkajs';

const kafka = new Kafka({ clientId: 'orders-app', brokers: ['localhost:9092'] });
const producer = kafka.producer();

async function main() {
  await producer.connect();                  // connect once, then reuse the producer
  try {
    // one message: topic "orders", key "order-42", JSON-serialized value
    const [meta] = await producer.send({
      topic: 'orders',
      messages: [{ key: 'order-42', value: JSON.stringify({ amount: 19.99 }) }],
    });
    // success — Kafka assigned this record a partition and a base offset
    console.log(`sent to ${meta.topicName}-${meta.partition} @ offset ${meta.baseOffset}`);
  } catch (err) {
    // the promise rejected — log, retry, or route to a dead-letter path
    console.error('send failed:', err);
  } finally {
    await producer.disconnect();             // flush buffered records, release resources
  }
}

main();

flowchart LR
  app["Your app builds a message"] --> ser["Serialize key/value yourself, e.g. JSON.stringify"]
  ser --> buf["Producer buffer (per partition)"]
  buf --> io["KafkaJS batches and sends"]
  io --> broker["Broker appends to partition"]
  broker --> cb["Promise resolves: metadata, or throws"]

From a message to an assigned offset