Skip to content

Configuration

klite is configured via command-line flags or environment variables. Environment variables use the KLITE_ prefix and are overridden by explicit flags. Both work in Docker:

Terminal window
docker run --network host \
-e KLITE_LOG_LEVEL=debug \
ghcr.io/klaudworks/klite --default-partitions 3
FlagEnvDefaultDescription
--listenKLITE_LISTEN:9092Address and port to bind to. Use :9092 for all interfaces, or 127.0.0.1:9092 for localhost only.
--advertised-addrKLITE_ADVERTISED_ADDR(derived)Address clients use to reconnect after metadata discovery. See below.
--health-addrKLITE_HEALTH_ADDR(disabled)HTTP server for /livez and /readyz health checks. Set to :8080 to enable.

The advertised address is what klite tells Kafka clients to connect to. Kafka clients first connect to any broker, receive a metadata response containing the advertised address, then reconnect to that address for all subsequent requests. Getting this wrong is the most common source of connection issues.

When you need to set this:

  • Docker with --network host — not needed, localhost works naturally.
  • Docker with port mapping (-p 9092:9092) — the container’s internal hostname differs from the host. Use --advertised-addr localhost:9092 so host-side clients can reconnect.
  • Kubernetes — set to the pod’s headless service address, e.g. klite-0.klite.default.svc.cluster.local:9092.
  • Behind a load balancer or NAT — set to the external address clients use.

If unset, klite derives it from --listen. When listening on 0.0.0.0 or ::, it defaults to localhost:<port> and logs a warning.

FlagEnvDefaultDescription
--data-dirKLITE_DATA_DIR./dataDirectory for WAL segments, metadata log, and meta.properties. Created automatically on first start.
--wal-max-disk-sizeKLITE_WAL_MAX_DISK_SIZE1073741824Maximum WAL size on disk in bytes (default 1 GiB). Oldest segments are removed when this limit is reached.

When running in Docker without S3, the data directory lives inside the container and is lost when it stops. For durable storage, configure an S3 backend.

When an S3 bucket is configured, klite periodically flushes WAL data to S3 for durable storage. The local WAL acts as a write-ahead buffer — writes are acknowledged immediately and flushed in the background.

Any S3-compatible store works: AWS S3, SeaweedFS, MinIO, Garage, etc.

FlagEnvDefaultDescription
--s3-bucketKLITE_S3_BUCKET(none)Bucket name. If unset, klite runs in local-only mode.
--s3-regionKLITE_S3_REGIONus-east-1AWS region.
--s3-endpointKLITE_S3_ENDPOINT(none)Custom endpoint for S3-compatible stores.
--s3-prefixKLITE_S3_PREFIXklite/<clusterID>Key prefix for all objects.
--s3-flush-intervalKLITE_S3_FLUSH_INTERVAL60sMax age of unflushed partition data before flush. A partition also flushes when it reaches 64 MB.

Credentials use the standard AWS environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and optionally AWS_SESSION_TOKEN. IAM roles, instance profiles, and IRSA work too.

When --s3-endpoint is set, path-style addressing is enabled automatically (required by most S3-compatible stores).

The flush interval and size threshold together determine your recovery point objective (RPO). With the defaults (60s / 64 MB per partition), at most 60 seconds of data per partition is at risk if the disk is lost before the next flush. Lower the interval for tighter RPO at the cost of more S3 PUT requests.

Compaction rewrites overlapping S3 objects for topics using cleanup.policy=compact, removing superseded keys and expired tombstones.

FlagEnvDefaultDescription
--compaction-check-interval(flag only)2mHow often klite scans partition dirty counters for compaction eligibility.
--compaction-min-dirty-objects(flag only)16Minimum dirty objects before a partition is considered for compaction.
--compaction-read-rate(flag only)52428800Max S3 read throughput for compaction in bytes/sec (50 MiB/s). Set 0 for unlimited.

Tuning guidance:

  • Low-traffic clusters: use lower thresholds (for example, --compaction-check-interval=30s and --compaction-min-dirty-objects=4) to reclaim space sooner.
  • High-throughput clusters: keep or raise defaults (for example, 2m/16 or 5m/32) to reduce constant background compaction pressure.
  • Fetch latency sensitive workloads: keep --compaction-read-rate capped so compaction reads do not contend with client fetches.
FlagEnvDefaultDescription
--default-partitionsKLITE_DEFAULT_PARTITIONS1Partition count for auto-created topics.
--auto-create-topicsKLITE_AUTO_CREATE_TOPICStrueCreate topics on first Produce or Metadata request. Set to false to require explicit CreateTopics.

See Topic configuration for per-topic overrides like retention.

klite supports SASL for client authentication. When enabled, all connections must authenticate before sending any other request.

FlagEnvDefaultDescription
--sasl-enabledKLITE_SASL_ENABLEDfalseRequire SASL authentication.
--sasl-mechanismKLITE_SASL_MECHANISMPLAINMechanism: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512.
--sasl-userKLITE_SASL_USER(none)Bootstrap admin username.
--sasl-passwordKLITE_SASL_PASSWORD(none)Bootstrap admin password.
FlagEnvDefaultDescription
--cluster-idKLITE_CLUSTER_ID(auto-generated)Kafka cluster ID. Generated on first start and persisted in meta.properties.
--node-idKLITE_NODE_ID0Broker node ID reported in Metadata responses.
--log-levelKLITE_LOG_LEVELinfoLog verbosity: debug, info, warn, error.