Kafka服务端配置


发布于 2024-04-02 / 93 阅读 / 0 评论 /
本文基于kafka2.7源码,介绍Kafka服务端相关的配置,属性定义在类kafka.server.KafkaConfig.scala中,通过功能分类,可分为以下20个大类,共215个配置属性,下面分别进行介绍。

Zookeeper Configuration

Zookeeper配置参数有20个

zookeeper.connect

zookeeper.session.timeout.ms

zookeeper.connection.timeout.ms

zookeeper.sync.time.ms

zookeeper.set.acl

zookeeper.max.in.flight.requests

zookeeper.ssl.client.enable

zookeeper.clientCnxnSocket

zookeeper.ssl.keystore.location

zookeeper.ssl.keystore.password

zookeeper.ssl.keystore.type

zookeeper.ssl.truststore.location

zookeeper.ssl.truststore.password

zookeeper.ssl.truststore.type

zookeeper.ssl.protocol

zookeeper.ssl.enabled.protocols

zookeeper.ssl.cipher.suites

zookeeper.ssl.endpoint.identification.algorithm

zookeeper.ssl.crl.enable

zookeeper.ssl.ocsp.enable

General Configuration

Kafka基本配置参数有13个

broker.id.generation.enable

reserved.broker.max.id

broker.id

message.max.bytes

num.network.threads

num.io.threads

background.threads

num.replica.alter.log.dirs.threads

queued.max.requests

queued.max.request.bytes

request.timeout.ms

socket.connection.setup.timeout.ms

socket.connection.setup.timeout.max.ms

Authorizer Configuration

认证相关参数有1个

authorizer.class.name

Socket Server Configuration

Socket Server相关配置有17个

port

host.name

listeners

advertised.host.name

advertised.port

advertised.listeners

listener.security.protocol.map

control.plane.listener.name

socket.send.buffer.bytes

socket.receive.buffer.bytes

socket.request.max.bytes

max.connections.per.ip

max.connections.per.ip.overrides

max.connections

max.connection.creation.rate

connections.max.idle.ms

connection.failed.authentication.delay.ms

Rack Configuration

机架感知相关配置有1个

broker.rack

Log Configuration

kafka日志相关配置有44个

num.partitions

log.dirs

log.dir

log.segment.bytes

log.roll.ms

log.roll.hours

log.roll.jitter.ms

log.roll.jitter.hours

log.retention.ms

log.retention.minutes

log.retention.hours

log.retention.bytes

log.retention.check.interval.ms

log.cleanup.policy

log.cleaner.threads

log.cleaner.io.max.bytes.per.second

log.cleaner.dedupe.buffer.size

log.cleaner.io.buffer.size

log.cleaner.io.buffer.load.factor

log.cleaner.backoff.ms

log.cleaner.min.cleanable.ratio

log.cleaner.enable

log.cleaner.delete.retention.ms

log.cleaner.min.compaction.lag.ms

log.cleaner.max.compaction.lag.ms

log.index.size.max.bytes

log.index.interval.bytes

log.flush.interval.messages

log.segment.delete.delay.ms

log.flush.scheduler.interval.ms

log.flush.interval.ms

log.flush.offset.checkpoint.interval.ms

log.flush.start.offset.checkpoint.interval.ms

log.preallocate

{LogConfigPrefix}message.format.version

{LogConfigPrefix}message.timestamp.type

{LogConfigPrefix}message.timestamp.difference.max.ms

{LogConfigPrefix}max.id.map.snapshots

num.recovery.threads.per.data.dir

auto.create.topics.enable

min.insync.replicas

create.topic.policy.class.name

alter.config.policy.class.name

{LogConfigPrefix}message.downconversion.enable

Replication configuration

复制备份相关参数有23个

controller.socket.timeout.ms

default.replication.factor

replica.lag.time.max.ms

replica.socket.timeout.ms

replica.socket.receive.buffer.bytes

replica.fetch.max.bytes

replica.fetch.wait.max.ms

replica.fetch.min.bytes

replica.fetch.response.max.bytes

replica.fetch.backoff.ms

num.replica.fetchers

replica.high.watermark.checkpoint.interval.ms

fetch.purgatory.purge.interval.requests

producer.purgatory.purge.interval.requests

delete.records.purgatory.purge.interval.requests

auto.leader.rebalance.enable

leader.imbalance.per.broker.percentage

leader.imbalance.check.interval.seconds

unclean.leader.election.enable

security.inter.broker.protocol

inter.broker.protocol.version

inter.broker.listener.name

replica.selector.class

Controlled shutdown configuration

controller shutdown相关参数有3个

controlled.shutdown.max.retries

controlled.shutdown.retry.backoff.ms

controlled.shutdown.enable

Group coordinator configuration

Group coordinator相关配置有4个

group.min.session.timeout.ms

group.max.session.timeout.ms

group.initial.rebalance.delay.ms

group.max.size

Offset management configuration

offset管理相关配置有10个

offset.metadata.max.bytes

offsets.load.buffer.size

offsets.topic.replication.factor

offsets.topic.num.partitions

offsets.topic.segment.bytes

offsets.topic.compression.codec

offsets.retention.minutes

offsets.retention.check.interval.ms

offsets.commit.timeout.ms

offsets.commit.required.acks

Transaction management configuration

事务管理相关配置有9个

transactional.id.expiration.ms

transaction.max.timeout.ms

transaction.state.log.min.isr

transaction.state.log.load.buffer.size

transaction.state.log.num.partitions

transaction.state.log.segment.bytes

transaction.state.log.replication.factor

transaction.abort.timed.out.transaction.cleanup.interval.ms

transaction.remove.expired.transaction.cleanup.interval.ms

Fetch Configuration

副本拉取leader数据相关配置有2个

max.incremental.fetch.session.cache.slots

fetch.max.bytes

Quota Configuration

kafka quota相关配置有13个

quota.producer.default

quota.consumer.default

quota.window.num

replication.quota.window.num

alter.log.dirs.replication.quota.window.num

controller.quota.window.num

quota.window.size.seconds

replication.quota.window.size.seconds

alter.log.dirs.replication.quota.window.size.seconds

controller.quota.window.size.seconds

client.quota.callback.class

delete.topic.enable

compression.type

Kafka Metrics Configuration

kafka监控指标相关配置有4个

metrics.sample.window.ms

metrics.num.samples

metric.reporters

metrics.recording.level

Kafka Yammer Metrics Reporters Configuration

Kafka Yammer Metrics Reporters相关配置有2个

kafka.metrics.reporters

kafka.metrics.polling.interval.secs

Common Security Configuration

基本安全配置有以下3个。

principal.builder.class

connections.max.reauth.ms

security.providers

SSL Configuration

SSL相关配置有以下21个。

ssl.protocol

ssl.provider

ssl.cipher.suites

ssl.enabled.protocols

ssl.keystore.type

ssl.keystore.location

ssl.keystore.password

ssl.key.password

ssl.keystore.key

ssl.keystore.certificate.chain

ssl.truststore.type

ssl.truststore.location

ssl.truststore.password

ssl.truststore.certificates

ssl.keymanager.algorithm

ssl.trustmanager.algorithm

ssl.endpoint.identification.algorithm

ssl.secure.random.implementation

ssl.client.auth

ssl.principal.mapping.rules

ssl.engine.factory.class

SASL Configuration

SASL相关配置有以下17个。

sasl.mechanism.inter.broker.protocol

sasl.jaas.config

sasl.enabled.mechanisms

sasl.server.callback.handler.class

sasl.client.callback.handler.class

sasl.login.class

sasl.login.callback.handler.class

sasl.kerberos.service.name

sasl.kerberos.kinit.cmd

sasl.kerberos.ticket.renew.window.factor

sasl.kerberos.ticket.renew.jitter

sasl.kerberos.min.time.before.relogin

sasl.kerberos.principal.to.local.rules

sasl.login.refresh.window.factor

sasl.login.refresh.window.jitter

sasl.login.refresh.min.period.seconds

sasl.login.refresh.buffer.seconds

Delegation Token Configuration

Delegation Token相关配置有以下4个。

delegation.token.master.key

delegation.token.max.lifetime.ms

delegation.token.expiry.time.ms

delegation.token.expiry.check.interval.ms

Password encryption configuration for dynamic configs

动态配置值中密码加密相关配置有以下6个

password.encoder.secret

password.encoder.old.secret

password.encoder.keyfactory.algorithm

password.encoder.cipher.algorithm

password.encoder.key.length

password.encoder.iterations