某一个分区不可用对整个topic的影响
因某些原因,某个分区不可用,比如leader节点挂了,无法切换leader。剩下的分区都是正常的,那么原来的生产者和消费者会以什么样的形态继续下去?
某个Broker节点因认证失败导致所有节点网络流量番两倍的原因
kerberos认证下,某个Broker节点因hostname变更,客户端连接此节点时认证失败,此后,集群所有节点的网络流量飙升。
Broker节点无故退出
Broker节点因未知原因退出,没有shutdown的日志,服务器dmesg、messages和audit都么有看到异常记录,以及操作历史记录中未发生人为的kill。
修改zookeeper.connect中的zk-root-node
在一个kafka集群中,如果修改zookeeper.connect的zk-root-node,而不修改其他配置,也不清理logs.dir对应的数据目录,则修改后,kafka服务无法重启,broker报以下错误:
[2025-06-03 18:03:54,299] [INFO ] [feature-zk-node-event-process-thread:1735] [kafka.server.FinalizedFeatureChangeListener] [Logging.scala:66] - Feature ZK node at path: /feature does not exist
[2025-06-03 18:03:54,299] [INFO ] [feature-zk-node-event-process-thread:1735] [kafka.server.FinalizedFeatureCache] [Logging.scala:66] - Cleared cache
[2025-06-03 18:03:54,440] [INFO ] [main:1876] [kafka.server.KafkaServer] [Logging.scala:66] - Cluster ID = Zd2tLmm2RCynoYc6pJaNBQ
[2025-06-03 18:03:54,446] [ERROR] [main:1882] [kafka.server.KafkaServer] [Logging.scala:82] - Fatal error during KafkaServer startup. Prepare to shutdown
kafka.common.InconsistentClusterIdException: The Cluster ID Zd2tLmm2RCynoYc6pJaNBQ doesn't match stored clusterId Some(8LNn5_BVR4C-FKcRGlP3vg) in meta.properties. The broker is trying to join the wrong cluster. Configured zookeeper.connect may be wrong.
at kafka.server.KafkaServer.startup(KafkaServer.scala:218) [kafka_2.13-2.8.2.jar:?]
at kafka.Kafka$.main(Kafka.scala:109) [kafka_2.13-2.8.2.jar:?]
at kafka.Kafka.main(Kafka.scala) [kafka_2.13-2.8.2.jar:?]
[2025-06-03 18:03:54,450] [INFO ] [main:1886] [kafka.server.KafkaServer] [Logging.scala:66] - shutting down
[2025-06-03 18:03:54,453] [INFO ] [main:1889] [kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread] [Logging.scala:66] - [feature-zk-node-event-process-thread]: Shutting down
[2025-06-03 18:03:54,454] [INFO ] [main:1890] [kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread] [Logging.scala:66] - [feature-zk-node-event-process-thread]: Shutdown completed
新生成的/brokers/ids为空
将meta.properties中的cluster.id改为与zknode(/cluster/id)中的集群id一致时,启动正常
但启动完成后,topic数据无法加载,zk上的topics为空。
结论:如果只是修改zookeeper.connect,无法加载原来的集群数据。修改后必定是一个新集群。