We recommend monitoring GC time and other stats and various server stats such as CPU utilization, I/O service time, etc. On the client side, we recommend monitoring the message/byte rate (global and per topic), request rate/size/time, and on the consumer side, max lag in messages among all partitions and min fetch request rate. For a consumer to keep up, max lag needs to be less than a threshold and min fetch rate needs to be larger than 0.
EDP
- 1. GETTING STARTED
- 2. APIS
- 3. CONFIGURATION
- 4. DESIGN
- 5. IMPLEMENTATION
- 6. OPERATIONS
- 6.1 Basic Kafka Operations
- 6.1.1 Adding and removing topics
- 6.1.2 Modifying topics
- 6.1.3 Graceful shutdown
- 6.1.4 Balancing leadership
- 6.1.5 Checking consumer position
- 6.1.6 Mirroring data between clusters
- 6.1.7 Expanding your cluster
- 6.1.8 Decommissioning brokers
- 6.1.9 Increasing replication factor
- 6.2 Datacenters
- 6.3 Important Configs
- 6.3.1 Important Client Configs
- 6.3.2 A Production Server Configs
- 6.4 Java Version
- 6.5 Hardware and OS
- 6.5.1 OS
- 6.5.2 Disks and Filesystems
- 6.5.3 Application vs OS Flush Management
- 6.5.4 Linux Flush Behavior
- 6.5.5 Ext4 Notes
- 6.6 Monitoring
- 6.6.1 Selector Monitoring
- 6.6.2 Common Node Monitoring
- 6.6.3 Producer Monitoring
- 6.6.4 Consumer Monitoring
- 6.6.5 Connect Monitoring
- 6.6.6 Streams Monitoring
- 6.6.7 Others
- 6.7 ZooKeeper
- 6.7.1 Stable Version
- 6.7.2 Operationalization
- 7. SECURITY
- 7.1 Security Overview
- 7.2 Encryption and Authentication using SSL
- 7.3 Authentication using SASL
- 7.4 Authorization and ACLs
- 7.5 Incorporating Security Features in a Running Cluster
- 7.6 ZooKeeper Authentication
- 7.6.1 New Clusters
- 7.6.2 ZooKeeper SASL Authentication
- 7.6.3 ZooKeeper Mutual TLS Authentication
- 7.6.4 Migrating Clusters
- 7.6.5 Migrating the ZooKeeper Ensemble
- 7.6.6 ZooKeeper Quorum Mutual TLS Authentication
- 7.7 ZooKeeper Encryption
- 8. KAFKA CONNECT
- 9. KAFKA STREAMS
- Home
- Docs
- EDP
- 6. OPERATIONS
- 6.6.7 Others