Kafka Admin

Job description

Job Summary We are seeking an experienced Kafka Administrator to manage and maintain enterprise-level Apache Kafka clusters. The ideal candidate will be responsible for installation, configuration, monitoring, performance tuning, and ensuring high availability of Kafka infrastructure in distributed environments. Educational Requirements Bachelor of Engineering, BTech, BSc, BCA, MTech, MCA, MSc Service Line Data Analytics Unit Responsibilities - Kafka Cluster Administration - Install, configure, and maintain Apache Kafka clusters - Manage Kafka brokers, topics, partitions, and replication - Perform cluster upgrades, scaling, and capacity planning - Monitoring Performance Tuning - Monitor Kafka cluster health using tools like Prometheus, Grafana, Confluent Control Center - Optimize broker performance, throughput, and latency - Tune configurations for producers and consumers - Data Streaming Integration - Manage real-time data pipelines using Kafka - Handle integration with Spark, Hadoop, Flink - Databases APIs - Support streaming use cases and event-driven architecture - Security Governance - Implement authentication and authorization (SSL, SASL, Kerberos) - Configure ACLs (Access Control Lists) - Ensure secure data transmission and compliance - Replication High Availability - Ensure fault tolerance and data replication across brokers - Configure multi-cluster setups (MirrorMaker, Cluster Linking) - Handle disaster recovery strategies - Troubleshooting Support - Diagnose and fix issues like Broker failures - Consumer lag - Performance bottlenecks - Analyze logs and perform root cause analysis - Automation DevOps - Automate deployments using Ansible, Terraform, or scripts - Manage CI/CD pipelines for Kafka environments - Version control using Git - Zookeeper / KRaft Management - Manage Zookeeper clusters (if applicable) - Configure Kafka in KRaft mode (Zookeeper-less architecture) Additional Responsibilities - Experience with Confluent Platform - Knowledge of Kubernetes (Kafka on K8s) - Experience with DevOps tools and CI/CD pipelines - Certification in Kafka / Confluent (nice to have) Technical and Professional Requirements - Primary skills: Kafka administration Preferred Skills - Domain- >Healthcare- >Administrative tools - Technology- >Integration- >Confluent Kafka