Lead Data Engineer

Job description

**About the Role** The Data Platform team at Chargebee builds and maintains scalable data systems that power internal analytics, business intelligence, and customer-facing data featur es.As **a Lead Data Engin** eer, you will play a key role in shaping the architecture, scalability, and reliability of Chargebee’s data platform. You will lead the design and development of large-scale data systems, mentor engineers on the team, and drive best practices across data engineering workflo ws.You will work closely with product engineers, analysts, platform teams, and leadership to ensure that data is ingested, processed, and made available efficiently for analytics and product use cases. This role involves designing robust data pipelines, optimizing distributed data processing systems, and guiding the evolution of the data platform to support Chargebee’s growing data need s.The team operates in a fast-paced and collaborative environment, building reliable and scalable infrastructure that powers data-driven decision making across the compa **ny What You Will Wor** k **On As a Lead Dat** a Engineer, you will lead the development and evolution of Chargebee’s data platform. This includes designing scalable data architectures, building robust ingestion and processing pipelines, and ensuring data systems operate reliably at scale.You will also guide the technical direction of the platform, mentor engineers, and collaborate across teams to enable efficient and scalable data work flows. The role provides expos - ure to: Large-scale data ingestion and processing p - ipelinesStreaming and event-driven archi - tecturesDistributed data processing fr - ameworksCloud-based data infras - tructureBuilding and maintaining data lake and data warehouse archi - tecture.Designing scalable data platforms powering both internal analytics and customer-facing - productsLeading architectural decisions and platform evolution for large-scale data **systems Key Respons** - ibilities Design and architect scalable, reliable data ingestion and processing pipelines across the data - platform.Lead the development and optimization of ETL/ELT workflows to support high-volume and scalable data p - rocessing.Build and maintain distributed data processing systems using framework **s such as Ap** a - che Spark.Design and implement event-driven data architectures using streaming system **s suc** h - as Kafka.Define data modeling standards and transformation strategies to support analytics and product - use cases.Ensure high sta **ndards of data reliability, integrity, scalability, and p** erformance across the data - platform.Lead troubleshooting and debugging efforts for complex production data pipelines and distribute - d systems.Collaborate with data analysts, product teams, and engineering teams to design scalable data - solutions.Mentor and guide data engineers, providing technical leadership and promoting engineering best - practices.Lead design discussions, architecture reviews, and technical decision-making within the data plat - form team.Participate in **and drive code reviews, technical design reviews, and agile development p** - rocesses. Document architecture decisions, platform standards, and data engineering **workflows. Minimum Qua** - lificationsBachelor’s degree in Computer Science, Mathematics, Engineering, or a related technical field, or equivalent practical - **experience.6+ years of** experience building and maintaining large-scale data processing systems and - pipelines.Strong experience designing and **operating production-grade distributed d** a - ta systems.Hands-on experience with distributed computing framewor **ks such as A** p - ache Spark.Strong prof **iciency in SQL and da** t - a modeling.Proficiency in at least one programming langua **ge such as Java, Pytho** n - , or Scala.Strong unders **tanding of data structures, distributed systems, and data platform a** r - chitecture.Experience working with relational databas **es such as PostgreSQL, MySQL, or simi** l - ar systems.Experience designing an **d building ETL/ELT dat** a pipeline - s at scale.Experience wo **rking with Gi** t workflows in collaborative development en - vironments.Experience wor **king in an Agile development** e **nvironment. Good-to-Have Qu** - alifications Experience workin **g within the** A - WS ecosystem.Experience building and operating **large-scale Apache Spark-based d** a - ta pipelines.Exp **erience with streaming systems such as Kafka or similar event-dri** v - en platforms.Strong unde **rstanding of data lake and data warehouse** - architecturesExperien **ce designing scalable cloud-native d** a - ta platforms.Knowledge or prior exp **erience with open table formats such as Delta Lake, Apache Iceberg, o** r - Apache Hudi.Strong technical leadership, communication, and problem-so - lving skills.Ability to investigate and debug i **ssues across large-scale distri** b uted systems.