Sr Manager - Customer Interaction Suite

Job description

Job Summary A Senior Data Platform Engineer with 12-14 years of experience to own and build the core data platform for a multi-region CPaaS system supporting multiple digital channels. The platform processes 40,000+ transactions per second, with multiple (10) state updates per transaction, and powers real-time metrics, billing, reporting, campaign analytics, and audits. This role is hands-on and ownership-driven, focused on data correctness, scalability, and reliability in large-scale streaming systems. Responsibilities Mandatory / Primary Responsibilities • Own the end-to-end data architecture for high-throughput, event-driven CPaaS systems. • Design state-transition and event-sourced data models handling millions of updates per second. • Build and operate streaming data pipelines using Apache Pulsar for ingestion and Apache Flink for stateful processing and real-time aggregation. • Design and maintain a data lake using Apache Hudi to support: Upserts and deletes • Incremental queries • Time-travel and auditability • Build real-time analytics datasets in ClickHouse for dashboards, tenant metrics, and campaign analysis. • Define and enforce processing semantics, including idempotency, deduplication, ordering, and replay safety. • Own data correctness, completeness, and freshness SLAs across streaming and analytics systems. • Design multi-region data ingestion and aggregation, including replication, failover, and reprocessing. • Lead hands-on development, code reviews, performance tuning, and production troubleshooting. Secondary Responsibilities • Mentor senior engineers and review architectural designs. • Define data standards, schema evolution practices, and platform guidelines. • Participate in release planning and technical prioritization. • Evaluate new data technologies and patterns through POCs. • Collaborate with product and operations teams on data-driven features. Skills Mandatory / Primary Skills • Apache Pulsar (topics, partitions, subscriptions, geo-replication) • Apache Flink (stateful stream processing, windows, checkpoints) • Apache Hudi (Merge-On-Read, Copy-On-Write, incremental processing) • ClickHouse (real-time analytics, high-cardinality datasets) • Event-driven and streaming data architectures • Event sourcing and state-transition modeling • Idempotent replay-safe data pipelines • Designing systems for high TPS and frequent state updates • Multi-region data design and disaster recovery Secondary Skills • Redis for low-latency caching, counters, and ephemeral state • PostgreSQL for transactional, relational, and reference data • MySQL for operational data stores and legacy integrations • Low-Level Design (LLD) for scalable, fault-tolerant components • High-Level Design (HLD) for distributed, multi-region systems • NoSQL stores for hot state (e.g., Mongo, DynamoDB, Cassandra) • Relational databases for reference/config data • Data lake query engines (Athena, Spark) • Observability for data pipelines (lag, freshness, reconciliation) • Containerized runtimes and cloud-native deployments • Prior experience with CPaaS, telecom, or large-scale messaging platforms Experience Background • 12-14 years of experience in data engineering, platform engineering, or distributed systems • Proven experience building large-scale, streaming-heavy data platforms • Strong hands-on background in real-time analytics and data consistency • Comfortable operating at Principal Staff Engineer level with full technical ownership