Sr Engineer, Data Engineering

Job description

TransUnion's Job Applicant Privacy Notice Team Overview • We are seeking a highly skilled Cloud Data Engineer with strong expertise in Google Cloud Platform (GCP) to join our Cloud Strategy team. The ideal candidate will have 5-8 years of experience designing, implementing, optimizing, and governing cloud-based data platforms and analytics workloads. This role requires deep technical expertise in GCP Data Engineering services, cloud cost optimization, workload right-sizing, architecture design, resource tagging/governance, and FinOps practices. The candidate should be comfortable partnering with engineering, architecture, platform, and business teams to improve performance, scalability, reliability, and cloud cost efficiency.

This is a hybrid position and involves regular performance of job responsibilities virtually as well as in-person at an assigned TU office location for a minimum of two days a week.Role Overview And Core Responsibilities • ata Platform Architecture & Engineering • Design, implement, and optimize enterprise-scale data platforms on GCP. • Build scalable batch and streaming data pipelines. • Architect data ingestion, transformation, storage, and consumption solutions. • Evaluate and recommend the most appropriate GCP services for specific use cases. • Define standards and best practices for cloud-native data engineering. • GCP Workload Optimization • Analyze existing cloud workloads and identify optimization opportunities. • Recommend right-sizing strategies for compute, storage, and networking resources. • Improve resource utilization across data engineering platforms. • Optimize cluster sizing, auto-scaling policies, job scheduling, and resource allocation. • Reduce operational costs without impacting performance or reliability. • FinOps & Cloud Cost Management • Perform detailed cloud cost analysis and chargeback/showback reporting. • Establish cloud cost governance frameworks and optimization programs. • Identify idle, underutilized, and overprovisioned resources. • Develop cloud cost dashboards and KPIs. • Track budget consumption and provide cost-saving recommendations. • Build forecasting models for cloud spending. • Governance & Resource Management • Define and enforce metadata, labeling, and tagging standards. • Ensure cloud resources are properly categorized for cost allocation and accountability. • Implement governance controls for cloud resource lifecycle management. • Collaborate with platform and security teams to ensure compliance with organizational policies. • Performance Optimization • Tune data pipelines for throughput, latency, and cost efficiency. • Optimize BigQuery queries and storage patterns. • Improve Spark workload performance through cluster tuning. • Optimize data partitioning, clustering, indexing, and caching strategies. • Establish SLIs/SLOs for data platforms and monitor operational efficiency. • Stakeholder Collaboration • Work closely with architects, engineering teams, finance teams, and business stakeholders. • Provide consulting and technical guidance on cloud adoption and optimization. • Present cost-benefit analysis and architecture recommendations to leadership teams. Required Knowledge And Experiences • 5-8 years of experience in Data Engineering, Cloud Engineering, or Cloud Architecture. • Minimum 3+ years of hands-on experience with Google Cloud Platform. • Strong experience designing cloud-native data platforms. • Experience implementing cloud cost optimization initiatives. • Experience with enterprise-scale data processing environments. Required Technical Skills Google Cloud Platform (GCP) Strong knowledge and hands-on experience with: Data Engineering Services • BigQuery • Dataproc • Dataflow • Pub/Sub • Cloud Composer (Airflow) • Cloud Storage • BigLake • Data Fusion • Datastream • Cloud SQL • AlloyDB • Spanner • Looker / Looker Studio Compute & Infrastructure • GKE (Google Kubernetes Engine) • Compute Engine • Cloud Run • Serverless technologies • Instance Groups • Managed Services Monitoring & Operations • Cloud Monitoring • Cloud Logging • Cloud Trace • Cloud Profiler • Cloud Asset Inventory Cost Optimization & FinOps Skills Strong understanding of: • GCP pricing and billing