Gcp Data Engineer

Job description

**Data Engineering & Pipeline Development** - Design, develop, and maintain scalable **batch and real-time data pipelines** on Google Cloud Platform (GCP). - Build reliable and efficient data ingestion, transformation, and processing frameworks using **BigQuery, Dataflow, Dataproc, BigTable, Pub/Sub, Cloud Storage, and Data Fusion**. - Create and operationalize data pipelines by integrating multiple enterprise data sources while ensuring data quality, consistency, and availability. - Develop prototype analytics pipelines to generate business insights and support product innovation. **AI/ML & Predictive Analytics** - Build, train, and optimize machine learning models for **VIN-level predictive risk scoring**, **loss ratio forecasting**, and **high-utilization contract identification**. - Perform advanced **Exploratory Data Analysis (EDA)** on historical claims, contracts, and operational datasets from sources such as OWS, PTS, UDB, and DMS. - Engineer business-driven features, including complex metrics such as **Claim vs. Unclaimed Learnings** and **High Time In Service (HTIS)**. - Support AI/ML solution design, experimentation, and prototype development using Python and GCP services. **MLOps & Model Lifecycle Management** - Manage the complete machine learning lifecycle, including: - Data preprocessing and feature engineering - Model training and validation - Deployment and productionization - Performance monitoring and retraining - Implement MLOps best practices to ensure model scalability, reliability, and maintainability. - Monitor model performance and proactively address model drift and data quality issues. **Cloud Platform & Architecture** - Design, build, secure, monitor, and optimize data processing systems on Google Cloud Platform. - Develop expertise in Google technologies, architectures, and data structures to support future product development and business initiatives. - Implement cloud-native solutions following scalability, security, and cost-optimization best practices. **Data Quality, Testing & Governance** - Perform unit testing, integration testing, and validation of data pipelines and analytical solutions. - Identify and resolve defects, data inconsistencies, and performance bottlenecks. - Establish data quality checks, monitoring frameworks, and governance standards. - Ensure compliance with organizational policies, ethical AI practices, and data privacy regulations. **DevOps & Automation** - Utilize **Git, Jenkins, Terraform, Tekton**, and CI/CD pipelines to automate deployments and infrastructure management. - Support Infrastructure as Code (IaC) practices for cloud resource provisioning and configuration management. - Implement automation to improve deployment efficiency, reliability, and repeatability. **Collaboration & Agile Delivery** - Collaborate with business stakeholders, product owners, data scientists, and engineering teams to understand requirements and deliver data-driven solutions. - Participate actively in Agile ceremonies, including: - Sprint Planning - Backlog Grooming & Prioritization - Daily Standups - Sprint Reviews - Retrospectives - Provide technical recommendations and contribute to product roadmap discussions. **Technical Skills** - Strong proficiency in **Python, SQL, and Google Cloud Platform (GCP)**. - Experience with Big Data technologies and distributed data processing frameworks. - Knowledge of machine learning, predictive analytics, feature engineering, and model deployment. - Familiarity with DevOps, CI/CD, monitoring, and cloud security best practices.