Senior Data Engineer - Machine Learning Support

Job description

EY - GDS Consulting - AIA -Data Engineer -ML Support - Senior At EY, you'll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we're counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. Career Family Data \& AI -- Data Engineering \& Machine Learning Support Role Type Full Time Experience : 7--10 Years The Opportunity We are the only professional services organization who has a separate business dedicated exclusively to the financial services marketplace. Join Digital Engineering Team and you will work with multi-disciplinary teams from around the world to deliver a global perspective. Aligned to key industry groups including Asset management, Banking and Capital Markets, Insurance and Private Equity, Health, Government, Power and Utilities, we provide integrated advisory, assurance, tax, and transaction services. Through diverse experiences, world-class learning and individually tailored coaching you will experience ongoing professional development. That's how we develop outstanding leaders who team to deliver on our promises to all our stakeholders, and in so doing, play a critical role in building a better working world for our people, for our clients and for our communities. Sound interesting? Well, this is just the beginning. Because whenever you join, however long you stay, the exceptional EY experience lasts a lifetime. We are seeking a highly skilled Data Engineer with ML Support expertise to join our data and AI engineering practice. This role focuses on building, maintaining, and optimizing data pipelines that enable and support machine learning (ML) workloads and deployed models, particularly within AWS-based environments, while also leveraging Azure services where required. This is an exciting opportunity for a professional with a strong foundation in data engineering and cloud technologies, combined with an understanding of machine learning systems and lifecycle management. You will play a key role in ensuring that ML models are supported by robust, scalable, and reliable data pipelines that drive real-time and batch analytics. As a senior individual contributor, you will collaborate closely with data scientists, ML engineers, and platform teams to ensure smooth integration between data pipelines and machine learning systems, enabling efficient model training, deployment, and inference workflows. Your Key Responsibilities ML Pipeline Engineering \& Data Enablement Design, develop, and maintain data pipelines that support ML workflows, including data ingestion, transformation, and feature preparation Enable training, validation, and inference pipelines by providing high-quality, structured datasets Build scalable workflows using: AWS S3, AWS Glue, AWS Lambda Azure Data Factory Ensure efficient movement of data across storage, compute, and ML systems Support for ML Models \& Production Systems Support deployed ML models hosted in AWS environments, ensuring reliable data inputs and outputs Work closely with data scientists to operationalize ML models in production environments Ensure pipelines correctly integrate with model serving, inference layers, and downstream applications Handle model-related data dependencies, ensuring versioning, consistency, and traceability Data Engineering \& Processing Build robust data pipelines using Python or PySpark to handle large-scale datasets Design and implement ETL/ELT processes optimized for ML use cases Integrate structured and unstructured data from multiple sources Optimize pipelines for performance, scalability, and cost efficiency Data Quality, Reliability \& Monitoring Ensure high standards of data quality, integrity, and consistency across all ML pipelines Implement monitoring and alerting mechanisms for pipeline failures and data anomalies Troubleshoot issues related to data workflows, model inputs, and pipeline performance Support continuous improvement in pipeline reliability and operational efficiency Collaboration \& Cross-Functional Integration Collaborate with data scientists, ML engineers, and data engineers to enable seamless end-to-end ML workflows Translate data requirements into scalable pipeline implementations Work with platform teams to ensure proper integration between data engineering and ML systems Communicate effectively with stakeholders on progress, challenges, and solutions DevOps, CI/CD \& Engineering Practices Contribute to implementation of CI/CD pipelines for data and ML workflows Ensure adherence to engineering best practices, version control, and documentation standards Support deployment pipelines for ML-related data workflows Leverage tools such as Databricks or Foundry (where applicable) to enhance data processing and collaboration Learning \& Continuous Improvement Stay updated on advancements in ML engineering, MLOps, and cloud data platforms Contribute to improving data architectures and ML integration patterns Participate in knowledge-sharing sessions and continuous learning initiatives Identify opportunities to enhance pipeline efficiency and scalability Required Skills Strong hands-on experience in: Python or PySpark SQL and data processing techniques Solid experience in cloud-based data engineering: AWS (S3, Glue, Lambda) Azure (Data Factory) Experience in building and maintaining data pipelines and ETL/ELT workflows Good understanding of machine learning pipelines and lifecycle (training, deployment, inference) Experience supporting ML models in production environments Strong understanding of data structures, distributed systems, and large-scale data processing Ability to work independently and collaborate across cross-functional teams Good to Have Experience with: Databricks Foundry Exposure to CI/CD pipelines and DevOps practices for data and ML workflows Understa