CPW Data Engineer

Job description

OVERVIEW Cereal Partners Worldwide (CPW) is a joint venture between General Mills and Nestl, two of the worlds leading food organizations. CPW combines the scale and capabilities of large organizations with the agility of a smaller, entrepreneurial business. The Data Engineer II will design, develop, and optimize scalable data assets and data platforms that support global analytics, reporting, and business decision-making. The role will work across modern data technologies, including Snowflake, dbt, Azure, and Databricks, while ensuring strong data quality, governance, security, and performance. KEY ACCOUNTABILITIES Data Pipeline Development and Architecture : • Design, develop, and maintain scalable ETL/ELT pipelines using Snowflake, SQL, dbt, Azure, and related technologies. • Build and maintain robust data architectures supporting Bronze, Silver, and Gold data layers. • Develop reliable data-processing workflows using incremental processing, deduplication, testing, and automation. • Create reusable, maintainable, and well-documented data engineering solutions. Data Integration, Modeling, and Harmonization • Integrate data from multiple sources, including Nielsen, Circana, internal systems, and other business datasets. • Develop scalable data models for reporting, analytics, and business intelligence use cases. • Ensure consistency across product, period, market, customer, and other business dimensions and hierarchies. • Review existing data models and processes to identify sustainable, scalable, and automated improvements. • Harmonize data and processes while balancing speed, quality, and business requirements. Data Governance, Quality, and Security 15% • Establish and maintain data-quality rules, validation metrics, monitoring processes, and issue-resolution workflows. • Ensure accurate, complete, consistent, and reliable data flows across the data environment. • Support data governance, stewardship, metadata, documentation, and data-security practices. • Apply appropriate Snowflake security controls, including role-based access, masking policies, and row-access policies. • Proactively identify and resolve data-quality and data-integrity issues. New Data Asset Integration • Develop an understanding of new datasets requested by business stakeholders. • Assess the structure, quality, and usability of new data sources. • Design and implement processes to integrate new data assets into the existing data environment. • Ensure that new data assets are scalable, governed, documented, and fit for analytics use. Stakeholder and Analytics Enablement • Understand data requirements from stakeholders and internal teams. • Deliver data transformations and analytical datasets that help answer business questions faster and more effectively. • Support reporting and visualization teams with backend data architecture and data-model development. • Translate business requirements into practical and sustainable technical solutions. • Communicate effectively with stakeholders, delivery teams, and business partners throughout the project lifecycle. Performance, Scalability, and Cost Optimization • Optimize SQL queries, dbt models, Snowflake workloads, Delta storage formats, and pipeline performance. • Apply performance-tuning techniques across Snowflake, dbt, Databricks, and Azure environments. • Design solutions that support scalability, reliability, maintainability, and cost efficiency. • Monitor data workflows and proactively address performance and operational issues. • Continuous Improvement and Team Contribution • Contribute to continuous-improvement initiatives across data engineering and analytics processes. • Share knowledge, provide peer support, and promote effective engineering practices. • Remain curious and adapt to evolving tools, technologies, and business needs. • Build strong working relationships and contribute positively as a team member. MINIMUM QUALIFICATIONS • Bachelors degree in Computer Science, Information Technology, Electronics and Telecommunications, or a related field. • Minimum 7 years of experience in Data Engineering. • Mandatory experience working with data lakes and multiple data sources. • Strong hands-on experience with - SQL, Snowflake, Snowpipe, Snowflake Streams and Tasks, Dynamic Tables, Stored Procedures, dbt, including models, Jinja templating, macros, tests, and documentation • Strong knowledge of data warehousing, data modeling, and ETL/ELT frameworks. • Experience with large-scale data processing and analytics engineering. • Experience developing data platforms or business intelligence solutions. • Strong understanding of data quality, governance, security, and data-access principles. • Effective communication, stakeholder-management, and problem-solving skills. • Ability to manage ambiguity, make timely decisions, and deliver high-quality work within agreed timelines. PREFERRED SKILLS • Masters degree in Computer Science, Information Technology, Electronics and Telecommunications, or a related field. • Experience in the FMCG, consumer goods, retail, or market research industries. • Experience working with Nielsen, Circana, panel data, retail measurement data, or similar datasets. • Experience with Snowpark, particularly Python, for complex transformation logic beyond standard SQL. • Experience with Azure Data Factory, Azure Storage Accounts, Azure Key Vault, and Azure DevOps. • Experience with CI/CD implementation for Snowflake and dbt deployments. • Experience migrating data pipelines from Databricks to Snowflake, including Delta Live Tables and Unity Catalog. • Familiarity with Snowflake RBAC, masking policies, row-access policies, and other security frameworks. • Experience with Databricks, PySpark, Delta Lake, or Azure Databricks. • Knowledge of Power BI or other business intelligence and visualization platforms. • Relevant certifications in Snowflake, dbt, Azure, or data engineering are desirable. • Continuous