Sr Data Engineer - Cognite Data Fusion

Job description

We are looking forward to hire Cognite Data Fusion Professionals in the following areas : : Experience required: 5+ years Job Title: Deployment Engineer Job Summary We are seeking a highly motivated and technically strong Deployment Engineer to design, deploy, and manage end-to-end industrial data pipelines and digital solutions built on Cognite Data Fusion (CDF). The successful candidate will be responsible for data onboarding, transformation, contextualization, quality validation, and data model management across a wide range of enterprise and industrial data sources. This role requires a strong background in data engineering, industrial data analysis, cloud-based deployments, and software delivery practices. The engineer will work closely with client stakeholders, data owners, domain experts, software development teams, and solution architects to ensure high-quality data foundations that enable successful deployment of client digital solutions such as OptiFlow and OptiSite. Success in this role requires excellent analytical skills, strong communication capabilities, a passion for learning new technologies, and the ability to rapidly adapt to the continuously evolving digital technologies by Client Cognite. Key Responsibilities Data Integration Pipeline Deployment • Design, configure, and deploy end-to-end data ingestion pipelines into Cognite Data Fusion (CDF). • Integrate diverse data sources including: • CSV and flat files • SQL and relational databases • PI Systems • SCADA systems • SAP systems • Historians and operational databases • Unstructured documents and files • Other industrial and enterprise data sources • Ensure reliable, scalable, and secure data ingestion architectures. • Troubleshoot ingestion issues and optimize pipeline performance. Data Transformation Contextualization • Transform raw source data into consumable business and engineering datasets. • Perform data contextualization and establish relationships between assets, events, time series, documents, and engineering objects. • Develop mapping strategies that align source systems with enterprise data models. • Configure and maintain metadata structures and contextualization frameworks within CDF. Data Quality Engineering Data Analysis • Assess source data quality, completeness, accuracy, consistency, and readiness for deployment. • Perform detailed data profiling and validation activities. • Analyze sample datasets to identify data gaps, anomalies, inconsistencies, and missing relationships. • Establish data quality checks and validation procedures throughout the deployment lifecycle. • Work directly with client data owners to resolve data issues and improve data reliability. • Serve as a subject matter expert for engineering and operational data structures. CDF Data Modeling Governance • Design, implement, and maintain data models within Cognite Data Fusion. • Define entities, relationships, metadata structures, and governance approaches. • Support lifecycle management of data models as business requirements evolve. • Ensure data structures align with downstream application and analytics requirements. OptiFlow OptiSite Deployment • Configure and deploy cleint digital solutions including OptiFlow and OptiSite. • Configure backend workflows, business logic, and application integrations. • Support UI configuration and solution customization based on customer requirements. • Validate end-to-end functionality from source systems through application consumption. • Participate in solution testing, commissioning, and production deployment activities. Stakeholder Collaboration • Engage with multiple client teams responsible for different data sources and operational systems. • Gather technical requirements and source system information from stakeholders. • Facilitate workshops, data discovery sessions, and deployment planning activities. • Communicate data findings, risks, and recommendations to technical and non-technical audiences. • Collaborate closely with software developers, architects, project managers, and customer teams. Software Delivery Cloud Operations • Work closely with software development teams that consume CDF data for application development. • Understand software development lifecycle (SDLC) methodologies and deployment practices. • Ensure data structures support current and future application requirements. • Support cloud-based deployment and operational activities. • Contribute to continuous improvement of deployment methodologies and standards. Continuous Learning Innovation • Stay current with new capabilities, updates, and best practices of OptiFlow, OptiSite, Cognite Data Fusion. • Rapidly learn and adopt emerging platform features and deployment approaches. • Evaluate new technologies and tools that improve deployment efficiency and data quality. • Contribute to internal knowledge sharing and capability development initiatives. Required Qualifications • Bachelors degree in computer science, Information Technology, Data Engineering, Engineering, or a related technical discipline. • 5+ years of experience in data engineering, deployment engineering, industrial digitalization, or related fields. • Experience implementing enterprise-scale data integration and data management solutions. • Experience working with cloud-based platforms and deployments. Required Technical Skills Data Engineering • Strong understanding of data integration, ETL/ELT, and data management practices. • Hands-on experience working with structured and unstructured data. • Expertise in data profiling, cleansing, validation, and quality assessment. • Experience with enterprise and industrial data architectures. Databases SQL • Advanced SQL scripting and query optimization skills. • Experience with relational databases and data warehousing concepts. • Ability to investigate and troubleshoot complex data issues. Python Automation