Senior Solution Engineer

Job description

Required total & relevant experience: 6-8 years Technical skills: • Early Joiners • Cloud Azure | No AWS • Python and pyspark • Open AI • Back end – Azure SQL DB • Azure Data Engineer + AI • Data bricks - ADB • Communication. • React and NodeJS – understanding. Desired candidate Profile Snapshot: Key Responsibilities: • Data Quality Engineering & Automation • Design and develop data quality profiling pipelines using PySpark, Python, and Databricks to ensure data accuracy and reliability. • Build automated validation, cleansing, and anomaly detection frameworks leveraging ML/AI models. • Integrate OpenAI APIs for intelligent rule generation, data summarization, and NLP-based insights. • Full-Stack Development (minimum 4 years in Web & API) • Develop responsive React-based user interfaces for self-service data quality monitoring and visualization. • Build and maintain RESTAPIs using Node.js and Python (Flask/Fast API) to serve ML models and DQ metrics. • Host and manage applications using Azure Web Apps, ensuring secure API connectivity and scalable architecture. • Cloud and Data Integration • Work with Azure SQL Database, Azure Storage, and Databricks for seamless data access and validation. • Integrate data pipelines with APIs, Dataverse, Snowflake, or other cloud data sources. • Support Azure Application Insights and monitoring integration for performance tracking. • ML/AI Enablement • Contribute to ML projects focused on data quality improvement and predictive monitoring. • Apply techniques like anomaly detection, clustering, or deep learning to identify data issues proactively. • Use OpenAI / LLM APIs to automate root cause analysis, data profiling, and rule suggestions. • Collaboration and Documentation • Partner with Data engineers, Architect team, Cloud support team and business stakeholders to translate requirements into technical solutions. • Document DQ frameworks, ML workflows, APIs, and integration processes for reusability and transparency. Preference will be given to candidate with : • 4–8 years of experience in software engineering, data engineering, cloud application development, data quality, or ML/AI solutions. • Strong experience designing and implementing web-based data applications and platforms using React, Python/Node.js, and Microsoft Azure. • Hands-on experience with Databricks, PySpark, Python, and data processing/data-quality frameworks. • Proven record of implementing AI-augmented data quality or observability solutions • Experience in OpenAI API integration or similar LLM-based applications preferred. • Good understanding of cloud security, authentication/authorization, API security, application monitoring, and scalable application architecture. • Visualization - Power BI, React based dashboards • DevOps / Infra - Github, CI/CD pipelines, Azure DevOps Soft Skills: • Analytical mindset with high attention to detail. • Strong communication and collaboration across technical and business teams. • Innovative problem-solver with a passion for automation and data integrity.