Azure Data engineer + AI - 12th Aug (Wednesday) - Bengaluru- Virtual

Job description

**Key Responsibilities** - **Data Pipeline & Architecture:** Design and build robust ETL/ELT data pipelines using **Azure Data Factory**, **Azure Databricks**, and **Microsoft Fabric** to process batch and real-time streaming data. [1, 2] - **AI & LLM Data Integration:** Prepare, clean, and structure enterprise data lakes/lakehouses to feed **Azure OpenAI**, vector databases, and Retrieval-Augmented Generation (RAG) architectures. [1] - **Data Modeling & Governance:** Implement medallion architectures (Bronze, Silver, Gold), ensure data quality, and manage data governance/lineage using tools like **Microsoft Purview** or **Unity Catalog**. [1, 2] - **Collaboration:** Partner with data scientists, AI engineers, and business analysts to operationalize machine learning models and AI agents. [1] **Required Skills & Qualifications** - **Cloud Platforms:** Hands-on experience with **Microsoft Azure** (ADF, Azure Databricks, Azure Synapse Analytics, ADLS Gen2, and Microsoft Fabric). - **Programming & Querying:** Advanced proficiency in **Python**, **PySpark**, **Spark SQL**, and complex **SQL**. - **AI & GenAI Familiarity:** Knowledge of vector embeddings, semantic search, prompt grounding, and integrating **Azure AI / OpenAI** endpoints.