Python Data Engineer (Chennai/Pune/Indore/Hyderabad/Delhi)

Job description

**Key Responsibilities** - Design, develop, and maintain **scalable data pipelines using Python and PySpark** - Implement **data ingestion, transformation, and validation workflows** - Work with **structured and semi-structured data sources** - Develop and optimize **ETL/ELT pipelines for performance and scalability** - Ensure **data quality, accuracy, and reliability across pipelines** - Collaborate with cross-functional teams including **architecture, product, and analytics teams** - Optimize data processing jobs for **efficiency and cost performance** - Participate in **code reviews and Agile development processes** - Support deployment and monitoring of data pipelines in production **MustHave Skills** **Programming & Data Processing** - Strong expertise in: - **Python** - **PySpark / Spark** - Experience working with: - Large-scale data processing **Data Engineering** - Strong experience in: - **ETL/ELT pipeline development** - **Data ingestion frameworks** - Understanding of: - Data transformation and validation **Database & Querying** - Strong knowledge of: - **SQL (complex joins, aggregations)** - Experience with: - Structured and semi-structured data **Cloud Platforms** - Experience working with: - **Azure / AWS / GCP data platforms** **Data Modeling** - Good understanding of: - **Data warehousing concepts** - **Dimensional modeling** **GoodtoHave Skills** - Experience with: - **Azure Data Factory / Databricks / Synapse** - Exposure to: - **Streaming systems (Kafka, Event Hubs)** - Experience with: - **CI/CD pipelines for data workflows** - Familiarity with: - **Agile methodology (JIRA)** - Domain knowledge: - **BFSI / Insurance**