PySpark Data Engineer

Job description

TCS is Hiring PySpark Data Engineer Location: **Face to Face Interview on 8th August at Tata Consultancy Services Limited, Gitanjali Park Campus, IT/ITES SEZ, Plot- IIF / 3, (Block K - Ground Floor) Action Area - II, New Town, Rajarhat, Kolkata - 700156, West Bengal** Role Summary We are seeking a **PySpark Data Engineer** with strong experience in building scalable data pipelines, big data processing, and cloud-based data platforms. The ideal candidate will have expertise in PySpark, Databricks, SQL, and modern data engineering practices to support enterprise analytics and data-driven initiatives. Key Responsibilities - Design, develop, and maintain ETL/ELT pipelines using PySpark and Databricks. - Process and transform large-scale structured and unstructured datasets. - Build scalable batch and real-time data processing solutions. - Optimize Spark jobs for performance, scalability, and cost efficiency. - Implement data quality, governance, and monitoring frameworks. - Integrate data from multiple enterprise and cloud sources. - Collaborate with business, analytics, and architecture teams to deliver data solutions. - Support CI/CD, deployment automation, and production troubleshooting.\\ Mandatory Skills - Strong hands-on experience in **PySpark, Python, and SQL** - Expertise with **Azure Databricks** or similar Spark platforms - Experience in designing and optimizing **data pipelines** - Knowledge of **Spark SQL, DataFrames, UDFs, performance tuning** - Experience with **Azure Data Factory (ADF)** and Azure Data Lake - Strong understanding of ETL/ELT concepts and data modeling