Data Engineering (Python +PySpark)

Job description

**Data Engineering (Python +PySpark)** **Greetings from TCS!** ! **!** **Walk in Drive planned on 4th July 2026 in TCS Kochi.** TCS has been a great pioneer in feeding the fire of young Techies like you. We are a global leader in the technology arena and there’s nothing that can stop us from growing together. Your role is of key importance, as it lays down the foundation for the entire project. Make sure you have a valid EP number before interview. To create an EP Number, please visit https://ibegin.tcs.com/iBegin/register Kindly complete the registration if you have not done it yet. Position: Data Engineering (Python +PySpark) Exp **erience: 4 to 10 years** L **ocation: Kochi** **JOB DESCRIPTIO** N **Role: Data Engineering (Python +PySpark)** **Desired Competencies (Technical/Behavioral Competency)** **Must-Have** - Strong hands-on experience in **Python and PySpark** for data processing and data engineering activities. - Experience in developing data solutions using **PySpark, Spark SQL** , and related frameworks/libraries. - Hands-on experience in building and maintaining **ETL/ELT pipelines, data ingestion pipelines, and data transformation processes** . - Experience in ingesting data from multiple sources such as **databases, files, cloud storage, APIs, S3/data lake platforms** , or similar. - Experience working with **structured and unstructured data** . - Good understanding of **data warehouse concepts, data lake concepts, and data processing patterns** . - Ability to develop **scalable, reusable, and maintainable data processing components** . - Experience in end-to-end data pipeline development, including **source ingestion, transformation, validation, and target load** . - Good knowledge of **SQL** for data analysis, transformation, validation, and basic performance tuning. - Ability to write **clean, efficient, reusable, and scalable Python code** . - Good understanding of **data quality checks, testing, monitoring, documentation, and production support practices** . - Awareness of **security and data protection principles** in data engineering solutions. - **Good-to-Have** - Exposure to cloud platforms such as **AWS, Azure, or GCP** and related data services. - Experience with cloud storage/data platforms such as **S3, ADLS, Blob Storage, Databricks, EMR, Synapse** , or similar. - Knowledge of orchestration/scheduling tools such as **Airflow, Control-M, Azure Data Factory, AWS Glue, Oozie** , or similar. - Exposure to **CI/CD, automation, pipeline deployment, and monitoring** concepts. - Knowledge of **data management principles** , metadata management, data governance, and data lineage. - Experience with Python frameworks such as **Flask, Django, or FastAPI** will be an added advantage. - Knowledge of **ORM concepts** will be preferred where application integration is required. - Exposure to streaming data processing using **Kafka, Spark Streaming, Kinesis, Event Hubs** , or similar. - Ability to create and maintain technical documents, data mapping documents, and support documentation. **Responsibility / Expectations** 1 Develop and maintain **data ingestion pipelines** from multiple sources such as databases, files, cloud storage, APIs, and data lake platforms. 2 Build and enhance **ETL/ELT pipelines** using Python, PySpark, Spark SQL, and related data engineering technologies. 3 Develop reusable **data processing components, scripts, and utilities** for common data engineering use cases. 4 Work with **structured, semi-structured, and unstructured data** to support reporting, analytics, integration, and downstream data consumption. 5 Perform **data validation, reconciliation, data quality checks, error handling, and exception logging** as part of pipeline development. 6 Support basic **performance tuning and optimization** of Spark/PySpark jobs, SQL queries, and data processing workflows. 7 Collaborate with **data engineers, data architects, analysts, QA teams, application teams, and business users** to understand requirements and deliver data solutions. 8 Follow best practices for **coding, testing, automation, monitoring, documentation, and production support** . 9 Support deployment, monitoring, issue resolution, and continuous improvement of data pipelines across environments. 10 Work across different projects/domains as per business demand and demonstrate flexibility for **common pool deployment** . **Also please refer below link for downloading application form. If you don’t remember your password, please click forgot password. Fill application form and then download.** TCS iBegin **Please bring all the mandatory documents below in hard copy for the interview:** **\\*Latest Copy of resume** **\\*TCS Application form** **\\*Original and Photocopy of Government ID proof (PAN or Adhaar)** **\\*Passport Size photographs - 2** **Professio** nal Skills: Strong analytical, problem‑solving, communication, and ownership mindset's Eligibility Criteria: \\*BE/B.tech/MCA/M.Sc./MS with minimum 3 years of relevant IT-experience post Qualification. \\*Only Full-Time courses would be considered. \\*Candidates who have attended TCS interview in the last 1 months need not apply. Referrals are always welcome!!! Kindly don't apply if already attended interview in last 1 months. Thanks & Regards Amrutha