Python-Spark Scala

Job description

## **Key Responsibilities:** - 5 9 years of experience in Data Engineering Big Data development - Strong hands on experience with Python and Apache Spark Scala PySpark - Deep understanding of Spark architecture RDD DataFrames Spark SQL - Experience in distributed computing and big data processing - Strong knowledge of SQL and data modeling concepts - Experience with data pipeline development ETL ELT - Familiarity with Linux Unix environments - Experience with version control tools Git ## **Technical Requirements:** - Primary skills Technology Big Data Data Processing Spark Technology Functional Programming Scala Technology Machine Learning Python ## **Additional Responsibilities:** - Experience with cloud platforms AWS Azure or GCP - Hands on with Databricks EMR Spark clusters - Knowledge of streaming technologies Kafka Spark Streaming Structured Streaming - Experience with workflow orchestration tools Airflow Oozie - Familiarity with Delta Lake Lakehouse architecture - Exposure to NoSQL databases MongoDB Cassandra - Knowledge of CI CD and DevOps practices ## **Preferred Skills:** Technology->AI-Data science->PYTHON,Technology->Big Data - Data Processing->Spark->SparkSQL,Technology->Functional Programming->Scala