Hadoop, Spark, Scala Data Engineer

Job description

**Desired Experience Range: 5-8 years** **Location: Bangalore, Chennai, Hyderabad** **Roles Responsibilities-** - Proficiency with Big Data, Spark, HDFS, Kafka, Scala, AWS, Azure, GCP, ETL, Hadoop, Hive etc. - Strong experience in programming languages such as Python, Java, or Scala for data manipulation and engineering tasks. - Expertise in SQL and NoSQL databases - Hands-on experience with big data technologies like Hadoop, Spark, Kafka, and Hive to handle large-scale data processing and real-time data streams. - In-depth knowledge of data warehousing solutions such as Amazon Redshift, Google BigQuery, and Snowflake for building and managing data warehouses. - Proficiency in designing, developing, and maintaining ETL (Extract, Transform, Load) processes using tools like Apache NiFi, Talend, or Informatica. - Familiarity with cloud platforms like AWS, Azure, or Google Cloud for deploying, managing, and scaling data infrastructure and services. - Design, develop, and maintain scalable data pipelines for extracting, transforming, and loading data from various sources to ensure seamless data flow and accessibility. - Collaborate with cross-functional teams to integrate data from multiple disparate sources, ensuring consistency, accuracy, and reliability of data. - Optimize data processing workflows and storage solutions for performance, scalability, and cost-efficiency - Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of data throughout the data lifecycle. - Monitor the performance of data pipelines and infrastructure, identifying and resolving issues to maintain system stability and reliability.