Specialist Architecture Big Data

Job description

5-7 years Job description: Job Summary The Specialist in Big Data Hadoop Ecosystem is responsible for designing developing and maintaining scalable big data solutions leveraging Hadoop and associated technologies Job Description Utilize expertise in Big Data Hadoop Ecosystem to build and optimize data processing pipelines Work with largescale data sets and ensure high availability and reliability of data infrastructure Collaborate with crossfunctional teams to translate business requirements into technical solutions Apply knowledge of Spark Python or Scala to develop efficient data transformation and analysis processes Ensure data quality governance and compliance with organizational standards Stay updated with emerging big data technologies and recommend improvements to existing systems Participate in code reviews testing and deployment activities to maintain software quality Support troubleshooting and performance tuning of big data applications and infrastructure Roles and Responsibilities Design develop and maintain components of the Big Data Hadoop Ecosystem including Hadoop Hive HBase and Spark Develop and optimize highvolume highthroughput data pipelines to support business analytics Collaborate with architects data scientists and business stakeholders to deliver scalable big data solutions Provide technical leadership and mentorship to junior team members to foster skill development Analyze complex data sets to identify trends and insights that support decisionmaking Implement best practices in coding data security and data governance Participate in agile development processes including sprint planning standups and retrospectives Assess and mitigate risks related to data processing and system performance Document technical designs processes and system configurations for knowledge sharing and compliance Skills: Mandatory Skills : Big Data Hadoop Ecosystem, Databricks