Lead Software Engineer - Data

Job description

*EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.* We are looking for a **Lead Software Engineer – Data** to design, build, and optimize large-scale data solutions while guiding a team through the full lifecycle of Big Data implementation. This role requires deep technical expertise in distributed computing and modern data processing frameworks, along with strong leadership capabilities to drive projects in an Agile environment. **Responsibilities** - Lead the design and implementation of Big Data solutions across the organization - Develop and optimize distributed data processing pipelines using Apache Spark - Write efficient, scalable code in Python to support data engineering initiatives - Build stream-processing systems leveraging technologies such as Apache Storm or Spark-Streaming - Integrate data from multiple sources, including RDBMS, ERP systems, and file-based inputs - Apply ETL techniques and frameworks to ensure reliable data transformation and delivery - Tune and optimize Spark job performance to meet business and technical requirements - Guide the team in adopting native cloud data services on Azure or AWS with Databricks - Manage and mentor a team of engineers to deliver high-quality data solutions efficiently - Drive Agile practices throughout the software development lifecycle **Requirements** - 8-14 years of experience in Big Data and related data technologies - Expertise in distributed computing principles and Apache Spark - Proficiency in Hadoop v2, MapReduce, HDFS, and Sqoop - Experience with messaging systems such as Kafka or RabbitMQ - Familiarity with Big Data querying tools such as Hive and Impala - Understanding of SQL queries, joins, stored procedures, and relational schemas - Experience with NoSQL databases such as HBase, Cassandra, or MongoDB - Knowledge of ETL techniques and frameworks - Experience with native cloud data services on Azure or AWS with Databricks - Capability to lead a team efficiently - Background in designing and implementing Big Data solutions - Practitioner of AGILE methodology **We offer** - Opportunity to work on technical challenges that may impact across geographies - Vast opportunities for self-development: online university, knowledge sharing opportunities globally, learning opportunities through external certifications - Opportunity to share your ideas on international platforms - Sponsored Tech Talks & Hackathons - Unlimited access to LinkedIn learning solutions - Possibility to relocate to any EPAM office for short and long-term projects - Focused individual development - Benefit package: - Health benefits - Retirement benefits - Paid time off - Flexible benefits - Forums to explore beyond work passion (CSR, photography, painting, sports, etc.)