Senior Big Data Architect

Job description

Job Summary The Senior Big Data Architect at Niveus is responsible for designing and developing large-scale data platform architectures, including ingestion, storage, processing, and analytics/consumption layers. The role requires hands-on expertise in Medallion architecture, big data technologies such as Spark and Kafka, and proficiency in Python. Candidates must have experience with data warehouses, data lakes, and cloud platforms (preferably GCP), as well as strong data modelling and ETL/ELT skills. The position involves collaborating with cross-functional teams, ensuring data quality and security, and staying updated on industry trends. Excellent problem-solving, communication, and teamwork abilities are essential. Relevant certifications in big data or cloud technologies are considered an advantage. Location Location: Bangalore, Karnataka, India Responsibilities • Design and develop large-scale, end-to-end data platform architectures covering ingestion, storage, processing, and analytics/consumption layers. • Architect and implement Data Warehouse, Data Lakehouse, and Data Lake solutions following Medallion architecture best practices. • Collaborate cross-functionally to gather requirements and translate them into scalable, secure data solutions. • Evaluate and implement appropriate big data and cloud-native technologies and tools. • Design and manage the consumption layer, including data access management, Role-Based Access Control (RBAC), and data marketplace enablement. • Ensure data quality, integrity, governance, and security across the entire data lifecycle. • Optimise existing data architectures and workflows for performance and cost-efficiency. • Proactively participate in client conversations, technical demos, presentations, and solutioning discussions, representing the technical/architecture point of view. • Stay updated on industry trends and emerging technologies in big data, cloud, and analytics. Key Skills • Demonstrated architectural understanding of data platforms including Data Warehouse, Data Lakehouse, and Data Lake (Medallion architecture). • Extensive experience with big data technologies such as Spark and Kafka. • Proficiency in programming languages such as Python. • Proven ability to design scalable, secure data pipelines, data warehouses, and data lakes capable of processing petabytes of data. • Strong knowledge of data modelling, ETL/ELT processes, Medallion architecture, and data warehousing concepts. • Hands-on experience with at least one cloud platform, with GCP strongly preferred; AWS or Azure also considered. • Experience designing data platforms end-to-end, including ingestion, storage, and analytics/consumption layers. • Working knowledge of the consumption layer, including data access management, RBAC, and data marketplace concepts. • Familiarity with database technologies including NoSQL, SQL, and data lakes.