Lead Data Architect

Job description

Experienced technology leader with a minimum of 14+ years of software development experience including 10+ years of data application or data platform architecture experience with deep technology expertise Deep knowledge and hands on experience in design and implementation of Big Data technologies (Apache Spark, Apache Airflow, Apache Flink, Streaming data, ADLS, S3, Object Data store, Trino, Databricks, Unity Catalog, Data governance tools) and familiarity with data Classification Internal architecture patterns (data warehouse, data lake, data lakehouse, data ingestion, curation and consumption) Expertise in metadata management, reference data management. Expertise in streaming and batching data processing and efficient and performing organization of data in devices set of data platforms Experience building and deploying a range of data engineering pipelines into production, including using automation best practices for CI/CD Experience with cloud providers such as Azure, AWS (preferably Azure) and their native services for ingestion, ELT/ ETL and consumption Experience in setting up and managing Aws, Azure based Data Capabilities, data flow automation for real time and batch data pipelines across multiple data sources (on prem & cloud). Solid understanding of data modeling and data transformation techniques General knowledge of Cloud solutions and technical architecture, as well as the impact of implementing them in an enterprise organization Highly organized, self-motivated, pro-active, and able to plan Experience managing users and vendors is a plus