Aws And Azure Data Engineer

Job description

Role & responsibilities • Create and maintain optimal data pipeline architecture, • Assemble large, complex data sets that meet functional / non-functional business requirements. • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. • Build the infrastructure required for optimal ingestion, transformation, and Publishing of data from a wide variety of data sources using Python/Spark and AWS big data’ technologies. • Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.