Senior Data Engineer

Job description

WHAT YOULL DO: • Design data pipelines utilizing ETL tools, event driven software, and other streaming software. • Design data architecture that is simple and maintainable while enabling Data Analysts, Data Scientists, and stakeholders to efficiently work with data. • Partner with both data scientists and engineers to bring our amazing concepts to reality. This requires learning to speak the language of statisticians as well as software engineers. • Ensure reliability and quality in data pipelines and enforce data governance, security and protection of our customer information while balancing tech debt. • Partner with product and engineering teams to design data models for downstream consumers. • Evaluate and champion new engineering tools that help us move faster and scale our team WHAT YOU’LL NEED: • You have 5+ years of data engineer experience coding in Python and using SQL. • Experience working with at least one major Datawarehouse in BigQuery, Snowflake, Databricks • Bachelor's degree in technology or data science or a related filed, work experience can be substituted for education • You are comfortable with data engineering tools such as Airflow, dbt, AirByte/ Fivetran, Atlassian suite, GitHub, Terraform as well as Kubernetes. • You understand standard ETL patterns, modern data warehousing ideas such as data mesh or data vaulting, and data quality practices regarding test driven design and data observability. • You have experience partnering with data analysts, data scientists, and engineering stakeholders. • You have a background in data engineering, computer science, software development or another engineering field • You can communicate with a team and articulate ideas to both team members and non-technical stakeholders • You have a drive to learn and master new technologies and techniques • Bachelor's degree or relevant experience can substitute for formal education Technologies You Will Use • Python/ Airflow for data pipeline and automation. • Airbyte for ETL purpose • dbt for data transformation • Snowflake/ BigQuery as Data Warehouse • AWS, GCP, Terraform, Kubernetes, GitHub and more: we keep adopting new tools as we grow!