Data Engineer(Snowflake/ SQL, Python, DWH concepts , data modeling

Job description

Job description :  **Required Technical skill set :** - **Strong proficiency in SQL for Snowflake.** - **Advanced proficiency in Python for** **data** **extraction, automation, and workflow orchestration (including experience with libraries such as pandas, NumPy, and integration with ETL tools).** - **Ability to develop and maintain Python scripts for transforming, cleaning, and migrating** **data** **to Snowflake.** - **Experience using Python for performance tuning, error handling, and logging in** **data** **migration processes.** - **• Hands-on experience with Excel** **data** **handling (e.g., VBA, macros, or scripting to clean and transform Excel** **data).** - **Expertise in ETL/ELT processes and tools (e.g., Talend, Informatica, Matillion, or dbt).** - **Familiarity with cloud platforms (AWS, Azure, or GCP) and their integration with Snowflake.** - **Performance Tuning: Ability to optimize queries and improve Snowflake performance for large** **datasets.** **Must have :** - **Experience in migrating** **data** **from Excel or other structured file sources to Snowflake.** - **Strong analytical skills to assess** **data** **quality and ensure accuracy during migration.** - **Ability to automate recurring workflows and streamline** **data** **ingestion processes using Python.** - **Excellent problem-solving skills with a focus on debugging and resolving migration challenges, especially with Python scripts.** - **Strong communication and collaboration skills to work effectively with cross-functional teams and stakeholders.** - **Proficiency in developing, maintaining, and optimizing Python scripts for** **data** **transformation, cleaning, and migration to Snowflake.** - **Snowflake certifications such as SnowPro Core or Advanced Architect. • Knowledge of** **data** **governance and compliance frameworks (e.g., GDPR, HIPAA).** - **Familiarity with real-time** **data** **pipelines and streaming tools such as Apache Kafka or Spark.** - **Experience with Python-based ETL/ELT frameworks (e.g., Airflow, dbt) for orchestrating** **data** **workflows.** - **Prior experience in large-scale** **data** **migrations and cloud-based** **data** **warehouse implementations.** - **Familiarity with integrating Python scripts into cloud platforms (AWS, Azure, GCP) for automated** **data** **operations.** - **Knowledge of using Python for building monitoring and alerting solutions in** **data** **migration projects.** **Roles and Responsibilities :** **• Design and implement scalable ETL/ELT pipelines to migrate** **data** **from Excel sources to Snowflake.** **• Analyze and understand Excel** **data** **structures, transforming them into Snowflake-compatible formats.** **• Optimize Snowflake performance by implementing best practices such as clustering and partitioning.** **• Collaborate with business and technical teams to define migration requirements and strategies.** **• Automate** **data** **workflows using Python, dbt, or Airflow for seamless migration processes.** **• Develop, maintain, and optimize Python scripts for** **data** **transformation, cleaning, and migration to Snowflake.** **• Use Python for performance tuning, error handling, and logging throughout the** **data** **migration lifecycle.** **• Build monitoring and alerting solutions using Python to ensure reliability and** **data** **quality in migration projects.** **• Ensure** **data** **accuracy, quality, and integrity throughout the migration lifecycle.** **• Implement** **data** **security measures and ensure compliance with organizational and industry standards.** **• Stay updated with Snowflakes latest features and industry trends to propose continuous improvements.**