Data Engineer(Snowflake/ SQL, Python, DWH concepts , Data Modelling

Job description

**Job Description :** **Required Technical skill set :** **Strong proficiency in SQL for Snowflake.** **• Advanced proficiency in Python for data extraction, automation, and workflow orchestration (including experience with libraries such as pandas, NumPy, and integration with ETL tools).** - **Ability to develop and maintain Python scripts for transforming, cleaning, and migrating data to Snowflake.** - **Experience using Python for performance tuning, error handling, and logging in data migration processes.** **• Hands-on experience with Excel data handling (e.g., VBA, macros, or scripting to clean and transform Excel data).** - **Expertise in ETL/ELT processes and tools (e.g., Talend, Informatica, Matillion, or dbt).** - **Familiarity with cloud platforms (AWS, Azure, or GCP) and their integration with Snowflake.** - **Performance Tuning: Ability to optimize queries and improve Snowflake performance for large datasets.** **Must have :** **• Experience in migrating data from Excel or other structured file sources to Snowflake.** - **Strong analytical skills to assess data quality and ensure accuracy during migration.** - **Ability to automate recurring workflows and streamline data ingestion processes using Python.** - **Excellent problem-solving skills with a focus on debugging and resolving migration challenges, especially with Python scripts.** **• Strong communication and collaboration skills to work effectively with cross-functional teams and stakeholders.** **• Proficiency in developing, maintaining, and optimizing Python scripts for data transformation, cleaning, and migration to Snowflake.** **• Snowflake certifications such as SnowPro Core or Advanced Architect. • Knowledge of data governance and compliance frameworks (e.g., GDPR, HIPAA).** - **Familiarity with real-time data pipelines and streaming tools such as Apache Kafka or Spark.** **• Experience with Python-based ETL/ELT frameworks (e.g., Airflow, dbt) for orchestrating data workflows.** - **Prior experience in large-scale data migrations and cloud-based data warehouse implementations.** **• Familiarity with integrating Python scripts into cloud platforms (AWS, Azure, GCP) for automated data operations.** - **Knowledge of using Python for building monitoring and alerting solutions in data migration projects.** **Roles and Responsibilities :** **• Design and implement scalable ETL/ELT pipelines to migrate data from Excel sources to Snowflake.** **• Analyze and understand Excel data structures, transforming them into Snowflake-compatible formats.** **• Optimize Snowflake performance by implementing best practices such as clustering and partitioning.** **• Collaborate with business and technical teams to define migration requirements and strategies.** **• Automate data workflows using Python, dbt, or Airflow for seamless migration processes.** **• Develop, maintain, and optimize Python scripts for data transformation, cleaning, and migration to Snowflake.** **• Use Python for performance tuning, error handling, and logging throughout the data migration lifecycle.** **• Build monitoring and alerting solutions using Python to ensure reliability and data quality in migration projects.** **• Ensure data accuracy, quality, and integrity throughout the migration lifecycle.** **• Implement data security measures and ensure compliance with organizational and industry standards.** **• Stay updated with Snowflakes latest features and industry trends to propose continuous improvements.**