Azure Data Engineer with PostgreSQL

Job description

Skills- Azure Data Factory, Python, PostgreSQL, Azure Blob Storage, and SQL Key Responsibilities • Develop data ingestion and migration pipelines using Azure Data Factory and Python. • Extract data from SharePoint lists, document libraries, Excel workbooks, XML files, and file repositories. • Build transformation logic to standardize heterogeneous tracker structures and align with the target data model. • Implement data cleansing, validation, reconciliation, and error-handling routines. • Develop bulk-load and delta migration processes for active case migration. • Migrate attachments to Azure Blob Storage and maintain metadata references within PostgreSQL. • Create SQL scripts and load processes for PostgreSQL data migration. • Support testing, UAT, reconciliation, cutover, and post-migration validation activities. • Monitor migration performance and optimize pipeline execution. • Prepare technical documentation and migration deployment artifacts. Required Skills & Experience • 4+ years of experience in Data Engineering, ETL development, or Data Migration projects. • Hands-on experience with: • Azure Data Factory (ADF) • Python • PostgreSQL • Azure Blob Storage • SQL • Experience working with SharePoint APIs, Excel ingestion, XML parsing, and document migration. • Knowledge of data quality validation, reconciliation processes, and ETL best practices. • Experience with Azure DevOps, Git, and CI/CD tools is desirable.