Databricks DB Migration Engineer

Job description

**Role: Databricks DB Migration Engineer** **Experience: 6 to 8 years** **Location: PAN India** **Key Responsibilities** - Assess existing source database environments, schemas, data volumes, dependencies, jobs, stored procedures, ETL pipelines, and reporting workloads for migration readiness. - Design migration approach for moving data and workloads from RDBMS, data warehouse, Hadoop, or cloud databases to Databricks Lakehouse using batch and incremental migration patterns. - Develop scalable ingestion and transformation pipelines using PySpark, Spark SQL, Databricks Workflows, Delta Live Tables, Auto Loader, and notebooks/jobs as applicable. - Convert and optimize SQL scripts, stored procedures, ETL logic, and business transformations into Spark SQL/PySpark- based implementations. - Implement Delta Lake best practices including partitioning, OPTIMIZE, Z-ORDER, schema evolution, time travel, ACID transactions, and data retention strategies. - Perform data profiling, cleansing, mapping, validation, reconciliation, and row/hash/count-level comparison between source and target systems. - Tune Spark jobs and Databricks clusters for performance, cost, concurrency, workload isolation, and operational efficiency. - Support migration dry runs, defect triage, production cutover, rollback planning, post-migration validation, hypercare, and operational handover. - Collaborate with architects, DBAs, data engineers, cloud teams, security teams, QA teams, and business stakeholders to ensure successful migration delivery. - Prepare technical documentation including migration design, mapping documents, runbooks, reconciliation reports, deployment plans, and operational support guides.