Advance Python Programming, Automation Frameworks

Job description

Advance Python Programming, Automation Frameworks JD • Automate common ServiceNow request workflows related to data platform access, onboarding, approvals, fulfilment, and operational requests. • Build or enhance self-service applications that integrate with ServiceNow and raise requests on behalf of users. • Develop automated backup scripts and scheduled jobs for platform components such as Databricks Unity Catalog/metastore, Dremio metadata, configuration, ADLS objects, and other platform assets. • Build tested restore scripts and runbooks for Dremio metadata, Databricks views, catalog objects, platform configurations, and other recoverable assets. • Support ADLS disaster recovery planning, backup validation, failover readiness, and recovery testing. • Responsible for both development and strong expertise to debug and fix any post production issues • Create reusable scripts, notebooks, templates, and automation patterns for Databricks workloads, views, jobs, and pipelines. • Work with platform users to review and optimize Databricks pipelines, improve performance, reduce cost, and promote reusable engineering patterns. • Develop and maintain Terraform modules for Azure/GCP infrastructure used by the data platform. • Build and maintain CI/CD pipelines using GitHub Actions/GitHub runners for deploying infrastructure, scripts, configurations, and platform automation. • Help reduce repetitive operational work by identifying automation opportunities and implementing robust, reusable solutions. • Own automation tasks end-to-end, from requirements gathering through development, testing, deployment, and support handover. • Leads: Role is 70% of time in Design, Architect, Coding and 30% of time in team mentoring, guidance, coordination • Required Technical Skills: • Strong scripting experience using Python, Bash, PowerShell, or similar languages. • Ability to build production-ready automation scripts. • Experience integrating with ServiceNow APIs, automating ticket creation, request fulfilment, approvals, and workflow-driven processes. • Good working knowledge of Databricks workspaces, jobs, notebooks, clusters, SQL warehouses, Unity Catalog/metastore concepts, views, and pipeline optimization. • Responsible for both development and strong expertise to debug and fix any post production issues for their developed code • Experience or willingness to learn Dremio administration, metadata backup/restore, source configuration, reflections, spaces, folders, and operational support. • Experience with Azure Data Lake Storage, storage accounts, RBAC, managed identities, private networking, and disaster recovery concepts. • Ability to design, automate, test, and document backup and restore processes for data platform metadata, configurations, and platform assets. • Strong Terraform knowledge, including module development, state management, variables, providers, workspaces, and reusable infrastructure patterns. • Strong Azure experience required. • Ability to work with REST APIs, authentication mechanisms, secrets, tokens, and service principals. • Nice to have skills • Azure DevOps, Azure Functions, Logic Apps, Key Vault, Event Grid, Automation Accounts. • Delta Lake, Unity Catalog permissions, job orchestration, DLT, cluster policies, cost/performance optimization. • Dremio REST APIs, reflection management, source automation, backup/restore testing, Kubernetes/AKS deployment knowledge. • AKS/GKE experience, Helm, kubectl, YAML-based deployments.