Senior ETL Engineer - Ab Initio Platform and Data Integration

Job description

Senior Engineer II Amex GBT is a place where colleagues find inspiration in travel as a force for good and -- through their work -- can make an impact on our industry. We're here to help our colleagues achieve success and offer an inclusive and collaborative culture where your voice is valued. Job Title: Senior ETL Engineer -- Ab Initio Platform \& Data Integration Job Summary We are looking for a highly skilled Senior ETL Engineer with strong expertise in Ab Initio, ETL/data integration, and platform administration/support in Linux environments. The ideal candidate should have deep hands-on experience in building, supporting, and maintaining enterprise-grade ETL and data integration solutions, along with strong knowledge of Ab Initio platform infrastructure, PostgreSQL, SQL, and Kafka. This role requires a professional who can contribute across both development and platform support functions --- including ETL design and support, platform administration, code promotion, performance tuning, and integration with multiple source and target systems. The candidate should also have experience in producing and consuming data through Kafka, strong SQL and database integration skills, and exposure to Airflow will be an added advantage. The ideal candidate must also demonstrate strong communication, prioritization, troubleshooting, and process orientation to effectively support business-critical data platforms in a fast-paced enterprise environment. Key Responsibilities 1. ETL Development, Support, and Data Integration Design, develop, enhance, and maintain scalable ETL/ELT workflows using Ab Initio. Build robust and reusable data integration pipelines for batch and near real-time data movement. Work with multiple source and target systems to support enterprise data integration requirements. Develop and optimize ETL jobs for performance, scalability, reliability, and maintainability. Support application teams in ETL job processing, troubleshooting, and issue resolution. Perform root cause analysis for ETL failures, data issues, and platform-related incidents. Ensure data quality, consistency, reconciliation, and integrity across integration processes. 2. Ab Initio Platform Administration \& Maintenance Install, configure, administer, and maintain Ab Initio software components in Linux/Unix environments. Manage and support the Ab Initio platform infrastructure, including routine maintenance and operational stability. Perform platform version upgrades, patch management, and Co\>Op upgrades with minimal downtime. Validate platform stability and performance after upgrades and deployments. Configure and manage dbc/ini files for connectivity with various source systems and target databases. Manage platform configuration, security settings, and user access controls. Automate routine operational procedures and maintenance tasks using shell scripting and system tools. Support metadata and repository administration where applicable. 3. Monitoring, Performance Optimization \& Tuning Monitor ETL job execution and platform performance in real time. Analyze ETL and platform performance metrics to identify tuning and optimization opportunities. Recommend and implement improvements for ETL job efficiency and platform stability. Optimize system resource utilization including CPU, memory, disk I/O, and process distribution. Conduct performance analysis and capacity planning, and recommend infrastructure improvements where necessary. Troubleshoot system bottlenecks affecting ETL throughput, data processing, and integration jobs. 4. Code Promotion, Deployment \& Release Support Manage code promotion and deployment activities across development, QA, UAT, and production environments. Coordinate with development and application teams on deployment schedules and release requirements. Validate successful code deployment and ensure post-deployment platform stability. Maintain deployment records, documentation, and change logs. Support rollback and recovery procedures where required. Ensure adherence to release management, change management, and operational governance processes. 5. Database Integration \& SQL Work extensively with PostgreSQL and other database technologies for ETL and data integration use cases. Write, troubleshoot, and optimize advanced SQL queries for data extraction, transformation, validation, reconciliation, and reporting. Support integration with various databases and data platforms, including relational and analytical systems. Configure and maintain database connectivity and integration parameters. Diagnose and resolve data flow, connectivity, and database performance issues. Apply strong understanding of indexing, joins, partitions, query plans, and SQL tuning techniques. 6. Kafka-Based Data Integration Design and support integrations involving Kafka for data movement and event-driven architectures. Develop or support Kafka producers and consumers for publishing and processing data. Work with Kafka topics, partitions, offsets, and consumer groups. Troubleshoot data flow and integration issues in Kafka-based pipelines. Support streaming and near real-time ingestion patterns using Kafka in enterprise data ecosystems. Required Skills and Experience 5 years of hands-on experience with ETL/data integration tools, preferably Ab Initio. 5 years of strong experience in Ab Initio platform administration, maintenance, and operational support. Strong understanding of Ab Initio architecture, components, configuration, and platform management. Proven experience in Ab Initio version upgrades, patching, and Co\>Op upgrades. Strong hands-on experience in supporting ETL job execution, troubleshooting failures, and resolving platform-related issues. Strong proficiency in Linux/Unix environments and command-line operations. Good hands-on expertise in shell scripting for automation and operational support. Strong experience in PostgreSQL and integration with multiple database technologies. Excellent knowledge of SQL, i