Senior Software Engineer -Data Engineer

Job description

Founded in 1976, CGI is among the largest independent IT and business consulting services firms in the world. With 94,000 consultants and professionals across the globe, CGI delivers an end to end portfolio of capabilities, from strategic IT and business consulting to systems integration, managed IT and business process services and intellectual property solutions. CGI works with clients through a local relationship model complemented by a global delivery network that helps clients digitally transform their organizations and accelerate results. CGI Fiscal 2024 reported revenue is CA$14.68 billion and CGI shares are listed on the TSX (GIB.A) and the NYSE (GIB). Learn more at cgi.com. Job Title: Data Engineer Position: Senior Software Engineer Experience: 4-7 Years Category: Software Development/ Engineering Shift: General Shift Main location: India, Karnataka, Bangalore Position ID: J0626-2182 Employment Type: Full Time Education Qualification: Bachelor's degree in Computer Science or related field or higher with minimum 4 years of relevant experience. #LI-SK38 **Your future duties and responsibilities** We are looking for an experienced Data Engineer to join our team. The ideal candidate should be passionate about building scalable data architectures, developing efficient ETL/ELT pipelines, and creating robust, high performance data systems. You will work closely with our Data Science, Analytics, QA, and DevOps teams to deliver seamless data integration and robust solutions that support our business intelligence and strategic objectives. Responsibilities: . Design, develop, and maintain scalable data pipelines and architectures using Python, SQL, and big data technologies. . Build, manage, and optimize modular, testable, and efficient data ingestion and transformation scripts. . Integrate data from various sources including RESTful APIs, third party services, and internal databases. . Collaborate with Data Scientists, Analysts, and Backend developers to deliver end to end data solutions. . Write clean, efficient, and maintainable code following data engineering best practices. . Optimize pipeline performance, processing speed, and scalability for large scale datasets. . Implement complex data flows and state management using orchestration tools and reactive data processing frameworks. . Troubleshoot, debug, and resolve data quality, pipeline failures, and performance bottlenecks. . Participate in code reviews and maintain high coding standards for data infrastructure. . Ensure data security, integrity, and protection practices are followed (e.g., encryption, access control, GDPR/SOC2 compliance). . Contribute to data architecture, modeling, and technical design discussions. . Support deployment processes and work with data centric CI/CD pipelines (DataOps). . Stay updated with emerging data technologies, cloud updates, and suggest improvements to the data stack. **Required qualifications to be successful in this role** Must Have Skills: . Strong proficiency in Python or Scala and advanced SQL (complex joins, window functions, optimization). . Hands on experience with core data technologies including Spark, Hadoop, or similar distributed processing frameworks. . Solid understanding of data modeling (Star/Snowflake schema), data warehousing, and lakehouse architectures. . Experience with workflow orchestration tools like Apache Airflow, Prefect, or Dagster. . Strong experience in building and consuming APIs for data ingestion and asynchronous data handling. . Proficiency in Git and version control workflows. . Understanding of software design principles (modular design, reusability, maintainability applied to data). . Experience with query optimization, performance tuning, and database indexing. . Familiarity with data security practices (IAM roles, encryption at rest/transit, data masking). . Solid understanding of batch and real time data processing patterns. . Good to Have Skills: . Experience with cloud data warehouses like Snowflake, BigQuery, or Redshift. . Exposure to NoSQL databases and search engines (e.g., MongoDB, Cassandra, Elasticsearch). . Familiarity with data validation frameworks (e.g., Great Expectations, dbt). . Experience with streaming technologies like Apache Kafka, Flink, or Spark Streaming. . Working knowledge of Infrastructure as Code (Terraform) and Cloud services (AWS, Azure, or GCP). . Understanding of Machine Learning pipeline integration (MLOps) and feature stores. . Familiarity with containerization (Docker, Kubernetes) for data workloads. . Basic understanding of CI/CD pipelines for automated data testing and deployment. . Working knowledge of BI and visualization tools (e.g., Tableau, PowerBI, Looker). . Experience working in Agile/Scrum environments. CGI is an equal opportunity employer. In addition, CGI is committed to providing accommodation for people with disabilities in accordance with provincial legislation. Please let us know if you require reasonable accommodation due to a disability during any aspect of the recruitment process and we will work with you to address your needs **Together, as owners, let’s turn meaningful insights into action.** Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you’ll reach your full potential because… You are invited to be an owner from day 1 as we work together to bring our Dream to life. That’s why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company’s strategy and direction. Your work creates value. You’ll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise. You’ll shape your career by joining a company built to grow and la