Gcp Data Engineer (Apache Icebarg, Python, Pyspark, Big Query, GCS)

Job description

Hi Greetings of the day we have opemings for **GCP Data Engineer** **Python(Apache Icebarg, Python, Pyspark, Big Query, GCS** **Experience: 7 to 10yrs** **Location : Pan-india** **jd:** - Strong experience with MongoDB for data lake management. • Proficiency in Scala programming language. - Hands-on experience with GCP Dataflow, Pub/Sub, and other GCP services (Big Query, Cloud Storage, etc.). - Solid understanding of distributed data processing and streaming architectures. - Experience with data modeling, ETL processes, and performance tuning. - Familiarity with CI/CD pipelines and version control (Git). - . candidate will design, develop, and optimize large-scale data processing pipelines and ensure efficient data storage and retrieval for analytics and business intelligence. - Experience with Apache Beam. - Knowledge of Kubernetes or containerized deployments. - Exposure to data governance and security best practices on cloud platforms. - Strong experience with **Apache Iceberg** for data lake management. - Proficiency in **Scala** programming language. - Hands-on experience with **GCP Dataflow**, **Pub/Sub**, and other GCP services (**BigQuery, Cloud Storage, etc**.). - Solid understanding of distributed data processing and streaming architectures. - Experience with data modeling, **ETL processes**, and performance tuning. - Familiarity with **CI/CD pipelines and version control (Git**). - Excellent problem-solving and communication skills. candidate will design, develop, and optimize large-scale data processing pipelines and ensure efficient data storage and retrieval for analytics and business intelligence. - Experience with **Apache Beam**. - Knowledge of **Kubernetes** or containerized deployments. Exposure to **data governance** and **security best practices** on cloud platforms. Design and implement scalable data pipelines using **Apache Iceberg** and **GCP Dataflow**. Develop and maintain data ingestion and transformation workflows leveraging **Pub/Sub** and other GCP services. Write efficient, maintainable code in **Scala** for data processing and transformation. Optimize data lake and warehouse solutions for performance and cost efficiency. Ensure data quality, governance, and compliance across all data processes. Collaborate with cross-functional teams including Data Scientists, Analysts, and Cloud Architects. Troubleshoot and resolve issues related to data processing and storage. Design and implement scalable data pipelines using **Apache Iceberg** and **GCP Dataflow**.