GCP Data Engineer

About the Role

As a GCP Data Engineer at Infosys, you will work on cutting-edge data engineering and analytics initiatives for global clients across industries. This role offers opportunities to work with cloud-native data platforms, modern analytics ecosystems, AI-driven solutions, and large-scale enterprise data modernization programs.

Key Responsibilities

• Design and develop scalable GCP Data Engineering solutions and cloud-native data platforms.
• Build batch and real-time pipelines using BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, Python, PySpark, and Databricks.
• Develop and optimize Databricks, Delta Lake, Spark, and Structured Streaming workloads for large-scale processing.
• Implement high-throughput streaming solutions using Pub/Sub and Dataflow, including windowing, watermarking, late-arriving data, duplicate handling, and dead-letter queues.
• Handle high-volume TB/PB-scale data with focus on performance, scalability, reliability, and cost optimization.
• Develop complex SQL transformations, data models, and analytics-ready BigQuery datasets using partitioning and clustering.
• Orchestrate pipelines using Cloud Composer / Airflow and implement CI/CD and Terraform-based Infrastructure as Code.
• Ensure data quality, governance, security, monitoring, alerting, and production support.
• Collaborate with architects, stakeholders, and cross-functional teams in an Agile delivery model.
• Troubleshoot and optimize end-to-end GCP, Databricks, and streaming data pipelines.

Requirements & Tech Stack

• GCP
• BigQuery
• Cloud Storage
• Dataflow
• Pub/Sub
• Dataproc
• Databricks
• Delta Lake
• Spark
• Structured Streaming
• Python
• PySpark
• SQL
• Cloud Composer / Airflow
• CI/CD
• Terraform
• Batch and real-time streaming data processing
• Data modeling
• Infrastructure as Code