Data Engineer-Data Platforms-Azure

About the Role

A career in IBM Consulting involves building long-term client relationships and close collaboration worldwide, helping leading companies shape their hybrid cloud and AI journeys.

Key Responsibilities

• Advise on, develop, and maintain data engineering solutions on the Azure Cloud ecosystem.
• Design, build, and operate batch and real-time data pipelines using various Azure services.
• Design, build, and operate batch and real-time data pipelines using Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub.
• Design, build, and operate the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB.
• Implement and maintain Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics with Spark and SQL, Azure functions with Python, Azure Purview, and Cosmos DB.
• Apply proficiency in open-source technologies like Apache Airflow and dbt, Spark / Python, or Spark / Scala to support data engineering solutions.
• Operate and maintain data engineering solutions to ensure seamless batch and real-time data processing.

Requirements & Tech Stack

• Proficiency in Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub.
• Experience working with Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics with Spark and SQL, Azure functions with Python, Azure Purview, and Cosmos DB.
• Exposure to open-source technologies like Apache Airflow and dbt, Spark / Python, or Spark / Scala.

Qualifications

• Master's Degree (preferred)
• Experience developing and maintaining data engineering solutions on the Azure Cloud ecosystem.
• Experience designing, building, and operating batch and real-time data pipelines.
• Experience designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB.