Data Engineer-Data Platforms-Azure

About the Role

A career in IBM Consulting involves building long-term client relationships and close collaboration worldwide. You will work with leading companies across industries, helping them shape their hybrid cloud and AI journeys.

Your Role And Responsibilities

As a Data Engineer specializing in Data Platforms on Azure, you will advise on, develop, and maintain data engineering solutions on the Azure Cloud ecosystem. You will design, build, and operate batch and real-time data pipelines using various Azure services.

Key Responsibilities

• Design and Build Data Pipelines: Design, build, and operate batch and real-time data pipelines using Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub.
• Develop Data Layer: Design, build, and operate the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB.
• Apply Azure Expertise: Apply proficiency in Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics with Spark and SQL, Azure functions with Python, Azure Purview, and Cosmos DB.
• Implement Open Source Technologies: Implement open source technologies like Apache Airflow and dbt, Spark / Python, or Spark / Scala to support data engineering solutions.
• Operate Azure Services: Operate Azure services such as Azure Event Hub and Streaming Analytics, Managed Streaming for Apache Kafka, and Azure DataBricks with Spark.

Qualifications

• Preferred Education: Master's Degree

Requirements & Tech Stack

• Exposure to Azure Toolset: Experience working with Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub to design, build, and operate batch and real-time data pipelines.
• Proficiency in Azure Data Platform: Exposure to Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics with Spark and SQL, Azure functions with Python, Azure Purview, and Cosmos DB.
• Data Pipeline Development: Experience working with batch and real-time data pipelines using Azure services, including designing, building, and operating data pipelines.
• Open Source Technologies: Exposure to open source technologies like Apache Airflow and dbt, Spark / Python, or Spark / Scala to support data engineering solutions.
• Azure Services Operation: Experience working with Azure services such as Azure Event Hub and Streaming Analytics, Managed Streaming for Apache Kafka, and Azure DataBricks with Spark.
• Proficiency in Open Source Technologies: Exposure to open source technologies like Apache Airflow and dbt, Spark / Python, or Spark / Scala to support data engineering solutions.
• Knowledge of Azure Services: Experience working with Azure services such as Azure Event Hub and Streaming Analytics, Managed Streaming for Apache Kafka, and Azure DataBricks with Spark.
• Familiarity with Data Pipeline Development: Exposure to designing, building, and operating batch and real-time data pipelines using Azure services.