Data Engineer-Data Platforms-Google

About the Role

As a Data Engineer specializing in Google's data platforms, you will design, build, and maintain data engineering solutions on Google's Cloud ecosystem. You will utilize various Google services to develop batch and real-time data pipelines, manage data migration, and design data layers.

Key Responsibilities

• Design and build data pipelines using Google DataProc, DataFlow, PubSub, BigQuery, and other Google services to meet business requirements.
• Develop and manage data engineering solutions using Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python.
• Schedule and manage the data platform using Google Cloud Scheduler and Cloud Composer (Airflow) to ensure seamless data pipeline operations.
• Develop and manage batch and real-time data pipelines for Data Warehouse and Datalake to ensure high-quality data delivery.
• Collaborate with stakeholders to optimize data solutions and ensure alignment with business objectives.

Requirements & Tech Stack

• Expertise with designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem.
• Experience with services such as Google DataProc, DataFlow, PubSub, BigQuery, Big Table, Cloud Spanner, CloudSQL, and AlloyDB.
• Experience in developing and managing batch and real-time data pipelines for Data Warehouse and Datalake using Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python.
• Experience with open source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala in data engineering solutions.
• Experience in scheduling and managing the data platform using Google Cloud Scheduler and Cloud Composer (Airflow).
• Experience with utilizing various data engineering tools and technologies on Google's Cloud ecosystem.

Qualifications

• Preferred Education: Master's Degree