Data Engineer

About the Role

This is a permanent, hybrid position located in Pune.

Key Responsibilities

• Prepare, transform, and validate structured and semi-structured data.
• Support reliable, documented datasets for reporting, analytics, and AI use cases.
• Develop and maintain ETL/ELT jobs using Python, SQL, and introductory PySpark.
• Ingest data to S3 and support Glue/Athena based processing.
• Apply data-quality checks, reconciliation, and basic observability.
• Create clear technical documentation, data definitions, and runbooks.
• Troubleshoot pipeline failures and complete fixes within team standards.
• Use GitLab CI and peer review for controlled delivery.
• Prepare clean, feature-ready datasets for data science teams.
• Support model training, experimentation, and batch-scoring data flows.
• Apply responsible handling of sensitive data, metadata, and lineage for AI datasets.
• Translate business and analytics needs into reliable, reusable data products.
• Apply Git-based development, CI/CD, testing, documentation, and operational monitoring.
• Partner with analysts, data scientists, platform teams, and business stakeholders.

Requirements & Tech Stack

• Python
• SQL
• Data Modeling fundamentals
• Basic understanding of Spark/PySpark
• Basic understanding of distributed data processing
• AWS S3, Glue, Athena, Lambda, IAM
• Understanding of Data Lake, Data Warehouse, and relational/RDS concepts
• GitLab CI
• Git-based development
• CI/CD
• Testing
• Operational monitoring

Qualifications

• Working knowledge of Python, SQL, and data modeling fundamentals.
• Basic understanding of Spark/PySpark and distributed data processing.
• Awareness of AWS S3, Glue, Athena, Lambda, and IAM.
• Understanding of Data Lake, Data Warehouse, and relational/RDS concepts.
• Exposure to Apache Iceberg, Hive, or Lake Formation is beneficial.

Additional Information

• Software & Tool Skills:
• Infrastructure: AWS CDK, Terraform, CloudFormation.
• AWS Services: SQS, SNS, EventBridge, Step Functions.
• Domain Exposure (Good to Have):
• Automotive, vehicle, fleet, and telemetry data.
• Infrastructure as Code using AWS CDK, Terraform, CloudFormation, or equivalent.
• Event-driven services such as SQS, SNS, EventBridge, and Step Functions.