Data Engineer

About the Role

As an Associate Data Engineer, you will join an engineering team dedicated to building resilient, high-throughput data platforms, modern pipelines, and data mesh architectures using cloud-native tools. In this entry-level role, you will design, develop, and maintain automated batch and streaming data pipelines. You will collaborate directly with software engineers, machine learning scientists, and product managers to ingest raw data, transform complex datasets, and deliver reliable data products to production data lakes and warehouses.

Key Responsibilities

• Data Pipeline Development: Write clean, modular, and maintainable code in Python, Scala, or Java to build ETL/ELT data pipelines.
• Cloud Architecture: Build, configure, and monitor data processing workloads using AWS native services (S3, EMR, Glue, Lambda, and Kinesis).
• Data Modeling & Storage: Support relational, document, and columnar data warehousing solutions (PostgreSQL, Snowflake, and Redshift) with efficient schemas, partitioning, and indexing.
• Data Quality & Governance: Implement automated data validation checks, monitoring, schema drift detection, and data lineage tracking.
• DevOps & Automation: Adopt CI/CD practices (GitHub Actions, Jenkins, Terraform) to deploy data infrastructure as code and schedule jobs via orchestrators like Apache Airflow.
• Agile Collaboration: Participate in sprint ceremonies, peer code reviews, backlog grooming, and technical problem-solving sessions.

Requirements & Tech Stack

• Programming: Academic, project, or internship experience coding in Python, SQL, Scala, or Java.
• Database Fundamentals: Strong foundation in relational database management systems (RDBMS) and writing optimized SQL queries (joins, aggregations, window functions).
• Cloud Architecture: AWS native services (S3, EMR, Glue, Lambda, and Kinesis).
• Data Modeling & Storage: PostgreSQL, Snowflake, and Redshift.
• DevOps & Automation: CI/CD practices (GitHub Actions, Jenkins, Terraform), Apache Airflow.

Qualifications

• Education: Bachelor’s or Master’s degree in Computer Science, Data Engineering, Software Engineering, Information Systems, Applied Mathematics, or a related quantitative STEM discipline (or equivalent practical technical training/bootcamp completion).
• Work Authorization: Must possess valid, unrestricted U.S. work authorization. Capital One will sponsor a new applicant for employment authorization for this entry-level position.

Preferred Skills

• Prior internship, capstone project, or open-source contribution in data engineering or cloud software development.
• Familiarity with distributed computing engines such as Apache Spark or distributed messaging queues such as Apache Kafka.
• Hands-on exposure to cloud environments, especially Amazon Web Services (AWS).
• Knowledge of workflow orchestration tools like Apache Airflow.
• Understanding of basic Git/GitHub version control workflows and automated testing practices.