Applied Researcher 4

About the Role

Capital One is building trustworthy and reliable AI systems to transform banking. The AI Foundations team is central to this vision, partnering with academia and building production systems to apply state-of-the-art AI to business challenges.

What You’ll Do

• Partner with data scientists, software engineers, machine learning engineers, and product managers to deliver AI-powered products.
• Leverage technologies including Pytorch, AWS Ultraclusters, Huggingface, and Lightning to analyze large volumes of numeric and textual data.
• Build AI foundation models through all development phases: design, training, evaluation, validation, and implementation.
• Conduct applied research to advance AI capabilities and develop next-generation customer experiences.
• Translate complex technical work into tangible business goals.
• Collaborate with senior researchers to prepare internal tech reports and conference submissions.

Requirements & Tech Stack

• Comfort with open-source languages and a passion for development.
• Hands-on experience developing AI foundation models and solutions using open-source tools and cloud computing platforms.
• Deep understanding of AI methodologies.
• Experience building large deep learning models (language, images, events, or graphs).
• Expertise in one or more of: training optimization, self-supervised learning, robustness, explainability, RLHF.
• An engineering mindset with a track record of delivering models at scale (training data and inference volumes).
• Experience delivering libraries, platform-level code, or solution-level code to existing products.
• A track record of generating high-quality ideas or improving existing machine learning ideas, demonstrated by accomplishments like first author publications or projects.
• Ability to own and pursue a research agenda, including selecting impactful problems and autonomously carrying out long-running projects.
• Pytorch, AWS Ultraclusters, Huggingface, Lightning.

Qualifications

• Basic: Currently has, or is in the process of obtaining, a PhD in Electrical Engineering, Computer Engineering, Computer Science, AI, Mathematics, or related fields, with an exception that the required degree will be obtained on or before the scheduled start date. OR an M.S. in Electrical Engineering, Computer Engineering, Computer Science, AI, Mathematics, or related fields plus 2 years of experience in Applied Research.
• Preferred:
• PhD in Computer Science, Machine Learning, Computer Engineering, Applied Mathematics, Electrical Engineering or related fields.
• Experience with LLM.
• PhD focus on NLP or Masters with 5 years of industrial NLP research experience.
• Multiple publications on topics related to the pre-training of large language models (e.g., technical reports of pre-trained LLMs, SSL techniques, model pre-training optimization).
• Member of a team that has trained a large language model from scratch (10B + parameters, 500B+ tokens).
• Publications in deep learning theory.
• Publications at ACL, NAACL and EMNLP, Neurips, ICML or ICLR.
• Behavioral Models: PhD focus on topics in geometric deep learning (Graph Neural Networks, Sequential Models, Multivariate Time Series).
• Multiple papers on topics relevant to training models on graph and sequential data structures at KDD, ICML, NeurIPs, ICLR.
• Worked on scaling graph models to greater than 50m nodes.
• Experience with large scale deep learning based recommender systems.
• Experience with production real-time and streaming environments.
• Contributions to common open source frameworks (pytorch-geometric, DGL).
• Proposed new methods for inference or representation learning on graphs or sequences.
• Worked with datasets with 100m+ users.
• Finetuning: PhD focused on topics related to guiding LLMs with further tasks (Supervised Finetuning, Instruction-Tuning, Dialogue-Finetuning, Parameter Tuning).
• Demonstrated knowledge of principles of transfer learning, model adaptation, and model guidance.
• Experience deploying a fine-tuned large language model.
• Data Preparation: Publications studying tokenization, data quality, dataset curation, or labeling.
• Contribution to a major open source corpus.
• Contribution to open source libraries for data quality, dataset curation, or labeling.
• Demonstrated ability to reproduce and extend peer-reviewed AI research using modern open-source frameworks.
• Experience designing controlled experiments and documenting reproducibility results for internal or public research.