Agent Evaluation & Evolution Machine Learning Engineer Graduate
ByteDance
About the Role
The Applied Machine Learning Ark team develops and operates Large Language Model (LLM) service platforms offering businesses Model-as-a-Service (MaaS) solutions. The US team focuses on the design, development, and operation of MaaS solutions for US and international markets outside mainland China. The role involves building full-stack, end-to-end solutions for text and multimodal LLM algorithms, LLM training/fine-tuning/inference frameworks, prompt engineering, model alignment, and intelligent agent systems. Beyond model serving, the team operates large-scale log analytics pipelines to extract insights from invocation logs for model improvement and system optimization.
Key Responsibilities
- Design evaluation systems for LLM-based agents, focusing on task success, tool use, reasoning quality, and reliability.
- Build benchmarks and automated judging pipelines incorporating rule-based checks, model-based judging, and human review.
- Analyze agent execution traces and user feedback to identify failure patterns and drive concrete system improvements.
- Support the closed loop from experience to capability, collaborating with research, platform, and product teams to implement methods in production.
Requirements & Tech Stack
- Solid foundation in machine learning and deep learning.
- Hands-on experience with LLM-based systems (e.g., agents, tool calling, retrieval, multi-agent systems) through research, internships, or projects.
- Strong Python skills and experience with a mainstream ML or agent evaluation framework.
- Demonstrated research or engineering ability through publications, substantial projects, internships, or open-source work.
Qualifications
- Individuals completing or recently completed a Bachelor's/Master's degree in Computer Science, Artificial Intelligence, Machine Learning, Data Science, or a related field.
- Publications at top-tier ML/NLP venues (e.g., NeurIPS, ICML, ICLR, ACL etc.), especially in agent learning, self-improving/self-evolving/RSI, or agent evaluation.
- Experience with evaluation methodology: metric design, model-based judging, or annotation and statistical analysis.
- Familiarity with LLM post-training, reasoning and planning methods, or continual learning.
- Experience with feedback-driven optimization loops, or with large-scale log and trace analysis.
About ByteDance
Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. The company offers a suite of products including TikTok, Lemon8, CapCut, Pico, Toutiao, Douyin, and Xigua.
Why Join ByteDance
ByteDance's mission to inspire creativity is at its core. The company fosters a global, diverse team to build innovative products that help people express themselves, discover, and connect. ByteDancers are encouraged to lead with curiosity, humility, and a desire to make an impact, embracing an "Always Day 1" mindset.
Note for Candidates
- Successful candidates must commit to an onboarding date by the end of the year.
- Please state your availability and graduation date clearly in your resume.
- Candidates can apply to a maximum of two positions globally and will be considered in the order of application.
- Applications are reviewed on a rolling basis; early application is encouraged.