Backend Inference Framework Engineer Graduate

About the Role

Data AML is ByteDance's Machine Learning mid-platform, providing training and inference systems for recommendation/advertising for businesses such as Douyin, Jinri Toutiao, and Xigua Video. It provides powerful Machine Learning computing power for internal business units within the company and conducts research on some general and innovative algorithms for issues in these businesses. As a graduate, you will get opportunities to pursue bold ideas, tackle complex challenges, and unlock limitless growth. Successful candidates must be able to commit to an onboarding date by the end of the year.

Key Responsibilities

• Responsible for the overall architecture design and implementation of model inference services, building a high-performance, highly available, and scalable enterprise-level inference system for large-parameter, high-complexity AI models.
• Overcome various architectural challenges in the implementation of complex model inference and support the efficient launch of models across all business scenarios.
• Responsible for the R&D and optimization of the core modules of the inference framework, covering core capabilities such as inference engine scheduling, monitoring and alerting, canary release.
• Continuously iterate on the framework performance and resolve performance bottlenecks, resource bottlenecks, and stability issues in high-concurrency and large-model inference scenarios.
• Keep track of the latest inference technologies in the industry.
• Conduct technology selection and innovation in combination with business scenarios.
• Accumulate distributed high-concurrency service architecture solutions.
• Promote the upgrade and standardization of the team's technical system.

Requirements & Tech Stack

• Familiar with basic Linux commands.
• Solid C/C++ programming skills and knowledge of data structures and algorithms.
• Familiar with the basic principles of multi-threaded concurrency.
• Proficient in basic usages such as thread usage, synchronization locks, and thread pools.
• Ability to identify common concurrency issues and perform basic performance tuning in multi-threaded scenarios.
• Experience in R&D projects of high-concurrency distributed services.
• Familiar with service latency and resource optimization.
• Good learning and execution abilities.
• Willing to proactively understand the model inference service architecture.
• Possess problem analysis and abstraction capabilities.
• Good cross-team collaboration skills, communication and presentation skills, and document writing skills.
• Strong sense of responsibility and stress tolerance.
• Ability to drive the resolution of complex technical issues and the implementation of projects.

Preferred Qualifications

• Practical project experience and understanding of source code in high-concurrency services/frameworks such as Redis, RocksDB, BRPC, GRPC.
• Understanding of the operating mechanism of GPUs, with relevant project experience and optimization capabilities in GPU service resource management and control.

Qualifications

• Individuals who are completing or have recently completed a Bachelor's/Master's degree in Software Development, Computer Science, Computer Engineering, or a related technical discipline.