Machine Learning Evaluation Specialist
Alignerr
Bengaluru, Karnataka, India1h agovia LinkedIn
About the Role
Organization: Alignerr
Type: Hourly Contract
Location: Remote
Commitment: 10–40 hours/week
What if your years of research expertise could directly shape the trajectory of AI development? Alignerr is looking for elite machine learning specialists to design problems that push state-of-the-art AI systems to their limits and help the field understand where they break down. This is a high-impact opportunity to apply domain knowledge at the frontier of AI evaluation and safety, influencing how the next generation of AI models is built, tested, and improved.
Key Responsibilities
- Design complex, original machine learning problems rooted in specialized domain expertise.
- Craft evaluation tasks that demand advanced knowledge beyond standard ML pipelines.
- Create challenges that would stump even the most capable LLMs by drawing from personal research.
- Define precise problem statements, gold-standard solutions, and robust evaluation criteria.
- Assess AI-generated ML solutions for correctness, creativity, and methodological rigor.
- Document expected failure modes, difficulty levels, and the domain knowledge required for each problem.
- Collaborate asynchronously with a global network of top researchers and engineers.
Requirements & Tech Stack
- Strong working knowledge of ML fundamentals: model selection, feature engineering, evaluation metrics, and pipeline design.
- Deeply embedded in active research problems in your field and understand their challenges.
- Ability to identify where general ML knowledge falls short and specialized domain expertise is essential.
- Communicate complex ideas with clarity and precision in writing.
- Self-directed, intellectually curious, and thrive on challenging work.
- Experience publishing or conducting original research (highly valued).
Qualifications
- Graduate degree (MS or PhD) in a scientific or technical field that intersects with machine learning.
Example Domains (Not Exhaustive)
- Computational biology, genomics, or bioinformatics
- Climate science and environmental modeling
- Medical imaging and healthcare ML
- Materials science and computational chemistry
- Astrophysics and signal processing
- NLP for low-resource or specialized corpora
- Robotics, control theory, or reinforcement learning
- Financial modeling and quantitative analysis