Agent Evaluation Engineer

Develop evaluations grounded in product needs, user tasks and how AI models and agents work. Use metrics, experiments and failure analysis to assess capability changes and investigate gaps in existing evaluations, informing system development, post-training and model selection. Responsibilities Design online and offline metrics that translate user tasks, output quality and practical value into measurable, testable evaluation criteria. Design evaluation tasks and experiments around agent plannin…