Strategic Projects Lead, Enterprise Evaluations

📋 Description

• Partner with enterprise stakeholders and Scale teams to define evaluation strategies.
• Co-design frameworks, rubrics, and golden datasets for high-signal feedback.
• Define the what, how, and why of human-in-the-loop data with scoring rubrics.
• Own scoping and execution: staffing, costs, and delivery plan.
• Orchestrate end-to-end evaluation engine; manage data labeling pipelines.
• Identify and resolve blockers; anticipate risks before impact.
• Analyze results to provide data-driven production readiness recommendations.
• Run open-source LLM benchmarks and share insights with engineering.
• Act as cross-pollinator for Enterprise BU; convert frameworks into SOPs.

🎯 Requirements

• Strong technical background; CS degree and Python; or SQL/Python analytics.
• 5+ years in high-stakes ops at tech, consulting, or banking.
• Strong problem-solving capabilities; experience with operational challenges or consulting.
• Systemic thinking; build infra to scale a new function.
• Research-adjacent interest in GenAI; capture human judgment to improve evals.
• Full-stack ownership; take projects from 0 to 1 and drive results.

🎁 Benefits

• Comprehensive health, dental, vision; retirement benefits.
• Learning and development stipend; generous PTO.
• Commuter stipend and other benefits.