Model Evaluation QA Lead

📋 Description

• Model Evaluation Automation: Build automated evaluation pipelines for every candidate model.
• Release Gate Integration: Add quality gates into CI/CD and release pipelines.
• Agent & Model Evaluation Frameworks: Stand up evaluation tools for voice agents and models.
• Active Learning & Data Ingestion Testing: Validate data ingestion and retraining.
• Industry Benchmark Automation: Automate benchmarks like LibriSpeech and CommonVoice.
• Language & Domain Validation: Create multi-language validation tests across domains.

🎯 Requirements

• 4–7 years of QA engineering or ML evaluation experience.
• Hands-on experience building automated ML evaluation pipelines.
• Strong Python skills; Pandas/NumPy; ML pipeline scripting.
• Familiarity with WER, SER, MOS and voice ML concepts.
• CI/CD integration for ML workflows (GitHub Actions, Jenkins, Argo, MLflow).
• Design and maintain reproducible benchmark environments across versions.

🎁 Benefits

• Medical, dental, vision benefits
• Annual wellness stipend
• Unlimited PTO
• Generous paid parental leave
• Flexible schedule
• 401(k) plan with company match