Senior MLOps Engineer

📋 Description

• Architect Annotation & Measurement Pipelines: Own the design and implementation of human-in-the-loop and auto-annotation workflows. You will set up systems (e.g., Label Studio) to establish tight confidence metrics and rigorous Inter-Annotator Agreement (IAA).
• Drive Cost-Efficiency via Evaluation: Create ground truth, golden sets, and by-model evaluation criteria to benchmark models and guide data science decisions.
• Enforce Applied MLOps Standards: Implement model-as-a-judge frameworks, deterministic PII scrubbing, and best practices in production pipelines.
• Collaborate on Model Strategy: Work with Data Science to drive generative decisions for stronger product outcomes.
• Build & Integrate: Partner with Engineering to integrate measurement loops into AWS/GCP infrastructure for automated model lifecycles.

🎯 Requirements

• Annotation & Evaluation Expert: Experience setting up annotation workstreams (Label Studio, Scale AI, or custom) and improving ground truth quality.
• Applied MLOps Practitioner: Focus on execution with model monitoring, versioning, and evaluation loops in production.
• Python Proficiency: Strong Python for glue code, scripts, API integrations with our stack.
• Model Optimization Mindset: Balance model performance, latency, and cost; evaluate smaller models.
• Cloud & Infrastructure Fluency: Comfortable in AWS (Sagemaker, S3) and GCP (Vertex AI) with DevOps collaboration.
• Collaborative Leader: Enable DS team to move faster with the right tools and data.

🎁 Benefits

• People: work with talented, collaborative, friendly people.
• Guidance: access learning platform for training from day one.
• Surprise meal stipends: meals while working from home.
• Work/life harmony: vacation, floating holidays, wellness allowance, paid parental leave.
• Whole Health Package: medical, dental, vision, life, disability insurance, and more.
• Work from home stipend: support for home office setup.