Related skills
aws python gcp rag evaluation📋 Description
- Contribute to AI evaluation pipelines, incl. offline evals, production tracing, and feedback systems.
- Implement and maintain performance metrics (quality, success, reliability) using established frameworks.
- Help create and maintain evaluation datasets and test cases to identify regressions.
- Analyze results and propose incremental improvements to model and agent quality.
- Contribute to AI system components (RAG pipelines, retrieval, multi-step workflows).
- Write clean, maintainable Python code that integrates with LLM providers and internal services.
🎯 Requirements
- 2–5 years of professional software engineering experience.
- Experience contributing to production systems as part of a team.
- Proficiency in Python or a similar language.
- Strong understanding of LLM concepts (prompting, RAG, evaluation).
- Familiarity with backend systems, APIs, and cloud environments (AWS, GCP).
- Exposure to logging, monitoring, or debugging tools.
🎁 Benefits
- Medical insurance; Dental, Life/AD&D & Disability Insurance
- Wellness apps; Natural Disaster Support Program
- Paid parental leave and PTO (holidays & sick time)
- Working remotely stipend and one-time WFH setup stipend
- Retirement plan, financial planning, and learning & development budget
- Learning opportunities and inclusive workplace culture
Meet JobCopilot: Your Personal AI Job Hunter
Automatically Apply to Engineering Jobs. Just set your
preferences and Job Copilot will do the rest — finding, filtering, and applying while you focus on what matters.
Help us maintain the quality of jobs posted on Empllo!
Is this position not a remote job?
Let us know!