Related skills
aws typescript rag langgraph llm📋 Description
- Design and ship end-to-end AI evaluation framework (offline and production).
- Define metrics: task completion, hallucinations, response quality, engagement.
- Build evaluation datasets, test harnesses, and automated scoring pipelines.
- Architect reusable agent infra: multi-turn flows, LLM DAGs, LangGraph.
- Scale RAG pipelines and retrieval infrastructure (vector stores).
- Lead build-vs-buy decisions across LLM providers and tooling.
🎯 Requirements
- 5+ years of production AI/ML software engineering
- Deep hands-on with LLM systems: prompts, RAG, agent orchestration, evals, tuning
- Data-focused experiments: statistics and analysis
- Build and operate production AI systems: multi-step workflows and multi-agent topologies
- LangGraph/LangSmith and AI evaluation tooling experience
- Vector databases (Pinecone) and retrieval design; AWS; TypeScript familiarity
🎁 Benefits
- Medical, dental, life, AD&D and disability insurance
- Wellness apps and mental health resources
- Paid parental leave and PTO holidays
- Remote work stipend and WFH setup stipend
- Retirement plan and financial planning
- Learning and development budget
Meet JobCopilot: Your Personal AI Job Hunter
Automatically Apply to Engineering Jobs. Just set your
preferences and Job Copilot will do the rest — finding, filtering, and applying while you focus on what matters.
Help us maintain the quality of jobs posted on Empllo!
Is this position not a remote job?
Let us know!