Prompt Engineer, Agent Prompts & Evals

📋 Description

• Design, test, and optimize system prompts and feature prompts.
• Build and maintain evaluation suites for model quality.
• Collaborate with product, research, and safeguards teams.
• Support model releases; catch regressions before users.
• Help build and improve frameworks and tools for prompt development and testing.
• Mentor product engineers on prompt engineering best practices and evaluations.

🎯 Requirements

• 5+ years of software engineering with Python or similar.
• Demonstrated experience with LLMs and prompt engineering.
• Strong understanding of evaluation methodologies and metrics for AI systems.
• Excellent written and verbal communication skills.
• Ability to manage multiple concurrent projects and prioritize effectively.
• Experience with version control, CI/CD, and modern software development practices.

🎁 Benefits

• Competitive compensation and benefits
• Optional equity donation matching
• Generous vacation and parental leave
• Flexible working hours
• Lovely office space