Research Engineer / Research Scientist, Pre-training

📋 Description

• Research and implement model architecture, algorithms, data processing, and optimizer work
• Lead small research projects and collaborate on larger initiatives
• Design, run, and analyze experiments on large language models
• Optimize and scale training infrastructure for efficiency
• Develop tooling to boost team productivity
• Contribute across stack from low-level optimizations to model design

🎯 Requirements

• Degree (BA required, MS/PhD preferred) in CS, ML, or related field
• Strong software engineering with track record of building complex systems
• Expertise in Python and deep learning frameworks
• Have worked on high-performance, large-scale ML systems, particularly in language modeling
• Familiarity with ML Accelerators, Kubernetes, and large-scale data processing
• Strong problem-solving skills and a results-oriented mindset
• Excellent communication skills and ability to work in a collaborative environment

You'll thrive in this role if you

• Have significant software engineering experience
• Balance research goals with practical engineering constraints
• Take on tasks outside your job description to support the team
• Enjoy pair programming and collaborative work
• Are eager to learn more about machine learning research
• Are excited to work in a cohesive, large-scale AI research team
• Have ambitious goals for AI safety and general progress in the next few years

Sample Projects

• Optimizing the throughput of novel attention mechanisms
• Proposing Transformer variants, and experimentally comparing their performance
• Preparing large-scale datasets for model consumption
• Scaling distributed training jobs to thousands of accelerators
• Designing fault tolerance strategies for training infrastructure
• Creating interactive visualizations of model internals, such as attention patterns