Research, Mid-Training

📋 Description

• Data Mix and Quality Uplift: Design high-quality data mixtures for late-stage training.
• Capability Injection: Improve coding, math, and long-horizon reasoning via curated data.
• Synthetic Data Research: Develop and evaluate synthetic data pipelines at scale.
• Annealing and Schedule Design: Optimize learning-rate schedules and compute allocation.
• Context Length Extension: Extend context length without hurting short-context performance.
• Evaluation and Iteration: Build evals to separate real gains from benchmark overfitting.

🎯 Requirements

• End-to-end LLM training pipeline familiarity (pre-training to post-training).
• Continual pre-training, annealing, or late-stage data mixing experience.
• Strong data quality intuition; scalable data filtering and curation.
• Experience with synthetic data pipelines for capability gains.
• Python and DL frameworks (PyTorch, JAX); distributed training at scale.
• Strong optimization, statistics, ML theory; distinguish real effects from noise.
• Original contributions: publications, open-source impact, or internal results.

🎁 Benefits

• Small, selective team where research and product move together.
• Compute not a constraint: thousands of GPUs available from day one.
• Fast-moving environment; autonomy and depth rewarded.