Staff + Sr. Software Engineer, Cloud Inference Launch Engineering

📋 Description

• Be on the critical path for frontier model launches on cloud platforms.
• Bring new inference features to cloud platforms, owning platform integration.
• Identify gaps causing cross-platform differences and fix at source.
• Design, build, and own CI/CD for the inference server and load balancer.
• Reduce merge-to-production cycles with faster, cost-efficient validation.
• Analyze observability data to identify bottlenecks and drive improvements.

🎯 Requirements

• LLM-serving interest; prior inference not required.
• Strong software engineering in high-performance, distributed systems.
• Track record building automation or test infra.
• AWS/GCP/Azure experience; Kubernetes and IaC.
• Thrive in cross-functional collaboration with internal/external partners.
• Fast learner; ramp on new tech, hardware, provider ecosystems.
• Highly autonomous and own end-to-end problems.

🎁 Benefits

• LLM inference optimization, batching, and caching.
• Capacity-constrained scheduling or shared test infra.
• Multi-region deployments, routing, load balancing, global traffic.
• Work with CSP partners to scale infra across platforms.
• Proficiency in Python or Rust.