Senior Software Engineer, AI Model serving - Kharkiv, Ukraine

Job Description

Speechify is seeking a Senior Software Engineer to lead AI model serving efforts in Kharkiv, Ukraine. You will design, implement, and maintain scalable infrastructure and microservices that deploy and serve machine learning models for real-time inference, ensuring low latency, high reliability, and robust monitoring. You will collaborate with ML researchers, data scientists, and product teams to turn cutting-edge models into production-ready features.

Responsibilities

• Build and maintain scalable AI model serving infrastructure and microservices for model inference
• Design and implement APIs (REST and gRPC) for online inference and batch processing
• Optimize performance, latency, throughput, and reliability of model serving systems
• Monitor, log, and instrument services with appropriate metrics and dashboards
• Collaborate with ML researchers and data scientists to deploy models from research into production
• Develop and maintain CI/CD pipelines, testing strategies, and secure deployment practices
• Participate in architectural decisions and code reviews; mentor junior engineers

Requirements

• 5+ years of software engineering experience
• Hands-on experience with ML model serving frameworks (e.g., TorchServe, TensorFlow Serving) and Python
• Strong experience with Docker and Kubernetes for containerized deployments
• Proficiency with cloud platforms (AWS/GCP/Azure) and distributed systems
• Experience building scalable REST/gRPC APIs and working with ML pipelines
• Bachelor’s or Master’s degree in Computer Science or a related field
• Strong English communication skills and ability to collaborate with cross-functional teams

Nice to Have

• Experience with PyTorch/TensorFlow in production
• Familiarity with MLOps tooling (Prometheus, Grafana, CI/CD, monitoring)

About Speechify

Speechify builds accessible speech and reading tools powered by AI to help people use and enjoy content in new ways. Join a team focused on delivering fast, reliable AI-driven features to users worldwide.