Overview
Inferact's mission is to grow vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and faster. Founded by the creators and core maintainers of vLLM, we sit at the intersection of models and hardware, a position that took years to build.
About the Role
We're looking for a Site Reliability Engineer to make Inferact's vLLM-powered inference services dependable in production. You'll work alongside the engineers building the platform, bringing a reliability perspective to how systems are designed, shipped, and operated. This is a hands-on engineering role for someone who thinks about failure before launch, simplifies operations, and writes software that makes reliable inference possible at scale.
You'll help own reliability across the production lifecycle, from release safety and observability to incident response and recovery. You'll define SLOs around availability, latency, and successful inference requests; build monitoring that helps engineers pinpoint failures; and automate recurring operational work. When incidents occur, you'll drive mitigation, coordinate escalation, and lead post-mortems that result in concrete prevention work. You'll also partner with engineering on capacity planning and operational readiness so the platform can grow without becoming harder to operate.
Skills and Qualifications
Minimum qualifications:
Bachelor's degree or equivalent experience in computer science, engineering, systems, infrastructure, or similar.
Hands-on experience building or operating production distributed systems, cloud infrastructure, or services where reliability has meaningful user or business impact.
Strong programming or scripting ability in Python, Go, Bash, or similar, with experience building tools and automation that solve operational problems.
Experience responding to significant production incidents, including mitigation, root cause analysis, escalation, and following corrective actions through to completion.
Practical understanding of service-level objectives (SLOs), service-level indicators (SLIs), error budgets, and alerting that distinguishes user-impacting failures from noise.
Strong Linux, networking, and systems debugging fundamentals, with the ability to investigate failures across application and infrastructure boundaries.
Clear communication, sound judgment under pressure, and the ability to work with engineering teams to identify failure modes and make systems simpler to operate.
Preferred qualifications:
Experience supporting AI inference, model serving, ML infrastructure, GPU workloads, or other latency-sensitive, high-throughput services.
Experience deploying, operating, and debugging Kubernetes-based services, with familiarity with Docker, Terraform, or comparable infrastructure tooling.
Experience building observability with metrics, logs, traces, dashboards, and actionable alerts across distributed systems.
Experience improving deployment safety and recovery through automation, testing, release gates, and rollback procedures.
Experience with resource scheduling, capacity planning, or workload isolation in multi-tenant cloud or GPU environments.
Bonus points if you have:
Owned reliability for an inference service or other mission-critical platform through a period of significant growth.
Built automation that measurably reduced operational toil, improved recovery time, or prevented repeat incidents.
Established practical SLOs, post-mortem processes, or production-readiness reviews that helped engineering teams ship with greater confidence.
Logistics
Location: This role is based in San Francisco, California. Will consider remote in the US for exceptional candidates.
Compensation: Depending on background, skills, and experience, the expected annual salary range for this position is $200,000 - $400,000 USD + equity.
Visa sponsorship: We sponsor visas on a case-by-case basis.
Benefits: We offers generous health, dental, and vision benefits as well as 401(k) company match.