AI Infrastructure Engineer (Senior/Staff)

About Us

We build the data and evaluation layer that frontier AI labs train on: expert marketplaces, licensed real-world work data, and RL environments. Compute is now a core product input. This hire joins a small, senior GPU platform team under the VP of Engineering and owns the clusters, schedulers, and serving stack that every model we train and run depends on. There is no finished platform to inherit; the engineer designs it, writes production code daily, and carries it through on-call.

What the role owns

• GPU clusters across AWS, GCP, neoclouds and on-prem: custom Kubernetes and Slurm operators and the metadata services behind them.

• Scheduling that keeps accelerators busy: gang scheduling, bin-packing, quotas and fair share for multi-tenant teams.

• Distributed training infra (PyTorch Distributed, Ray, Megatron) with fast checkpointing; high-throughput LLM serving on vLLM (TP/EP, prefill/decode disaggregation, KV-cache offload, FP8/FP4).

• Fleet health: hardware/network/software checks with automated remediation; DCGM, Prometheus, Grafana; utilization and billing ingestion for cost attribution and capacity planning.

Must-haves 

• 5+ years building and operating production distributed systems or ML infra (7+ for Staff), counted from full-time industry roles only.

• Has already run GPU workloads at real scale in production. Resume names the systems and the numbers: cluster or node counts, GPU generations, QPS, GPU-hours recovered. Zero handholding; this is not a learn-on-the-job role.

• Owned at least one of: a GPU cluster or scheduler (Kubernetes + Slurm), distributed training infrastructure, or an LLM serving stack. Hands-on, not "partnered with the scheduling team."

• Production Python plus Go or C++. Deep Kubernetes. Working Slurm or other HPC resource manager. One major cloud in depth. Comfortable on-call.

• Individual contributor who still writes code. Directors, architects and managers who stopped shipping code several years ago are not a fit (we have a separate Head of Infrastructure search).

Process

Recruiter screen → initial technical screen with the VP of Engineering → onsite: infra coding (Python), algorithms, AI-specific system design, and a 45-minute GPU-specific interview.