Senior Deployment Engineer - LLM Inference & Optimization

Senior Deployment Engineer - LLM Inference & Optimization

• Understand customer requirements: Lead technical discovery to define use cases, workloads, infrastructure constraints, and targets for model quality, latency, throughput, and cost.
• Optimize inference: Apply techniques such as quantization, batching, prefix caching, KV-cache optimization, and parallelism to improve performance while preserving output quality.
• Adapt models to customer needs: Improve results through prompt optimization, retrieval-augmented generation, and fine-tuning where appropriate. Evaluate the impact of each approach.
• Troubleshoot performance: Investigate issues across model execution, GPU utilization, memory consumption, serving infrastructure, and application behavior.
• Lead technical customer engagement: Run workshops, present benchmark findings, explain tradeoffs, and guide customers through deployment and optimization decisions.
• Build reusable capabilities: Create evaluation tools, benchmark harnesses, deployment configurations, and runbooks. Bring recurring customer needs and performance findings back to product and engineering.

• 5+ years of experience in software engineering, ML engineering, AI infrastructure, or a related hands-on technical role.
• Strong Python programming skills and experience with frameworks such as PyTorch, Hugging Face Transformers, or comparable tools.
• Hands-on experience deploying and optimizing AI models in production, particularly LLMs.
• Experience with model-serving frameworks such as vLLM, SGLang, TensorRT-LLM, or similar technologies.
• An understanding of GPU memory constraints, numerical precision, batching, caching, and parallelism, including how these affect inference performance.
• Experience building evaluations and benchmarks to measure model quality, latency, throughput, and resource efficiency.
• Experience developing APIs and integrating model endpoints with production applications and customer data systems.
• Familiarity with GPU-based cloud environments, containers, and monitoring tools used to operate model deployments.
• The ability to work directly with customers, communicate technical tradeoffs clearly, and translate business requirements into measurable engineering targets.
• Strong ownership and troubleshooting skills, with the ability to move from an ambiguous problem to a tested solution.

• Experience with parameter-efficient fine-tuning techniques such as LoRA or QLoRA.
• Experience with speculative decoding, multi-GPU inference, or distributed model serving.
• Experience profiling GPU workloads using CUDA, NVIDIA Nsight, or related tools.
• Experience optimizing embedding models, rerankers, vision models, or multimodal systems.
• Experience improving retrieval quality and evaluating end-to-end RAG applications.
• Contributions to open-source model-serving, optimization, or evaluation projects.
• Experience delivering enterprise AI implementations or customer-facing technical engagements.