Senior Machine Learning Engineer, Conversational AI - Services Special Projects - Jobs - Careers at Apple
- Orchestrate tools and grounding: Own the agent's planning and tool-calling layer, including tool schemas, dynamic tool selection, and structured output generation. Ground every response in retrieval, knowledge graph, and real-time data.
- Build conversational understanding: Design and ship conversational flows, including intent understanding, context tracking, and disambiguation, using LLM- and small-model-based query parsers that feed downstream retrieval and ranking.
- Optimize LLM inference: Profile and reduce serving latency and cost through distillation, quantization, compact output formats, dynamic batching, KV-cache tuning, and semantic caching. Use inference runtimes such as vLLM, SGLang, TensorRT-LLM, and ONNX Runtime, and decide when a request needs a large model, a small model, or none at all.
- Build autograded evaluation: Design and own automated evaluation for conversational and generative quality, including multi-turn rubrics, LLM-as-a-judge pipelines, judge calibration against human ratings, and error-analysis tooling. Integrate eval runs into CI.
- Connect offline metrics to online outcomes: Define offline metrics that predict user-facing wins, and partner on the A/B tests that confirm them.
- Ship models and services to production: Take models and services from experiment to the live request path with proper versioning, monitoring, and rollback, and own their quality, latency, and reliability after launch.
- Collaborate cross-functionally: Work closely with research scientists, engineers, and product teams to turn recent research in conversational AI, retrieval, and inference into systems that hold up in production.
- Master's degree in Computer Science, Machine Learning, NLP, or a related field
- 7+ years of experience as a software or machine learning engineer, with a significant focus on conversational AI, search, or LLM-powered systems
- Strong programming skills in Go and Python, plus at least one of Java or Scala
- Hands-on experience building and shipping conversational AI or dialogue systems: intent and slot understanding, multi-turn context management, clarification, and disambiguation
- Experience building LLM-powered applications in production, including prompt design, tool/function calling, structured output generation, and agentic orchestration patterns
- Demonstrated LLM inference-optimization experience: distillation, quantization, batching, KV-cache tuning, caching, and profiling to find real bottlenecks rather than guessing at them
- Hands-on experience deploying and serving LLMs or ML models directly in the production request path, using serving frameworks such as vLLM, SGLang, Triton, or TorchServe
- Experience with knowledge graphs and low-latency lookup and retrieval services
- Fluency with PyTorch and the Hugging Face ecosystem (transformers, PEFT)
- Familiarity with cloud environments (including AWS) and containerization (Docker, Kubernetes)
- Excellent communication skills and a collaborative mindset
- Ph.D. in Computer Science, Machine Learning, NLP, or a related field
- Experience fine-tuning and distilling LLMs for task-specific use, such as query parsing, classification, and response generation, using techniques such as LoRA/adapters and teacher/student setups
- Experience designing knowledge graph schemas, building graph compilation and ingestion pipelines, and working with graph databases such as TigerGraph or Neo4j
- Deep experience with KV stores and in-memory serving for low-latency lookups (e.g., Cassandra, Redis, SSTables)
- Experience calibrating LLM judges against human judgments for multi-turn conversations, including inter-annotator agreement and judge reliability tracking
- Experience with personalization and memory in conversational systems, and applying privacy principles such as data minimization and on-device processing
- Experience implementing safety guardrails for LLM-powered agents, including prompt-injection defenses and adversarial evaluation
- Familiarity with experiment tracking and data/model versioning tools (MLflow, Weights & Biases, DVC)
- Deep understanding of distributed systems and service design, and experience architecting and leading projects with cross-functional stakeholders
- Published work or patents in conversational AI, information retrieval, knowledge graphs, or efficient inference
Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant
At Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.
Learn about accessibility in Apple’s workplace
Learn about reasonable accommodations for job applicants
Apple accepts applications to this posting on an ongoing basis.
Required Skills
Required Languages
🇬🇧 English