Software Engineer - ML Infrastructure

Software Engineer – ML Infrastructure

Location

San Francisco, CA

Company Stage of Funding

Series A2
Total Funding: Approximately $28M

Office Type

Fully on-site — 5 days per week in the San Francisco office on Market Street.

Salary

$250,000–$300,000 USD annually.

Equity

Competitive equity.

Visa

Open to visa transfers and new visa sponsorships, including OPT, H-1B transfers, new H-1B, and TN.

Employment Type

Full-time

Hiring Count

1

Company Description

This company is building a radiology-as-a-service platform powered by a multimodal AI copilot, with the goal of improving diagnostic accuracy and patient care through AI-powered medical imaging.

The company combines AI technology with a radiology practice and works with major medical institutions across the United States. It has served more than 300,000 patients and is working toward reading 1% of all daily X-rays in the US by year-end.

The platform is built around multimodal medical imaging, including X-rays, MRI scans, CT scans, echocardiograms, and ultrasounds. The company has access to more than 250,000 unique radiology images for model training and has secured its first customer through a signed $300,000 annual contract.

Backed by Bedrock Capital, Trust Ventures, and Jack Altman, the company has raised approximately $28M and is operating as a small, fast-moving team.

The engineering and research teams work closely together, with a strong emphasis on building foundational AI systems and infrastructure that allow researchers to experiment, train, evaluate, and deploy models efficiently.

What You Will Do

1. Build Distributed Training Infrastructure

• Design and build the core infrastructure required to train large foundation models at scale.

• Own distributed training systems for large-scale medical imaging datasets.

• Build infrastructure that supports long-context parallelism and checkpointing for volumetric CT and MRI training.

• Ensure training workloads are scalable, efficient, and reliable.

• Identify and remove system bottlenecks that slow down research and experimentation.

2. Build High-Throughput Data Loading and Preprocessing

• Design high-throughput data loading and preprocessing systems for large volumetric and multimodal datasets.

• Optimize data pipelines to keep GPUs fully utilized during model training.

• Build efficient workflows for handling large-scale medical imaging data.

• Improve data throughput and preprocessing performance to support faster experimentation.

3. Own Reinforcement Learning Infrastructure

• Build the reinforcement learning training stack for foundation-model and post-training workflows.

• Develop high-throughput rollout generation systems.

• Build and manage reward-model serving infrastructure.

• Implement experience collection systems to support large-scale training.

• Enable researchers to run online, multi-reward reinforcement learning at scale.

4. Build Model Inference and Serving Systems

• Own inference systems that take models from experimentation into production.

• Build efficient model-serving infrastructure.

• Contribute to production serving and deployment pipelines.

• Work on systems that support reliable and scalable model inference.

• Help ensure models can move from research environments into production applications.

5. Build Evaluation and Experimentation Systems

• Develop evaluation infrastructure to support model experimentation and research.

• Build reliable systems for testing and evaluating models throughout the development lifecycle.

• Work closely with researchers to understand their workflows and anticipate evolving infrastructure needs.

• Make experimentation and training faster and more reliable so researchers can focus on scientific work.

6. Partner With Research and Engineering Teams

• Work directly with researchers to understand their training and experimentation workflows.

• Prototype new ideas alongside researchers and translate them into production-ready systems.

• Own end-to-end delivery, from experimentation through deployment and monitoring.

• Collaborate with the backend team on model rollout, canary deployments, and monitoring.

• Help establish the infrastructure foundations as an early member of the engineering team.

Ideal Candidate Background

Experience Requirements

• Strong software engineering background with experience in ML infrastructure, distributed systems, or large-scale model training.

• Experience building infrastructure for training, serving, or evaluating machine learning models.

• Ability to work closely with research teams and translate research requirements into production-ready systems.

• Experience with distributed training and high-performance computing environments.

• Comfortable owning infrastructure end to end in a small, fast-moving startup.

• Able to operate autonomously and make technical decisions with limited management overhead.

• Strong problem-solving skills and ability to identify and remove infrastructure bottlenecks.

The intake discussion indicates flexibility on years of experience for ML Infrastructure candidates. Prioritize technical depth, relevant infrastructure ownership, and ability to execute independently over a rigid years-of-experience requirement.

Technical Requirements

Programming and ML Frameworks

• Strong Python experience.

• Hands-on experience with PyTorch and/or JAX.

• Ability to work with research-oriented machine learning code and production systems.

Distributed Training

• Experience with distributed model training.

• Familiarity with FSDP, DeepSpeed, and/or Megatron.

• Understanding of distributed training architecture, parallelism, and checkpointing.

• Ability to optimize training efficiency across large-scale workloads.

Data Infrastructure

• Experience building high-throughput data loading and preprocessing systems.

• Understanding of GPU utilization and performance bottlenecks.

• Experience handling large datasets and complex data pipelines.

• Familiarity with multimodal or volumetric data is valuable.

Inference and Model Serving

• Experience with vLLM and/or SGLang.

• Familiarity with TensorRT and Triton.

• Understanding of model inference, serving, and deployment.

• Ability to build efficient production model-serving systems.

Infrastructure and Systems

• Experience with Kubernetes and Docker.

• Strong understanding of distributed infrastructure and production deployment environments.

• Familiarity with CUDA and GPU-based workloads.

• Ability to build reliable infrastructure for experimentation, training, and production.

Reinforcement Learning Experience

• Experience building or supporting reinforcement learning infrastructure.

• Familiarity with high-throughput rollout generation.

• Experience with reward-model serving or experience collection.

• Understanding of online reinforcement learning workflows.

• Ability to support multi-reward RL training at scale.

Ownership and Execution

• Can own infrastructure projects from initial design through deployment and monitoring.

• Comfortable working directly with researchers to understand and solve infrastructure challenges.

• Able to prototype quickly and turn experimental systems into reliable production code.

• Takes initiative in identifying performance and scalability bottlenecks.

• Comfortable operating in a small engineering team with minimal management layers.

• Strong ability to prioritize and execute independently.

Collaboration and Communication

• Strong communication skills when working with research and engineering teams.

• Comfortable translating research requirements into infrastructure solutions.

• Able to explain technical decisions and performance trade-offs.

• Collaborative and proactive when debugging complex systems.

• Comfortable working in a high-ownership startup environment.

Nice-to-Have Qualifications

• Experience training large foundation models.

• Experience with multimodal models.

• Experience working with medical imaging data, including CT, MRI, X-rays, or other imaging modalities.

• Experience with volumetric data processing.

• Experience with long-context parallelism and distributed checkpointing.

• Experience building online reinforcement learning infrastructure.

• Experience with GPU optimization and CUDA.

• Background at a strong AI research lab, infrastructure company, or technically demanding startup.

• Experience across both Big Tech and early-stage startup environments.

Compensation & Benefits

• Base salary of $250,000–$300,000 USD annually.

• Competitive equity.

• Full-time position.

• On-site work, 5 days per week in San Francisco.

• Visa transfers and new sponsorships supported, including OPT, H-1B, and TN.

Why Join

• Work on foundational ML infrastructure for a multimodal AI platform focused on healthcare and medical imaging.

• Own distributed training, reinforcement learning, inference, and evaluation systems end to end.

• Work directly with researchers and influence how large models are trained and deployed.

• Build infrastructure for large-scale medical imaging datasets, including CT, MRI, X-rays, and other modalities.

• Join a small engineering team with significant ownership and minimal management layers.

• Help establish core ML infrastructure at an early stage, with direct impact on research velocity and production capabilities.

• Work on a product with an existing customer contract and access to a substantial medical imaging dataset.