AI Product Analyst

AI Product Analyst - Voice AI Agents | MEDvidi
#job #vacancy #AI #VoiceAI #ProductAnalytics #LLM #AIagents #DataAnalytics #Startup #remote #MEDvidi #telehealth
MEDvidi is a US-based AI-powered mental healthcare platform helping patients across 30+ states receive treatment for ADHD, anxiety, depression, and other mental health conditions.
We're looking for an AI Product Analyst to help us evaluate, monitor, and improve Voice AI Agents that interact directly with patients and automate critical parts of the patient journey.
This role sits at the intersection of Product, AI, Engineering, and Operations. You'll investigate how AI behaves in production, identify failure patterns, measure AI quality, and help teams make better product decisions using real-world data.
What you'll work on:
- Analytics for Voice AI Agents and conversational AI systems
- Investigation of hallucinations, inconsistencies, silent failures, and quality degradation
- Analysis of call transcripts, conversation logs, AI outputs, and operational datasets
- AI quality evaluation and reliability measurement
- Building monitoring frameworks and dashboards for AI performance
- Detecting quality shifts, anomalies, and emerging failure modes
- Measuring AI impact on patient experience, operations, and business outcomes
- Partnering with Product, Engineering, QA, and Operations teams
What we expect:
- 3+ years of experience in Product Analytics, Data Analytics, or Operational Analytics
- Strong SQL skills
- Python for analytical investigations and working with large-scale text datasets
- Experience working with ML pipelines or AI-powered products
- Understanding of LLM limitations, hallucinations, prompt sensitivity, and AI failure modes
- Experience building monitoring systems, analytical frameworks, or dashboards
- Strong communication skills and ability to translate insights into product decisions
- Fluent Russian and English B2+
- CET-overlapping working hours
Nice to have:
- Experience with conversational AI or AI agents
- Experience evaluating LLM outputs
- Experience with AI quality metrics and evaluation frameworks
Why join MEDvidi:
- Work on one of the most visible AI products inside the company
- Analyze real Voice AI systems used by patients every day
- Help shape the future of AI in healthcare
- High ownership and direct impact on product decisions
- Fully remote long-term B2B collaboration
- Health insurance, wellness compensation, paid vacation & sick leave
- Personalized English lessons via Preply
Apply here: