AI Systems Quality Specialist

The AI Systems Quality Specialist will design, build, and oversee the quality assurance framework for EVERSANA’s AI Agency Platform.

This role ensures that every feature, integration, and AI-driven process operates reliably and compliantly. The QA Automation Lead will establish automated testing frameworks covering functional, regression, and compliance scenarios, manage integration test suites across connected systems (Google Cloud, Veeva, SFMC, Aprimo), and collaborate closely with DevOps, Product, and AI teams to uphold quality benchmarks.

The ideal candidate combines deep QA automation experience with an understanding of regulated environments and AI-enabled software systems.

ESSENTIAL DUTIES AND RESPONSIBILITIES:
Our employees are tasked with delivering excellent business results through the efforts of their teams.  These results are achieved by:

Automation Framework Design & Execution

• Design, implement, and maintain a comprehensive automation framework for functional, regression, and integration testing.
• Build automated test scripts for APIs, UI workflows, data pipelines, and AI agent interactions using tools such as PyTest or Selenium.
• Integrate automated tests into CI/CD pipelines (Jenkins, GitHub Actions, or Cloud Build) to support continuous delivery.
• Ensure test scripts cover both core platform functionality and partner integrations (Veeva Vault, Vertex AI, Snowflake).
• Standardize QA practices for cross-team use and document testing methodologies.

Functional, Regression, and Compliance Testing

• Develop and execute test cases for new features, enhancements, and bug fixes.
• Conduct regression testing prior to each release to ensure platform stability.
• Define and perform compliance and validation testing to ensure platform adherence to HIPAA, GDPR, SOC 2, and MLR workflow standards.
• Work with the AI Validation and Compliance teams to evaluate AI output quality (accuracy, hallucination, tone, compliance alignment).
• Ensure traceability between business requirements, test cases, and validation results through detailed test documentation.

Integration & System Reliability Testing

• Oversee integration testing for all platform connectors and data flows (Veeva Vault, Aprimo, SharePoint, SFMC, Google Vertex AI).
• Develop synthetic test environments and mock APIs to validate complex, multi-system workflows.
• Partner with Data Engineering to validate ETL pipelines and analytics dashboards for data accuracy.
• Track and report system reliability metrics (latency, failure rate, API response times).

Defect Management & Continuous Improvement

• Identify, log, and prioritize defects using Jira, tracking issue resolution through sprints.
• Collaborate with Product and Engineering teams to verify bug fixes and perform root cause analysis for recurring issues.
• Lead post-release validation and QA retrospectives to improve process efficiency.
• Proactively identify automation opportunities to reduce manual testing effort and cycle time.

Documentation & Audit Readiness

• Maintain test documentation, validation reports, and evidence logs for internal and external audits.
• Support compliance reviews by ensuring full traceability from requirement → test case → result → release.
• Create and maintain QA dashboards and reporting templates for leadership visibility.

• Demonstrate a commitment to diversity, equity, and inclusion through continuous development, modeling inclusive behaviors, and proactively managing bias.
• All other duties as assigned

MINIMUM KNOWLEDGE, SKILLS AND ABILITIES:

The requirements listed below are representative of the experience, education, knowledge, skill and/or abilities required.

• 7+ years of experience in QA automation or software testing, with 3+ years in a lead or senior role.
• Proficiency in Python, JavaScript, or .Net for test automation scripting.
• Experience with test automation frameworks such as PyTest, Selenium.
• Strong understanding of API testing (Postman) and data validation (SQL, Snowflake, BigQuery).
• Proven experience testing integrated SaaS systems in cloud environments (preferably Google Cloud).
• Familiarity with CI/CD and continuous testing practices.
• Excellent documentation, reporting, and stakeholder communication skills.

OUR CULTURAL BELIEFS:

Patient Minded I act with the patient’s best interest in mind.

Client Delight I own every client experience and its impact on results.

Take Action I am empowered and empower others to act now.

Grow Talent I own my development and invest in the development of others.

Win Together I passionately connect with anyone, anywhere, anytime to achieve results.

Communication Matters I speak up to create transparent, thoughtful and timely dialogue.

Embrace Diversity I create an environment of awareness and respect.

Always Innovate I am bold and creative in everything I do.

Our team is aware of recent fraudulent job offers in the market, misrepresenting EVERSANA. Recruitment fraud is a sophisticated scam commonly perpetrated through online services using fake websites, unsolicited e-mails, or even text messages claiming to be a legitimate company. Some of these scams request personal information and even payment for training or job application fees. Please know EVERSANA would never require personal information nor payment of any kind during the employment process. We respect the personal rights of all candidates looking to explore careers at EVERSANA.

EVERSANA is committed to providing competitive salaries and benefits for all employees. If this job posting includes a base salary range, it represents the low and high end of the salary range for this position and is not applicable to locations outside of the U.S.  Compensation will be determined based on relevant experience, other job-related qualifications/skills, and geographic location (to account for comparative cost of living). This role is eligible for hire in select U.S. locations based on business and operational considerations. For a full list of locations, visit eversana.com/careers. More information about EVERSANA’s benefits package can be found at eversana.com/careers. EVERSANA reserves the right to modify this base salary range and benefits at any time.

From EVERSANA’s inception, Diversity, Equity & Inclusion have always been key to our success. We are an Equal Opportunity Employer, and our employees are people with different strengths, experiences, and backgrounds who share a passion for improving the lives of patients and leading innovation within the healthcare industry. Diversity not only includes race and gender identity, but also age, disability status, veteran status, sexual orientation, religion, and many other parts of one’s identity. All of our employees’ points of view are key to our success, and inclusion is everyone's responsibility.

Follow us on LinkedIn | Twitter