Latest AI JobsBrowse Jobs
A

QA Engineer, Voice AI Agents

arbiter-aiRemote, USEngineering
Apply on company site
Salary
$180K-$200K.
Job type
Full-time
Work mode
Remote
Experience
Mid · 3+ yrs
Posted
8 Oct 2026
Status
Open · checked 7d ago

About this role

Remote, US (or onsite in New York City or Boca Raton) is the location for this full-time QA Engineer position focusing on voice AI agents at arbiter-ai. This role suits a quality assurance professional with technical operations experience who enjoys debugging complex conversational systems and healthcare platforms.

Role Overview

arbiter-ai is hiring a QA Engineer to investigate call and platform issues, review conversations, and trace failures through transcripts, logs, and campaign configurations. Reporting to the Head of Product, this position involves collaborating with engineering, deployment, and clinical operations teams to establish structured quality assurance processes and growing automation.

Key Responsibilities

Investigate live campaign issues such as dropped calls, incorrect agent responses, failed handoffs, scheduling errors, and data mismatches.
Trace problems across call recordings, system logs, transcripts, campaign settings, and integration data to determine root causes.
Create reproducible bug reports detailing member impact, severity, and supporting evidence for the engineering team.
Triage incoming problems to separate agent behavior from configuration mistakes, platform bugs, and data issues.
Perform pre-launch quality checks on new campaigns, testing edge cases, scripts, escalation paths, and staff handoffs.
Conduct ongoing conversation evaluations on active campaigns against established quality benchmarks.
Verify campaign design execution including targeting, sequencing, outcome capture, and disposition logic.
Build and maintain testing frameworks consisting of test cases, scoring rubrics, regression suites, and release checklists.
Automate routine checks involving transcript analysis, configuration validation, and outcome reconciliation.
Track quality metrics and report recurring patterns to product, engineering, and deployment departments.

Requirements

Three to five years of professional background in QA, technical operations, support engineering, or related production software roles.
Strong debugging instincts with persistence in finding underlying system failures.
Capability to read logs, query data using SQL, and handle APIs alongside JSON or YAML configurations.
Demonstrated experience authoring trusted test plans and bug reports.
Clear written communication and precise documentation skills.
Sound judgment regarding issue severity and prioritization during simultaneous live incidents.

Nice to Have

Background working with voice AI, conversational AI, IVR, or contact center systems.
Scripting proficiency in Python or similar languages for task automation and analysis.
Familiarity with LLM evaluation, conversation quality scoring, or prompt testing.
Healthcare sector exposure, particularly involving payer or provider outreach, HIPAA compliance, and PHI handling.
Familiarity with call analytics or observability tools.

Skills and Tools

SQL
APIs
JSON
YAML
Python

Salary and Benefits

Annual base salary range of $180,000 to $200,000.
Equity package designed to rival top compensation levels.
Comprehensive health, vision, and dental insurance coverage.
Life and disability insurance.
Simple IRA matching program.
Generous paid time off.
Professional development budget for conferences, certifications, and courses.
Wellness programs supporting physical and mental health.

About the Company

arbiter-ai provides an AI-powered care orchestration system uniting healthcare through a patient-facing agentic platform. Backed by a major seed round in health tech history, the organization builds connected infrastructure designed to optimize complex healthcare workflows and patient outcomes using multimodal approaches.

How to Apply

Interested candidates should apply on the official company careers page.

Skills and tech stack

SQLAPIsJSONYAMLPythonLLM

Questions candidates ask

What is the salary range for the QA Engineer role?

The annual base salary range is $180,000 to $200,000, accompanied by an equity package and comprehensive benefits.

Where is the position located?

The role is remote within the US, or can be performed onsite at offices in New York City or Boca Raton.

How many years of experience are required?

Candidates must have 3 to 5 years of experience in QA, support engineering, technical operations, or a similar role on a production software platform.

What technical skills are necessary for this position?

Required technical skills include reading logs, querying data with SQL, working with APIs, and handling JSON or YAML configurations.

Who does this position report to?

The QA Engineer reports directly to the Head of Product.

Ready to apply?

The application is on the official arbiter-ai careers page.

Apply on company site