Sr. Software Development Engineer in Test (Agentic)
About this role
Developing automated tests for end-to-end product experiences across frontend interfaces, backend services, APIs, and audio or text interactions forms the core daily work for this position. The engineering team focuses on building real-time AI agents for customer experience that resolve issues and eliminate manual tasks instead of merely routing or deflecting inquiries.
Ownership of test automation and quality frameworks for AI Voice Agent services defines the primary scope of the job, requiring close partnership with developers to enhance system testability and stability. Qualified candidates bring substantial software development experience combined with a strong background in testing distributed cloud-native applications and probabilistic systems.
What you will do
- Own end-to-end quality for agentic features and workflows, spanning strategy, development, execution, and release qualification.
- Design and build automation tooling and frameworks for AI and LLM-driven systems, incorporating prompt flows and agent orchestration.
- Develop and maintain evaluation frameworks to measure response quality, accuracy, and hallucination rates.
- Drive automation coverage to achieve over eighty percent coverage for critical AI workflows using deterministic and probabilistic validation.
- Integrate AI quality checks into CI/CD pipelines to maintain fast feedback cycles under fifteen minutes.
- Build tooling for LLM observability and debugging, including prompt tracing and response analysis.
- Partner with Applied AI teams on prompt engineering, model selection, and evaluation strategies.
- Design and execute performance and load tests for AI services focusing on latency, throughput, and cost efficiency.
What you need
- Bring six or more years of experience in software engineering or SDET roles with a strong emphasis on software development.
- Demonstrate strong programming skills in Python, Java, or JavaScript.
- Possess experience testing distributed, cloud-native SaaS systems and APIs.
- Show hands-on exposure to Large Language Models or other AI and machine learning systems.
- Understand non-deterministic systems and probabilistic testing approaches.
- Demonstrate experience building test frameworks and scalable automation systems.
- Hold a Bachelor’s degree in Computer Science or possess equivalent practical experience.
Benefits
- Competitive salary
- Comprehensive benefits
Skills and tech stack
Questions candidates ask
Where is this job located?
This position is based at one of Dialpad’s Canadian offices in Kitchener, Canada.
What is the salary range for the role?
For exceptional talent based in Ontario, Canada, the target base salary range is $151,000 to $175,000 CAD.
Who does the role report to?
The position reports to a QA Engineering Manager located in the United States.