About this role
Ensuring the reliability and quality of non-deterministic Agentic AI systems and backend services requires specialized test automation frameworks and evaluation strategies. PlayStation Global is hiring a Software Development Engineer in Test Contractor in Waterloo, Ontario, to design and execute comprehensive quality engineering plans for complex AI-driven applications and distributed systems.
In this role, the contractor will build scalable automated test frameworks, validate API contracts and microservices, and develop methodologies for assessing non-deterministic AI behavior and guardrails. Working alongside software engineers and product managers, the person in this position will integrate test automation into CI/CD pipelines and establish quality metrics for release readiness.
An ideal candidate brings solid quality engineering fundamentals alongside hands-on experience testing generative or agentic AI applications. Comfortable navigating systems where traditional pass-and-fail testing falls short, this professional will help maintain the reliability of network services used by millions.
What you will do
- Develop and execute comprehensive quality engineering strategies and test plans for Agentic AI systems, backend services, and APIs.
- Design and maintain scalable automated test frameworks across functional, integration, contract, and end-to-end testing layers.
- Create testing approaches for non-deterministic AI behavior and establish meaningful evaluation criteria for accuracy and groundedness.
- Identify quality risks early and apply shift-left testing practices throughout the software development lifecycle.
- Integrate automated testing into CI/CD pipelines to establish effective quality gates and release readiness.
- Partner with software engineers and product managers to improve overall product quality, testability, and system observability.
- Analyze defects and production issues to determine root causes and identify opportunities for continuous testing improvement.
What you need
- Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field, or equivalent practical experience.
- At least 4 years of professional experience in Software Development Engineering in Test, Quality Engineering, or Software Engineering.
- Strong background testing REST APIs, backend services, microservices, and distributed systems.
- Hands-on experience building automated tests using Python, Java, JavaScript, TypeScript, or similar programming languages.
- Demonstrated experience designing and maintaining automated test frameworks and integrating tests into CI/CD pipelines.
- Practical experience testing Generative AI, LLM-powered applications, or Agentic AI systems and non-deterministic behavior.
Nice to have
- Experience testing Agentic AI workflows including planning, tool calling, and state management.
- Familiarity with AI evaluation techniques and observability tools such as LangSmith, DeepEval, Ragas, or Promptfoo.
- Experience testing RAG systems, contract testing tools like Pact, or API specifications like OpenAPI and Swagger.
- Familiarity with event-driven architectures, Kafka, Docker, Kubernetes, or AWS.
- Experience with performance, scalability, reliability, or resiliency testing.
Skills and tech stack
Questions candidates ask
What is the pay range for this role?
The estimated Ontario base pay range for this role is $51.61 to $77.42 CAD. Individual base pay is determined based on job-related factors such as knowledge, skills, experience, and location.
Where is the position located?
This role is based in Canada, Waterloo, ON. Base pay ranges may vary in line with the company hybrid working policy.
What kind of systems will I be testing?
You will design and implement automated testing strategies across AI and LLM-driven applications, APIs, microservices, and distributed systems. This includes validating Agentic AI workflows, multi-step execution, tool selection, and AI guardrails.
What background checks are required?
Sony Interactive Entertainment conducts background checks at the offer stage for all new employees. This may include criminal background checks depending on the specific role.
What programming languages are required for automation?
You should have hands-on experience developing automated tests using Python, Java, JavaScript, TypeScript, or a similar programming language. Experience with Git and CI/CD development workflows is also required.
The application is on the official PlayStation Global careers page.
Apply on company site