About this role
Testing modern AI-powered applications forms the core of daily operations for this team, focusing on UI and API test automation alongside Agentic AI workflows. The incoming specialist will design, develop, and maintain scalable automation frameworks while validating large language models, AI agents, and evaluation frameworks. Capco operates this search to strengthen quality assurance delivery for complex financial and energy sector transformations. Professionals who possess deep technical expertise in test automation and a collaborative mindset will find strong alignment with the organization's goals.
What you will do
- Design, develop, and maintain robust UI and API automation frameworks using Playwright and TypeScript.
- Automate functional, regression, smoke, integration, and end-to-end test scenarios across modern software systems.
- Validate AI-driven applications, Agentic AI workflows, LLM-powered systems, and intelligent agents.
- Design and execute test strategies for AI models, prompts, workflows, and autonomous agent behaviors.
- Develop and execute evaluation frameworks to measure AI response quality, accuracy, reliability, safety, and performance.
- Collaborate with developers, product owners, business analysts, and AI/ML teams to understand business and technical requirements.
- Integrate automation suites into CI/CD pipelines and ensure continuous quality delivery.
- Mentor junior QA engineers and contribute to automation best practices.
What you need
- Possess 8 to 10 years of experience in software testing and automation.
- Demonstrate strong expertise in UI and API test automation using Playwright with TypeScript.
- Bring hands-on experience in Agentic AI testing, including AI-driven workflows and Large Language Models.
- Show familiarity with AI agents and evaluation frameworks for automated testing.
- Have the ability to perform defect tracking, root cause analysis, and support issue resolution.
- Prepare comprehensive test plans, test cases, automation scripts, execution reports, and quality metrics.
Skills and tech stack
Questions candidates ask
What locations are available for this position?
The job posting lists locations in Bangalore, Chennai, Hyderabad, and Mumbai.
What is the required experience level for this role?
Candidates must have 8 to 10 years of experience in software testing and automation.
What technical skills are mandatory for the position?
Applicants need strong expertise in UI and API test automation using Playwright with TypeScript, along with hands-on experience in Agentic AI testing and Large Language Models.