This is a QA engineering role with AI responsibilities layered on top. You’ll test the full product, UI/UX, functionality, and integrations while also assuring the quality, safety, and reliability of the AI agents and automated workflows the team ships.
Testing AI is not the same as testing software. Outputs are probabilistic, behaviour drifts, and a flow that passed yesterday can fail today on the same input. This role covers both worlds and we’re looking for someone who understands the difference and thrives in it.
What You'll Do
- Build and maintain UI and API test automation
- Design and run test plans for AI workflows and agent behaviours
- Build regression suites and golden datasets to re-check behaviour after any change or model swap
- Evaluate prompt and response quality across models — accuracy, consistency, tone, and failure modes
- Test edge cases, adversarial inputs, hallucination risk, and prompt-injection exposure
- Verify that human-in-the-loop checkpoints trigger and escalate correctly as designed
- Validate integrations end to end — APIs, data retrieval, automated hand-offs, and data handling
- Monitor live AI agents and flag behavioural drift before it reaches end users
- Log defects clearly, retest fixes, and own the quality gate before handover
What We're Looking For
- Required
- 3+ years in software QA, test automation, or quality engineering
- Strong test design skills — functional, regression, edge-case, and exploratory
- Comfortable working with non-deterministic systems where "correct" is a range, not a fixed value
- API and integration testing experience with structured data validation
- Clear, precise defect reporting that an engineer can act on without a follow-up call
- Methodical and sceptical — you won't sign something off just to meet a deadline
- Experience testing LLM, chatbot, or conversational AI systems
- Hands-on experience with OneReach.ai or a comparable orchestration platform
- Familiarity with LLM evaluation frameworks and prompt-evaluation tooling
- Test automation scripting in Python or similar
- Awareness of AI safety standards — EU AI Act, ISO 42001, or NIST AI RMF
Tools & Technologies
Technologies, platforms, and tools you’ll be working with.
#J-18808-Ljbffr