Mar 11, 2026
Mphasis Corporation
Role Summary
This role requires deep understanding of AI fundamentals, handson experience with LLM testing, guardrail design, data sanitization,
and evaluation of agent trajectories. The tester will work closely with AI engineers, data scientists, and product teams to ensure safe, reliable, and highquality AI behavior.
5-8+ years in QA, with at least 2-3 years in AI/ML or LLM testing.
Strong understanding of AI fundamentals: embeddings, vector stores, RAG, LLMs, agentic frameworks.
Handson experience testing:
LLM prompts and outputs
Agentic flows (e.g., LangChain, AutoGen, Semantic Kernel, AIP agents)
Model evaluation metrics
Experience writing guardrails, safety rules, and evaluation prompts.
Knowledge of data sanitization, PII detection, and data governance.
Familiarity with pipeline validation and MLOps workflows.
Ability to evaluate model performance using LowPick Score or similar ranking metrics.
Strong Python skills for...
Professional Diversity Network
New York, NY, USA
Full-Time
