Devarshi Ojha

Lead SDET · AI Quality & GenAI Evaluation

I build the test systems that let teams ship AI and web products with confidence, from Playwright frameworks to evaluation gates for LLM pipelines.

  1. Jira specs
  2. AI agents generate tests
  3. Web · API · Mobile
  4. AI eval gates
  5. Release
years in quality engineering
9+
regression after Playwright migration
8h → 3h
test cases from agentic generation
~500
AI evaluation run time
5–6h → ~3h
business verticals on my CoE framework
16
QA engineers trained
20+

Case studies

Side project

Solar business platform

A multilingual site and CMS built with AI assistance (Next.js, TypeScript) and tested with Playwright.

  • Next.js
  • TypeScript
  • Playwright
Visit site (opens in a new tab)

What I do

Experience

  1. 06/2026 – 09/2026 · Gurugram

    KlearNow.AI

    Lead Quality Assurance Engineer (AI Ops)

    • Led a QA team of ~6: test strategy, risk-based testing and release sign-off for production GenAI document processing
    • Owned and strengthened the AI evaluation framework; cut run time from 5–6 hours to ~3
    • Introduced agentic test generation (~500 test cases from Jira and Confluence specs)
    • GenAI evaluation
    • Agentic QA
    • Playwright (TypeScript)
    • DeepEval
    • Arize
    • REST Assured
  2. 02/2025 – 06/2026

    Career Break

    Family care, upskilling & freelance

    • Family medical emergency, then upskilling in GenAI evaluation and agentic QA
    • 4 freelance Playwright and API testing engagements for Southeast Asian clients
    • Playwright
    • API testing
  3. 07/2022 – 01/2025 · Gurugram

    StatusNeo Consulting

    Senior Consultant

    • Quality CoE framework adopted by 16 verticals; trained 20+ engineers
    • Led Selenium → Playwright migration for Avis Budget Group: regression 8h → 3h
    • Owned microservices, Kafka and data-replication testing in a mainframe-to-AWS modernization
    • Playwright
    • Selenium
    • REST Assured
    • Kafka
    • AWS
    • Docker
  4. 06/2019 – 06/2022 · Noida

    Publicis Sapient

    Senior QE Associate

    • Sole QA for a headless AEM CMS; authored its test framework from scratch
    • Contributed to Appium mobile automation on BrowserStack and real devices
    • AEM
    • Selenium
    • Appium
    • BrowserStack
  5. 04/2016 – 06/2019 · Noida

    I3infosoft

    QA Consultant

    • Selenium (Java) + TestNG automation for e-commerce web apps
    • Selenium
    • Java
    • TestNG

Skills

AI/ML & GenAI Testing

  • Golden datasets
  • Precision/Recall/F1
  • Field-level validation
  • JSON schema validation
  • LLM-as-judge
  • Prompt regression
  • CI evaluation gates
  • DeepEval
  • RAGAS
  • Promptfoo
  • Arize

Automation

  • Playwright (TypeScript)
  • Selenium (Java)
  • REST Assured
  • Appium
  • Cucumber/BDD
  • TestNG
  • Pytest
  • GraphQL
  • Kafka

Agentic QA

  • Playwright Test Agents
  • MCP (Playwright, Atlassian)
  • Claude Code
  • Cursor

NFR

  • JMeter
  • k6
  • OWASP ZAP
  • Applitools
  • Lighthouse
  • Datadog
  • Grafana
  • Dynatrace

CI/CD & Cloud

  • GitHub Actions
  • Jenkins
  • Azure DevOps
  • Docker
  • Kubernetes
  • AWS
  • Azure

Languages

  • TypeScript
  • Java
  • Python
  • SQL

About

I've spent 9+ years making sure software works before users find out it doesn't, from e-commerce sites to mainframe migrations to GenAI pipelines. Lately my focus is the hardest version of that problem: testing systems whose answers aren't fixed. I believe a good test suite is a product in its own right: fast, trusted and easy for the next engineer to extend.

Contact