QA Automation Engineer / Senior QA Automation Engineer (AI Platforms), AVP

Position Overview

Job Title: QA Automation Engineer / Senior QA Automation Engineer (AI Platforms), AVP

Location: Pune, India

Role Description

The QA Automation Engineer ensures quality, reliability, and stability of AI-powered applications and platforms. The role involves designing automation frameworks, validating machine-learning–driven outputs, and ensuring consistent performance of both traditional and AI-driven systems. You will work with developers, data scientists, ML engineers, and product teams to build robust test strategies that validate model behavior, data quality, and AI user experiences.

What we’ll offer you

As part of our flexible scheme, here are just some of the benefits that you’ll enjoy

• Best in class leave policy
• Gender neutral parental leaves
• 100% reimbursement under childcare assistance benefit (gender neutral)
• Sponsorship for Industry relevant certifications and education
• Employee Assistance Program for you and your family members
• Comprehensive Hospitalization Insurance for you and your dependents
• Accident and Term life Insurance
• Complementary Health screening for 35 yrs. and above

Your key responsibilities

Core QA Responsibilities

• Design and maintain automation frameworks for UI, API, and backend testing.
• Develop test automation scripts using Selenium, Playwright, Cypress, or similar.
• Build API automation suites using RestAssured, Postman/Newman, etc.
• Perform test planning, execution, reporting, and defect lifecycle management.
• Implement automated tests in CI/CD pipelines (Jenkins, TeamCity, GitHub Actions).
• Perform root-cause analysis and support L3 production issues.

AI-Focused QA Responsibilities

• Collaborate with Data Science and ML teams to test AI/ML model behavior.
• Validate model outputs for accuracy, precision, recall, thresholds, and stability.
• Test AI-driven features: recommendations, NLP/chatbots, classification, anomaly detection, predictive insights, etc.
• Automate validation of model inference APIs, streaming outputs, and batch pipelines.
• Conduct data-quality testing for ML feature inputs (schema, drift, distribution checks).
• Validate model retraining, deployment workflows, and versioning pipelines.
• Test UI interfaces showing model predictions, confidence scores, and explainability outputs.
• Ensure fairness, bias detection, and responsible AI presentation in product workflows.
• Collaborate with MLOps to test end-to-end ML lifecycle.

Tools & Frameworks for AI Model Testing

• AI/ML Testing Frameworks
• Deepchecks – ML model validation, data integrity, drift checks
• DeepEval / TruEra / Arthur AI – model evaluation, quality monitoring, bias & fairness checks
• Evidently AI – automated monitoring for model drift, data drift, data quality
• Fiddler AI – model explainability, fairness, performance dashboards
• MLflow – model validation, experiment tracking, test comparison
• Great Expectations – data validation for ML pipelines

Data & Pipeline Testing

• PyTest + custom ML test harnesses
• Soda Data / Deequ – data quality checks for ML features
• Airflow/Kubeflow test modules – pipeline DAG testing

Model Inference / API Testing Tools

• Postman/Newman – testing ML inference endpoints
• RestAssured – API-based model validation
• Locust/JMeter – load testing of inference throughput & latency

Observability & Monitoring Tools

• Prometheus + Grafana – monitoring inference latency & performance
• Elastic Stack (ELK) – logging of model behavior & anomalies
• OpenTelemetry – tracing AI pipeline performance

Your skills and experience

Test automation using Selenium, Playwright, Cypress, or equivalent.

• Strong programming/scripting in Python, Java, or JavaScript.
• Framework design experience (POM, Hybrid, BDD, Data-Driven).
• Strong background in API testing.
• Experience with ML/AI testing frameworks (Evidently, Deepchecks, MLflow, Fiddler, GE).
• Understanding of ML workflows, model metrics, and data validation.
• Experience with cloud platforms and containers.

Soft Skills

• Strong analytical and debugging skills.
• Attention to detail and quality mindset.
• Ability to collaborate with engineering, AI, and product teams.
• Clear communication skills; ability to explain ML test findings.

How we’ll support you

• Training and development to help you excel in your career
• Coaching and support from experts in your team
• A culture of continuous learning to aid progression
• A range of flexible benefits that you can tailor to suit your needs

About us and our teams

Please visit our company website for further information:

https://www.db.com/company/company.html

We strive for a culture in which we are empowered to excel together every day. This includes acting responsibly, thinking commercially, taking initiative and working collaboratively.
Together we share and celebrate the successes of our people. Together we are Deutsche Bank Group.
We welcome applications from all people and promote a positive, fair and inclusive work environment.