AI QA Specialist

Role Overview

Dry Ground AI helps companies become AI native by building custom software applications, AI agents, and automation workflows that operate in production environments.

We are seeking a full time AI QA Specialist responsible for validating system quality across applications, interfaces, AI behavior, data, and workflows before client review and production release.

Most of our QA is already automated through pipelines and AI evaluation agents. This role focuses on the final human validation layer, ensuring accuracy, consistency, and completeness before client demos and production releases.

This is not a narrow testing role. You will be responsible for QA across multiple system types and ensuring nothing is missed.

Key Responsibilities

• Test custom software applications and user interfaces for functionality and usability
• Validate AI agents and conversational flows for correctness, consistency, and adherence to defined rules
• Review AI agent evaluation results and validate edge cases that automated tests may not cover
• Test automation workflows, integrations, and multi step system behavior
• Verify data accuracy and system outputs across different scenarios
• Identify edge cases, failure modes, and unexpected behavior across systems
• Review outputs from automated QA processes and flag gaps or inconsistencies
• Document bugs with clear reproduction steps, expected versus actual results, and severity
• Maintain structured QA documentation, test cases, and release checklists
• Ensure all systems meet QA standards before demos and production releases

What We’re Looking For

• Experience in QA or software testing across applications or systems
• Strong attention to detail and organizational discipline
• Ability to think in systems and workflows, not just individual screens
• Clear written communication and structured documentation skills
• Comfortable working independently and managing multiple priorities

Nice to Have

• Experience with AI systems, chatbots, or automation tools
• Familiarity with APIs, integrations, or backend workflows
• Experience with QA tools or test management systems
• Familiarity with LangSmith, LangFuse, or similar tracing tools for reviewing AI agent behavior and outputs
• Experience with AWS

Working Model

• Works across multiple projects and release cycles
• Collaborates closely with engineering and product teams

Important Note

This role requires consistency, precision, and ownership of QA standards across all systems. Success is defined by catching issues before client exposure and maintaining clear, structured QA processes.