New Quality Assessment Service for AI Agents Launched

VeriServe Co., Ltd. (Headquarters: Chiyoda-ku, Tokyo; President and CEO: Tadahiro Shigihara; hereinafter 'VeriServe'), a provider of services supporting software quality improvement, has launched a new service today called 'QA4AI (Q-A-for-AI) Agent,' which evaluates the quality of all types of AI agents※1.

This service evaluates the quality of AI agents—including not only output results but also behaviors such as decision-making processes and tool usage—from a third-party standpoint, based on evaluation perspectives and evaluation programs※2.

※1 AI that autonomously performs tasks or business operations by connecting with external tools or data

※2 A framework for evaluating AI agent quality based on evaluation metrics, scoring methods, and judgment criteria

Figure 1: Evolution toward evaluating AI agents including their 'behavioral processes'

■ Background

The use of generative AI is expanding from simple chat-based output generation to AI agents capable of autonomously executing human-like business tasks. However, AI agents have the characteristic of autonomously breaking down and executing multiple tasks, selecting next actions based on intermediate results. This makes it difficult to fully ensure quality unless evaluation includes not only final outputs but also task decomposition, execution processes, and the validity of decisions—highlighting a growing challenge.

Additionally, enterprises considering AI agent adoption or production deployment face issues such as 'not knowing how to evaluate quality' or 'inability to properly assess the impact of changes.'

To address these challenges, VeriServe systematizes evaluation perspectives specific to AI agents and provides end-to-end support—from applying these perspectives to test execution—enabling objective quality assessment of any AI agent and supporting enterprises in confidently moving from adoption to production use (Figure 1).

■ Service Overview

'QA4AI Agent' is a new service targeting companies developing or planning to adopt AI agents, enabling continuous quality assessment before deployment, prior to production use, and during model or configuration changes.

VeriServe acts as a third party, taking responsibility from evaluation design through execution, and assesses AI agent quality based on objective criteria.

【Main Support Offerings】

- Quality assessment and visualization prior to production use

- Verification to identify and mitigate risks

- Evaluation of quality impact due to software changes from specification updates

- Provision of evaluation results necessary for deployment decisions

【Main Implementation Activities】

- Current state analysis and scope definition

- Organization of evaluation perspectives

- Design of evaluation metrics

- Dataset design

- Implementation of evaluation scripts, execution, and reporting of results

Figure 2: Quality evaluation of AI agents

■ Key Features

1. Quality evaluation including behavioral aspects

While traditional AI evaluation has focused primarily on output accuracy, this service evaluates the overall behavior of AI agents from the following perspectives (Figure 2):

- Whether intended deliverables are being produced

- Whether tasks are being completed appropriately

- Whether tool usage is appropriate

- Whether unauthorized data access is avoided

- Whether stable responses are provided even to unexpected inputs

- Whether there are no safety or compliance issues

2. Systematized evaluation perspectives and use of evaluation tools

VeriServe has systematized evaluation perspectives specific to AI agents and developed evaluation tools based on these perspectives.

This enables consistent quality evaluation across different outputs, allowing quality to be assessed and compared continuously over time—even after software modifications.

3. Third-party quality evaluation

VeriServe possesses extensive knowledge in software quality improvement, built through years of R&D and practical experience across diverse industries. By leveraging AI-agent-specific evaluation perspectives and programs, VeriServe supports the detection of risks that developers might overlook, from an independent third-party standpoint.

■ Specific Use Cases

(1) Companies developing AI agents

By incorporating third-party quality evaluation during development, companies can visualize risks before deployment and proceed confidently to production use.

- Verify behavior under unexpected inputs or complex scenarios before production use

- Evaluate quality impact when changing models, prompts, or adding tools

- Confirm not only output correctness but also the appropriateness of tool usage and decision-making

(2) Companies adopting AI agents

By objectively evaluating quality before adoption, companies can reduce risks in business application and make confident deployment decisions.

- Conduct verification in real-world-like conditions based on business scenarios

- Identify incorrect responses, inappropriate behaviors, and risks

- Evaluate suitability against internal business requirements

- Provide decision-making materials for adoption approval and scope definition

■ Future Initiatives

VeriServe will continue advancing quality assurance solutions for evolving AI agents. By enhancing evaluation methodologies and expanding evaluation perspectives and programs, the company aims to realize a new form of quality assurance through 'humans × technology × AI,' contributing to safe and secure software development for its customers.

■ About VeriServe Co., Ltd.

Established: July 24, 2001

Representative: President and CEO Tadahiro Shigihara

Headquarters: Jimbocho Kitatokyu Building, 3-1-16 Kanda Misakicho, Chiyoda-ku, Tokyo

Business: Software services

1. Software testing and quality-related services

2. Cybersecurity-related services

3. Consulting-related services

4. Software development-related services

5. Other services

URL: https://www.veriserve.co.jp/

【Service Inquiry】

https://www.veriserve.co.jp/contact/

【Press Inquiry】

Public Relations Department, Sato, Ota

TEL: 050-3640-8194

MAIL: press@veriserve.co.jp

*Product names, company names, and service names listed are trademarks or registered trademarks of their respective companies.