An AI Safety Foundation to Support Safe Collaboration Between Humans and AI Has Been Established

In a project promoted by NEDO, titled "Research, Development, and Verification for Ensuring AI Safety / R&D for Strengthening AI Safety" (hereinafter, this project), five parties—NEDO, the National Institute of Advanced Industrial Science and Technology (AIST), Citadel AI Inc., Corpy & Co., Inc., and the University of the Ryukyus—have developed, formulated, and publicly released guidelines and evaluation protocols that will serve as a common foundation for ensuring the safety of AI systems.

The guidelines formulated in this project focus on ensuring safety from the planning and design stages of AI systems to their evaluation and operation. They organize the concepts and basic procedures for businesses that develop and introduce AI-powered systems to identify risks and consider appropriate countermeasures.

By widely utilizing these guidelines and evaluation protocols, the aim is to ingrain a common understanding and procedure for AI system safety in society, thereby accelerating the development of a common foundation for AI safety that allows for the confident use of AI.

1. Background

Against the backdrop of initiatives like the Hiroshima AI Process, launched at the 2023 G7 Hiroshima Summit, discussions and system development concerning AI safety are advancing in various countries. In light of these international circumstances, Japan has also established the AI Safety Institute (AISI)*1 to participate in international discussions.

This project*2 supports public-private initiatives from a research and development perspective to promote international rule-making for the safe and secure use of generative AI within Japan. As instances of humans and AI collaborating in judgment and action increase, there is a growing need to address the common challenge of how to design, evaluate, and operate AI safety.

In this project, R&D was conducted to establish a common foundation for evaluating and operating AI safety. While AI implementation technologies and application fields are diverse, the challenges of how to safely design situations where humans and AI interact, and how to ensure safety through judgment, verification, and operation, are common across fields.

As shown in Figure 1, this project is structured to develop evaluation and management technologies that serve as a "yardstick" for safety (R&D Item 1), develop AI safety evaluation and implementation technologies for specific application areas assuming real-world environments (R&D Item 2), and organize and systematize these results into a form usable in corporate practice, leading to the formulation of guidelines for implementing AI safety (R&D Item 3).

This project does not aim to demonstrate the completion of social implementation for any specific AI, but rather to present a common foundation that will lead to future technological development, demonstration, and standardization.

2. Recent Achievements

This project has broadly developed guidelines, evaluation methods, templates, and evaluation environments that span the "design, evaluation, and operation" stages to address the diverse challenges of AI safety.

(1) Formulation of Multimodal AI Quality Management Guidelines (R&D Item 3)

A core achievement of this project is AIST's formulation of guidelines that organize quality management perspectives and processes for multimodal AI*3, which receives images and text and primarily responds with text. As a basic evaluation perspective unique to multimodal AI, it focuses on cross-modal coreference capability*4, classifying this ability into four levels. To ensure the safety and quality of multimodal AI systems, it is particularly important to determine the required level of cross-modal coreference capability, and the guidelines systematically organize the responses to be implemented at each stage of the lifecycle according to that level.

The guidelines also feature three case studies—automatic caption generation from images, image-based diagnosis of infrastructure deterioration, and content moderation on social media—highlighting points to consider and quality management issues in situations involving human judgment and supervision.

These guidelines present a common design and evaluation framework for ensuring safety and quality based on the characteristics of multimodal AI, serving as a base for the practical application of AI safety.

(2) Formulation of Guidelines and Case Studies to Support AI Social Implementation in Corporate Settings (R&D Item 3)

Citadel AI, through interviews with companies actually developing and operating generative AI and AI agents, has organized evaluation perspectives, standards, and methods to translate AI safety principles and guidelines into a form that can be implemented and operated on-site. It has extracted common patterns and practical know-how from technological, process, and organizational culture perspectives and systematized them as a "Generative AI Practical Guide and Corporate Case Studies."

To verify and concretize the extracted know-how for developers, a chatbot was implemented and released, targeting users who require special consideration in understanding language and systems, such as foreigners, focusing on information related to administrative procedures. This chatbot utilizes information useful for daily life issued and published by local governments to clearly present the generative AI's answers and their sources.

These achievements present the "ideal state" shown in the guidelines as concrete evaluation and operational methods for companies to implement in practice, playing a role in making AI safety applicable at a practical level.

(3) Development of Implementation Guides and Evaluation Templates Connecting Organizational Management and Technical Evaluation (R&D Item 3)

Corpy & Co. has developed an "AI Management System-based Generative AI Safety Evaluation Protocol and its Implementation Guide" and evaluation templates, consisting of three phases—analysis, testing, and reporting—for practically implementing generative AI safety evaluations in line with ISO/IEC*5 42001 (AI Management System Standard)*6.

Using a customer support system with a visual language model*7 as a case study, it conducted evaluations through methods like red teaming*8 and examined their effectiveness and practical issues.

These achievements provide a practical framework connecting organizational management requirements with technical safety evaluations, serving as a foundation for companies to consistently implement AI safety.

The following two points are achievements that lead to concrete ideas and evaluation/verification methods for ensuring safety in human-AI collaboration, assuming AI utilization in real-world environments such as healthcare and daily life.

(4) Proposal of Safety Assurance Methods for Decision-Making Processes in Human-AI Teaming (R&D Item 1)

AIST and the University of the Ryukyus jointly conducted a study on safety in Human-AI Teaming, where humans and AI collaborate on judgments, assuming a medical setting. This initiative materializes the idea of "quality requirements according to use and usage scenarios" presented in the multimodal AI guidelines.

Specifically, they analyzed what process should be followed for final judgment when AI and a doctor's diagnoses of medical images do not match, organizing ways of consensus-building and risk-avoidance measures.

Through joint discussions with doctors, they organized the additional information and judgment rationale that AI should present, and the nature of interfaces that support decision-making, systematizing them as design guidelines for ensuring the safety of decision-making processes in human-AI collaboration.

They also confirmed that in relationships where humans have the final decision-making authority, humans may not be able to correctly perceive the AI's accuracy, and that this perception can affect the utilization of the AI.

This achievement organizes the concept of ensuring safety on the premise of situations where human and AI judgments do not match, contributing to the practical design of AI safety in Human-AI Teaming.

(5) Construction of an AI Safety Evaluation and Verification Platform for Daily Life Domains (R&D Item 2)

AIST has worked on building an AI safety evaluation and verification platform to promote the safe use of AI systems in daily life environments such as homes and nursing care facilities.

AI systems for daily life face challenges: data for development and verification is difficult to obtain due to the diversity of life situations and privacy protection concerns, and evaluation methods for safety and robustness*9 are not well-established.

In this initiative, targeting monitoring AI systems, they organized anticipated behaviors and events from a human safety perspective as scenarios and developed a real-virtual fusion environment that enables the collection, generation, and verification of data necessary for AI safety evaluation.

By combining actual behavior data acquired in physical living labs built in the real world with data augmentation technology in a cyber living lab in a virtual space, they have constructed a dataset that contributes to the safety evaluation of monitoring AI, including behaviors that are difficult to measure in reality, such as falls and stumbles.

Furthermore, looking ahead to the social implementation of monitoring AI, the "Human-centric AI Lifetech Consortium (HAIL)" was launched in April 2026 as a forum to discuss technical and social issues related to safety evaluation and verification, with activities starting in June. This dataset is scheduled to be used in the consortium.

This achievement establishes a platform that makes it possible to verify the safety of AI in the daily life domain, which has been difficult to evaluate, based on the characteristics of the real environment, contributing to the advancement of practical evaluation and verification of AI safety.

3. Future Plans

This project has yielded a variety of results, including the formulation of AI safety guidelines as a core, as well as implementation methodologies for corporate settings, practical methods linking organizational management and evaluation, technical knowledge on human-AI collaboration, and verification platforms for real-world environments.

A key feature is that these results are positioned as elements for a multifaceted approach to the diverse challenges of AI safety at each stage of "design, evaluation, and operation," laying the groundwork for broadly realizing countermeasures suited to expanding AI use cases.

In the future, we plan to systematically organize the common understanding of AI safety obtained in this project, along with specific evaluation and implementation methods, into a practical common foundation for AI safety for a society where humans and AI collaborate.

This release highlights representative achievements of the project. Many other results not mentioned here have been obtained in each research item shown in Figure 1, and these are introduced on a dedicated webpage.