Developing Generative AI Technology for Low-Memory RAG Execution on In-Vehicle Edge Devices

Mechanism and Application Image of Generative AI Technology for Low-Memory RAG Execution

Denso Ten (Headquarters: Kobe City, Hyogo Prefecture, Representative Director and President: Yoshinori Yonemoto) has developed generative AI technology that enables low-memory execution of RAG (Retrieval-Augmented Generation: Retrieval-Augmented Generation) on in-vehicle edge devices. This technology is characterized by its ability to lightweight search vector databases through re-learning of embedded models, which is essential for improving the accuracy of generative AI, and to implement them while maintaining search accuracy even in in-vehicle SoC environments. It is envisioned to be applied as an HMI that caters to individuals through dialogue assistants using generative AI (LLM). By enabling the implementation of generative AI on in-vehicle edge devices, we contribute to the realization of a natural and reassuring dialogue experience tailored to each driver and passenger. We will continue to collaborate with automakers and partner companies to contribute to the development of mobility society.

Development Background

The use of generative AI (LLM) is advancing in areas such as smart cockpits and smart cabins in in-vehicle HMI (for example, setting destinations and navigation through dialogue assistants, searching and proposing information on news/entertainment content, and controlling vehicle functions). On the other hand, there is a challenge that generative AI (LLM) alone cannot accurately respond to the latest information or vehicle/user-specific information that it has not learned. RAG has attracted attention as a technology to compensate for this, but it has been difficult to implement in in-vehicle SoC and edge environments with constraints on memory capacity and processing performance because it requires embedded models and large search vector databases. In other words, there is a trade-off where 'accuracy makes it heavy' and 'lightweighting reduces accuracy'.

Technical Features

・Achieving lightweight and high-precision vector databases for in-vehicle edge execution through unique embedded model learning technology
・In evaluations using public datasets, both memory reduction and search accuracy are achieved (reducing memory capacity by 30-60% while maintaining high accuracy compared to existing models)

Comment from Professor Takiguchi, Graduate School of Systems Information Science, Kobe University (Voice Dialogue)

To realize a dialogue system that caters to each individual, responses that take into account the intentions and preferences of each user are required. On the other hand, since such information is highly privacy-sensitive, local operation that does not depend on the Internet is desired. From this perspective, the importance of technology to realize a dialogue system using RAG on in-vehicle edge devices will further increase in the future.

Under the corporate vision 'VISION2030', the Denso Ten Group is working to enhance the appeal of cars through 'HMI that caters to individuals', 'environmentally friendly electrification', and 'data linkage between cars and society'. We aim to solve mobility society issues such as zero traffic accidents and carbon neutrality, and contribute to enriching people's lives by solving various mobility-related issues. We will continue to work towards the realization of an 'environmentally friendly' and 'safe and secure' mobility society.

※1 RAG (Retrieval-Augmented Generation: Retrieval-Augmented Generation): A technology where generative AI (LLM) searches for related information from external data sources and generates responses based on that information.
※2 Embedded Model: An AI model that converts text, images, etc. into arrays of numbers (feature vectors).
※3 Vector Database: A database that stores text, images, etc. as arrays of numbers (feature vectors) and performs high-speed and high-accuracy searches based on the 'similarity' between the content to be searched and the aforementioned data.
※4 LLM (Large Language Model: Large Language Model): An AI model trained using vast amounts of text data and deep learning (deep learning).
※5 This learning technology was presented at the '32nd Annual Meeting of the Linguistic Society Matryoshka Representation Learning Considering Embedded Model Distillation'.
(To be held at Light Cube Utsunomiya from March 9 to 13, 2026) https://anlp.jp/nlp2026/

Through this initiative, we aim to achieve the following SDGs.
Keywords: