MakeSomethingNew Releases LLM Implementation Benchmark Results and Offers Free Prompt Engineering Tool

MakeSomethingNew Inc. announced that it has published research findings examining which models are useful when integrating LLMs into web services and business systems. The company has also begun offering a free prompt engineering tool that supports prompt design, evaluation, and improvement, which are key parts of LLM implementation. While the use of generative AI such as ChatGPT has expanded rapidly in recent years, companies integrating LLMs into their own web services or business systems cannot rely only on choosing a high-performance model. In real development environments, design decisions must consider multiple factors, including response accuracy, response speed, cost, stability, integration with existing systems, and prompt maintainability. MakeSomethingNew has supported enterprises through its AI agent development services, covering everything from AI concept planning and requirements definition to design, development, deployment, and operational support. Through the newly released research results and free tool, the company aims to provide more concrete support for the early evaluation and implementation process of bringing LLMs into real services. As generative AI becomes more widespread, many companies are beginning to use LLMs for new service development, operational efficiency, customer inquiries, internal knowledge search, and document creation support. However, when actually integrating LLMs into web services or business systems, companies often face challenges such as not knowing which LLM to choose, difficulty comparing model accuracy, speed, and cost, relying on individual experience to create prompts, being unable to manage prompt improvement history or evaluation results, unstable quality after moving from development environments into production services, and uncertainty over how to connect LLM outputs to business workflows or existing systems. LLM usage is shifting from the stage of trying models through chat to the stage of embedding them into business operations and services while continuously improving them. For that reason, model selection and prompt engineering need to be treated as part of the development process. In this study, MakeSomethingNew compared and evaluated major LLMs from multiple perspectives, assuming their integration into web services. The results showed that LLM implementation should not always rely on the highest-performance model. Instead, it is important to use different models depending on factors such as response time. For example, high-performance models are useful for complex decisions and high-precision text generation, while faster and more cost-effective models are more practical for routine classification, summarization, information extraction, and simple inquiry handling. For web service integration, systems also need to be designed for continuous improvement, not just one-time response accuracy. This includes response speed, retry design for failures, prompt version control, log analysis, and the accumulation of evaluation data. The free prompt engineering tool supports prompt creation, comparison, and improvement when integrating LLMs into web services and business systems. Its main features include creating and saving prompts, comparing multiple prompts, checking outputs by model, managing prompt improvement history, and batch checking with web service integration in mind. This enables developers and business teams to move away from individually managed prompts and instead validate and improve LLM implementations collaboratively. In addition to the research results and free tool, MakeSomethingNew supports several areas of enterprise LLM adoption, including LLM-powered web service development, adding LLM features to existing services, AI agent design and development, integration with internal data and business systems, RAG architecture design and implementation, prompt design and evaluation improvement, model selection and cost optimization, and support from PoC through production deployment. The company says it will help businesses move generative AI beyond experimentation and embed it into real services and business workflows as a system that continuously creates value. Going forward, MakeSomethingNew will further strengthen its support for LLM-based web service development, AI agent development, business system integration, and prompt engineering. It will also continue publishing industry- and task-specific LLM templates and information on model selection and prompt evaluation to help companies adopt generative AI safely and effectively. MakeSomethingNew Inc. is led by Representative Director Eiichi Sugiyama. Its business includes AI agent development support, AI adoption support, LLM implementation support, and system planning and development support. The company website is https://makesomethingnew.jp/ . Inquiries can be submitted through the service page or the company contact form.