To build a Multi-Agent System
Starting with GPT, the emergence of various LLM models has brought significant changes to work processes in the IT field. As the LLM field has evolved from simple Generative AI (ChatGPT, Gemini, etc.) toward Agentic AI (Cursor, ClaudeCode, Opencode, etc.), it feels as though the range of previously required skills has narrowed considerably.
In the past, skills like "proficiency in target languages, knowing how to use vulnerability analysis tools, reversing, and reporting" were crucial for vulnerability analysis. In the current LLM era, however, rather than needing a deep mastery of those skills as before, the scope seems to have shifted toward understanding concepts and methodologies, and being able to judge false positives by evaluating the outputs the AI produces.
In an age where specific parts of work can be replaced by AI, I believe the most important skill is the ability to "delegate" one's tasks to AI. To me, this ability to delegate feels like the capacity to build a system that creates "cloned personnel" capable of performing the tasks I need to do. Ultimately, the major differentiator will be whether or not one can personally build an AI Agent—the most active form of AI operation at this point—to create that cloned personnel.
Therefore, in this post, I will discuss "what it means to create an AI Agent, necessary elements, Multi-Agent Systems, and conclusions."
What is an AI Agent?
Let me briefly explain AI Agents by comparing them to Generative AI.
Generative AI refers to the chat-based AIs we commonly subscribe to and use, such as ChatGPT, Claude, and Gemini. Generative AI fundamentally serves the role of creating output based on the inputs we request. Every time we need to modify the results, we must input into the chat again, receive the result, and check it ourselves. In other words, it takes a form where the person works directly rather than the AI performing the task autonomously.
Conversely, with Agentic AI, once the system is established, it works on the requested task independently. If a problem arises in the code, it debugs, fixes, and applies the solution itself. The major difference between Generative AI and Agentic AI is that Agentic AI removes human intervention.
Now, let’s discuss the details involved in creating an Agent System that minimizes human intervention for requested tasks.
1. Narrowing the Domain
Think of the GPT, Claude, and Gemini we use as "General-Purpose LLMs." These are high-versatility LLMs designed to be used in any field or situation, rather than being specialized for one specific area. Looking only at the keyword "high versatility," one might think, "Isn't it better to have high versatility?"
It’s not bad, but let’s consider the inconveniences we face when using LLMs daily. The most representative case is "receiving different answers every time despite asking the same question." This happens because general-purpose LLMs lack a fixed reasoning guide specialized for that specific question. Since reasoning and conclusions are drawn based on default settings, the starting point for reasoning can vary for every question and session, increasing the likelihood of different answers for identical queries.
Ultimately, to specialize the AI for a desired task, one needs the skill to narrow down the LLM from a broad application domain to the specific target domain you want.
1-1. Regarding Fine-Tuning and System Prompts
To briefly explain Fine-Tuning:
When using GPT, Claude, or Gemini, you might have experienced a session where, after discussing a specific topic, the AI continues to relate its answers back to that original topic even if you move on to something else. It is easiest to think of Fine-Tuning as a methodology for creating a session for the LLM focused on a specific conversation topic. While Fine-Tuning techniques are still used today when necessary, using this methodology requires collecting "chat-result" datasets and incurs additional API costs to apply those datasets.
System Prompts are as follows:
You may have seen features like "Create my own [X]" in chatbot services like ChatGPT, Gemini, Grok, or Claude. Simply put, this feature allows you to set the AI to "answer according to my taste or purpose." Think of the requirements we input here as the "System Prompt." Unlike Fine-Tuning, System Prompts do not require separate datasets or API costs; they only require prompt content regarding guidelines.
In the days of GPT-3, the basic performance of Pre-Trained LLM models was lacking, so System Prompting didn't reach its full potential. However, current models in the GPT-5 era have sufficient basic performance for System Prompts to become truly effective, which is why the concept of "Agents" has begun to be properly utilized.
1-2. Agent
As seen in the documents above, there are methodologies for creating an Agent that performs specific tasks. In the case of Claude Code, the definition in the Agent.md file becomes the Agent. The Agents most commonly encountered in actual use are likely "Explore" and "Plan" Agents. Those who have used Claude Code, Cursor, or GitHub Copilot may have seen settings like "Plan" or "Build." These settings represent specifically designated Agents.
For example, if we use a "Plan" mode Agent, it will only perform the role of planning and nothing else. Think of the System Prompt as the tool that ensures the AI only performs these specific actions.
Plaintext
---
description: Reviews code for quality and best practices
mode: subagent
model: anthropic/claude-sonnet-4-20250514
temperature: 0.1
tools:
write: false
edit: false
bash: false
---
You are in code review mode. Focus on:
- Code quality and best practices
- Potential bugs and edge cases
- Performance implications
- Security considerations
Provide constructive feedback without making direct changes.
As shown in the Agent definition method in the opencode document, Agent development is completed by designing guidelines for the tasks the Agent must perform in the bottom section.
2. Multi-Agent System
Currently, rather than using a single Agent by selecting one for specific details of a requested task, we input the task into the system, and the system calls the necessary Agents for the sub-tasks involved, operating a Multi-Agent System. Simply put, Multi-Agent System operation is a framework where Agents specialized for tasks like "Front-end, Back-end, Design," etc., have been created and share their work with each other, operating like a single team. For this Multi-Agent System, orchestration is vital.
2-1. Orchestration
An Orchestration Agent, simply put, creates the role of a lead who assigns tasks to each department.
Before the release of Claude Code Teams, many evaluated opencode as having better performance than Claude Code due to the appearance of the oh-my-opencode plugin. The reason for this was the Sisyphus Orchestration Agent in oh-my-opencode. Sisyphus works roughly as follows:
-
Identify the received request.
-
Receive a plan from Prometheus (the Plan Agent).
-
Design and call the Agents required for each step based on the received plan.
Instead of handling the task with a single Agent, it designs and calls a FrontEnd Agent if a front-end developer is needed at a certain stage, a BackEnd Agent if back-end is needed, or a Code Review Agent if a review is needed. Because the Agents are designed and called specifically for their assigned tasks, confusion regarding the work is reduced. This structure appears to move exactly like a development "team" at a company.
Ultimately, while creating an Agent for a specific task is important, creating an Orchestration Agent that understands the requested work and distributes tasks well is also one of the key abilities for producing good results through AI. If you want to get ideas for orchestration design, you can look at the System Prompt definition of the Sisyphus Agent in the oh-my-opencode git mentioned above.
3. Methodologies for Improving Quality
Let’s discuss a few simple methodologies to make an Agent provide more accurate and higher-quality answers.
3-1. XML tags
One methodology for creating a higher-quality Agent is organizing the System Prompt using XML tags. When we write a System Prompt, even if we divide sections using major and sub-headings, there are cases where the AI fails to distinguish those section divisions. If these guideline sections are not properly separated, confusion can occur. Using XML tags is one way to properly segment tasks.
XML
<avoid_excessive_markdown_and_bullet_points>
When writing reports, documents, technical explanations, analyses, or any long-form content, write in clear, flowing prose using complete paragraphs and sentences. Use standard paragraph breaks for organization and reserve markdown primarily for `inline code`, code blocks (```...```), and simple headings (###, and ###). Avoid using **bold** and *italics*.
DO NOT use ordered lists (1. ...) or unordered lists (*) unless : a) you're presenting truly discrete items where a list format is the best option, or b) the user explicitly requests a list or ranking
Instead of listing items with bullets or numbers, incorporate them naturally into sentences. This guidance applies especially to technical writing. Using prose instead of excessive formatting will improve user satisfaction. NEVER output a series of overly short bullet points.
Your goal is readable, flowing text that guides the reader naturally through ideas rather than fragmenting information into isolated points.
</avoid_excessive_markdown_and_bullet_points>
As in the example above, using the <avoid_excessive_markdown_and_bullet_points> tag completely separates the guideline section and establishes the guide. You can view XML tags as a language used to set guides for the Agent. If you need a guide for Code Review prompting, you can separate it with something like a <Code_Review> tag.
3-2. Defining the Output Format
Defining the Output Format is a common topic in posts like "How to use AI smartly." This is important because of the "different answers to the same question" issue mentioned earlier. If the format of the code the AI produces is somewhat fixed, it becomes easier for a person to debug if they need to step in, regardless of the circumstances under which the code was generated. Additionally, having a set format makes it easier to share analysis results from one Agent to another. Since AI thinks and works arbitrarily every time, it is important to define the format for the final output as well as the reasoning process guidelines.
3-3. ETC.
Beyond these methods, there are also architectural concepts like RAG, MCP, and Skills. Applying all of these methodologies doesn't necessarily result in a good Agent System; you must apply only the concepts necessary for your specific work system. Otherwise, the Agent may become confused about which one to use, making the work inefficient and incurring high API costs.
4. So, do we not need to study anymore?
Looking at the content above, you might think, "So we don't have to study hard anymore?" However... you actually have to study even harder.
4-1. Hallucination
Hallucination is the biggest problem with LLMs. It is a phenomenon where the AI provides incorrect information as if it were correct. To detect this, we need the ability to judge whether the result is actually true, which requires building deep expertise in that work domain. Not only to identify hallucinations but also to build guidelines that prevent the Agent from producing them in the first place, only someone with expertise and solutions for that domain can minimize them. Thus, while we still study hard, the direction of that study has changed.
4-2. Personal Experience
Recently, I took the process I use for Vulnerability Research and applied it to a Multi-Agent System, creating and testing a system where 11 Agents operate organically. The application field was Web3 Smart Contract Audit. When I used a single Agent in a "do it for me" style like "find vulnerabilities," the results were mostly hallucinations. Once I built the system based on the principles mentioned above, hallucinations decreased significantly, and valid vulnerabilities began to appear.
However, I soon hit a limit. I do not have very deep knowledge in the Web3 Smart Contract Audit field. Because I was the one who built the system, I noticed that the Agent System only found the types of vulnerabilities that someone at my level would find. On the plus side, while it might take me two weeks to understand documents, identify the codebase, and analyze it for an audit, this system could complete the work within a few hours.
Ultimately, this experience taught me that to create a capable Agent System, the creator themselves must be capable in that field. So, it seems we still have to study more...
Closing
In this post, I have included only enough content for non-experts to read. I hope this post makes people interested in Agent development or facilitates their entry into it. I will return with a future post sharing the development process and results of the Agent System mentioned in my experience.