How Is the Rise of LLMs Changing the Security Industry?

Subtitle: What GPT-5.5-Cyber and Claude Mythos Preview Show Us, and How We Should Prepare

Written on: May 14, 2026

English translation assisted by GPT-5.5.

1. Why I Am Writing This

LLMs are advancing rapidly.

Tasks that once required years of study and hands-on experience, such as code analysis, reverse engineering, vulnerability discovery, PoC verification, and exploitability assessment, can now be completed much faster with the help of LLMs.

In particular, OpenAI’s GPT-5.5-Cyber and Anthropic’s Claude Mythos Preview have created a strong ripple effect across the security industry. These models go beyond simply explaining code or identifying the name of a vulnerability. They can find real vulnerability candidates, analyze attack paths, and, in limited evaluation or research environments, even demonstrate parts of exploit construction.

This naturally leads to an important question.

Are LLMs tools that improve the productivity of security professionals, or are they a threat that may replace existing security talent?

This article does not try to force one answer over the other. Within the TOOR team, we reviewed the public materials on GPT-5.5-Cyber and Claude Mythos Preview, then organized both the optimism and anxiety that LLMs are bringing to the security industry. We avoided simple conclusions such as “AI will completely replace security professionals” or “AI is only a basic assistant.” Instead, we focused on calmly examining the direction of change shown by the publicly available materials.

There is also one point that needs to be made clear from the beginning.

The performance of GPT-5.5-Cyber and Claude Mythos Preview does not mean that “anyone with enough credits can instantly gain elite hacker-level abilities.” GPT-5.5-Cyber is provided as a limited preview for vetted defenders and authorized security workflows. Claude Mythos Preview is also not a generally available public model.[1][3]

In this article, these two models are treated not as “offensive capabilities immediately available to everyone,” but as signals showing how far frontier models can accelerate security work.

And that signal is not small. LLMs can help defenders find and patch vulnerabilities faster. At the same time, they may also lower the cost for attackers to discover vulnerabilities, analyze patches, assess exploitability, and explore automated attack paths. The same technology is accelerating both offense and defense.

This article looks at three perspectives.

1. How should we interpret the publicly disclosed security capabilities of GPT-5.5-Cyber and Claude Mythos Preview?
2. Based on what we are seeing now, how can we predict the future of the security industry?
3. If that future arrives, what capabilities should we prepare?

2. How Should We Understand GPT-5.5-Cyber and Mythos?

2.1 GPT-5.5-Cyber Is Closer to a “More Permissive Limited-Access Model” Than a “Stronger Hacking Model”

OpenAI’s GPT-5.5-Cyber should be understood less as a separate model with dramatically stronger cybersecurity capabilities than GPT-5.5, and more as a limited preview designed to behave more permissively for verified defenders and authorized security workflows.

In its explanation of Trusted Access for Cyber, OpenAI stated that the model is intended to avoid unnecessary refusals in legitimate defensive tasks such as vulnerability identification, malware analysis, binary reverse engineering, detection engineering, and patch validation. At the same time, OpenAI explains that malicious activities such as credential theft, stealth, persistence, malware deployment, and exploitation of third-party systems remain blocked.[1]

One sentence is especially important.

Early previews of cyber-permissive models such as GPT-5.5-Cyber are not primarily intended to significantly increase cyber capability beyond GPT-5.5. Instead, they are mainly trained to behave more permissively on security-related tasks.[1]

For that reason, the significance of GPT-5.5-Cyber is not the arrival of a “superpowered hacking model.” It is closer to an access-control experiment for using the existing capabilities of frontier models more directly in verified defensive security work.

This distinction matters. If we fail to separate an increase in capability from a decrease in refusal rate for existing capability, we risk exaggerating what GPT-5.5-Cyber represents. Based only on the currently public materials, GPT-5.5-Cyber appears to be less about “new offensive capability” and more about “an access policy that allows strong models to be used more effectively for defensive security work.”

2.2 The Numbers Are Strong, but the Evaluation Conditions Matter

The UK AI Security Institute published its evaluation results for GPT-5.5’s cyber capabilities.

According to AISI’s GPT-5.5 evaluation post, at the Expert level of advanced cyber tasks with a 50M token budget, GPT-5.5 achieved an average success rate of 71.4% ± 8.0, while Claude Mythos Preview achieved 68.6% ± 8.7. The evaluation included tasks close to real security work, such as reverse engineering, web exploitation, cryptanalysis, and vulnerability research.[2]

If we only look at the numbers, the results can feel shocking. However, the conditions need to be separated clearly.

In a separate evaluation post on Claude Mythos Preview, AISI states that Mythos Preview achieved a 73% success rate on expert-level CTF tasks. Rather than treating this as a contradiction with the earlier 68.6% figure, it is safer to understand it as a result from a different evaluation post, task composition, and comparison context.[4]

AISI also reported that Mythos Preview solved The Last Ones, a 32-step enterprise network attack simulation, from start to finish in 3 out of 10 attempts. Across all attempts, it completed an average of 22 out of 32 steps.[4]

That is an impressive result. However, it should not be directly generalized into real-world enterprise attack capability.

AISI explains that the evaluation was conducted in a controlled environment where the model was explicitly instructed to carry out an attack and was given network access. The Last Ones did not include an active defender or defensive tooling, and there was no separate penalty even if the model triggered security alerts. For these reasons, AISI cautioned that the results do not prove Mythos Preview could attack well-defended systems in the same way.[4]

The conclusion we can draw from the currently public AISI evaluations is roughly this:

Modern frontier models are showing near-human-expert performance on some difficult security tasks. Vulnerability discovery, code analysis, reverse engineering, exploitability assessment, and attack-path exploration are becoming increasingly automated. However, this performance must be understood in the context of controlled evaluation environments, high token budgets, tool-use scaffolding, and limited-access models. The claim that “AI will immediately replace all security professionals” is an exaggeration. But the claim that “the way security work is done will change significantly” is realistic.

2.3 Claude Mythos Preview’s Zero-Day Claims Should Be Read with the Type of Source in Mind

According to Anthropic’s own red-team report, Claude Mythos Preview identified undisclosed vulnerabilities in major operating systems and web browsers under limited evaluation and research conditions. In some cases, it also progressed to exploit construction.[3]

In particular, Anthropic reported that Mythos Preview showed much stronger exploit-construction ability than Claude Opus 4.6 on a Firefox 147 JavaScript engine benchmark. According to Anthropic, Opus 4.6 produced a working exploit 2 times across hundreds of attempts, while Mythos Preview produced 181 working exploits and achieved register control 29 times.[3]

These are strong numbers. However, this result comes from Anthropic’s own red-team report. Many of the vulnerabilities had not yet been patched, so detailed information was not publicly available. Anthropic also stated that more than 99% of the vulnerabilities it found were still unpatched, meaning the details could not yet be disclosed.[3]

This part is best read as follows:

Mythos Preview demonstrates real vulnerability discovery and exploit-construction ability in limited research environments. However, many of the results are not yet fully reproducible public benchmarks. Anthropic’s own report, AISI’s independent evaluation, and the Mozilla collaboration case should be read separately. Still, it is clear that multiple sources point in the same direction. The security-related reasoning, code analysis, and tool-use capabilities of frontier models are improving quickly.

2.4 The Firefox Case Shows the Potential for Defensive Use

The collaboration between Anthropic and Mozilla on Firefox shows that LLMs can contribute to security improvements in large real-world software projects.

Anthropic reported that Claude Opus 4.6 found and reported a Use-After-Free vulnerability in Firefox’s JavaScript engine after about 20 minutes of exploration, and that researchers later verified it. Over two weeks, Opus 4.6 found 22 vulnerabilities in Firefox, 14 of which were classified by Mozilla as high-severity vulnerabilities.[5]

Mozilla also stated that its collaboration with Opus 4.6 led to fixes for 22 security-sensitive bugs in Firefox 148, and that early testing with Mythos Preview helped fix 271 vulnerabilities in Firefox 150.[6]

This case is optimistic. LLMs can help defenders find and remove vulnerabilities faster than before. In complex and long-lived codebases such as browsers, a workflow where an LLM finds new vulnerability candidates, human maintainers verify them, and the findings lead to patches can create a major productivity boost for defenders.

Mozilla also pointed out an important limitation. It stated that it had not yet seen vulnerabilities found by AI that would have been beyond the reach of elite human researchers.[6]

This does not mean that AI-discovered vulnerabilities are meaningless. Rather, it suggests that, so far, AI is finding vulnerabilities that are still within the range of human researchers’ understanding, but it is doing so much faster and at a much broader scale.

The lesson from the Firefox case is clear. AI can find meaningful vulnerabilities in large real-world software projects. It can also greatly increase the speed of vulnerability discovery for defenders. However, it is still difficult to say that AI has opened an entirely new world of vulnerabilities beyond human researchers’ understanding. Verification and interpretation still remain human responsibilities.

3. Optimistic View: LLMs Make Security Professionals Stronger

The core of the optimistic view is simple.

Improvements in the security capabilities of LLMs are more likely to amplify the abilities of security professionals than to replace them.

Security is not a field where being good at just one technology is enough. Operating systems, networks, computer architecture, the web, browsers, compilers, cryptography, programming languages, and system design are all connected. One of our team members described hacking as “a comprehensive applied art of the IT field.”

From this perspective, it is natural that as LLMs improve their broad CS knowledge and reasoning ability, their security capabilities also improve.

LLMs can remember a massive amount of knowledge, connect ideas quickly, and carry out repeated experiments without getting tired. These traits fit well with vulnerability analysis and exploitability verification.

LLMs become especially powerful tools for people who already have expertise. Experts can identify which of the model’s hypotheses are meaningful, verify incorrect analysis, and adapt the results to real environments. In this case, the LLM is not just a simple automation tool. It is closer to an assistant researcher that can multiply the speed of research.

The Big Sleep case from Google Project Zero and Google DeepMind points in the same direction. Big Sleep found an exploitable stack buffer underflow in SQLite, and the issue was fixed before the official release, so users were not affected.[9]

This case is meaningful because it shows that AI is not only a tool for attackers. It can also be used by defenders to find and remove vulnerabilities before software is released.

From an optimistic perspective, LLMs accelerate vulnerability analysis, reduce repetitive code exploration and documentation work, and lower the cost of trial and error in CTFs, reverse engineering, and vulnerability research. Security professionals can focus on more complex and creative problems. The productivity of existing experts can increase significantly, and defenders gain more opportunities to find and remove vulnerabilities before deployment.

So the important question does not end with “Are LLMs good at security?”

The more important question is how well security professionals can use LLMs, and how accurately they can verify the results.

4. Pessimistic View: The Speed Gap Between Attack and Defense May Grow Wider

There were also many pessimistic opinions.

The core concern was that LLM capabilities are improving too quickly.

Models such as GPT-5.5-Cyber and Mythos can significantly accelerate vulnerability discovery, reverse engineering, malware analysis, PoC verification, and exploitability assessment. It would be ideal if these capabilities were available only to defenders, but that is not how reality works. Attackers will also use the same technology in similar ways.

The problem is that offense and defense do not move at the same speed.

AI may make it possible to discover vulnerabilities faster. However, the process of fixing vulnerabilities in real companies or open-source projects remains slow. Teams need to reproduce the issue, analyze the impact, create a patch, test it, deploy it, and notify customers. This process still depends heavily on the capabilities of organizations and people.

In other words, AI may speed up vulnerability discovery, but patching and response may not become faster at the same rate.

If that happens, security may not simply improve. Instead, more vulnerabilities may be exposed more quickly.

The UK NCSC also assesses that frontier AI may make vulnerability discovery and exploitation easier, faster, and cheaper, increasing patch pressure on organizations.[7]

Fortinet’s 2026 Global Threat Landscape Report, based on telemetry from FortiGuard Labs, also analyzes that the time-to-exploit for critical outbreaks has shortened to around 24 to 48 hours, and that AI-enabled offensive tooling is increasing the speed and accessibility of attackers’ workflows.[8]

Of course, Fortinet is a security vendor, and these figures are based on Fortinet’s own visibility and analytical framework. Rather than treating them as an absolute rule for the entire industry, it is safer to read them as one observation that attack speed is increasing. Still, NCSC and Fortinet point in a similar direction. The cost of vulnerability discovery and exploitation is decreasing, and the patch pressure on defenders is increasing.

LLMs also lower the barrier to entry for attackers.

In the past, advanced vulnerability analysis or exploitability assessment required years of experience. High-performance LLMs can compensate for a user’s lack of knowledge, explain the analysis process, and automate experiments. This does not mean that “anyone can hack any company.” Still, it is clearly a risk that the barriers of time, cost, and skill required for attacks are becoming lower.

The following tasks, in particular, are likely to be automated quickly:

- Repetitive web vulnerability testing
- CVE-based PoC reproduction and patch impact analysis
- Basic reverse engineering and crash triage
- Summarizing scanner results
- Drafting reports
- Simple log analysis
- Standardized penetration testing procedures

This change may have an especially strong impact on entry-level workers in the security industry.

Traditionally, junior security professionals gained experience and grew by performing repetitive analysis tasks. If those tasks are replaced by AI, the opportunities to build real-world experience may shrink.

From a pessimistic perspective, LLMs increase the speed of attack automation, cause a surge in vulnerability reports and analysis results, and also increase invalid reports and noise. Security roles centered on repetitive work will face pressure quickly, and the growth path for junior security professionals may become weaker.

The arrival of GPT-5.5-Cyber and Mythos is technically impressive, but it may also become a major restructuring force across the entire security industry.

5. The Future of the Security Industry Based on What We Are Seeing Now

Our team’s opinions were broadly divided into two directions.

One was a conditional optimism that “security will become even more important.” The other was a pessimistic view that “the structure of the security workforce will be significantly disrupted.”

5.1 The Center of Security Will Shift from Vulnerability Discovery to Management and Verification

From the optimistic perspective, the value of the security industry will not disappear. However, its center of gravity will change.

In the future, AI will find more vulnerability candidates at a much faster pace.

But finding vulnerability candidates and judging real security risk are different problems.

From a company’s perspective, the important question is not simply, “Does a vulnerability exist?”

- Is this vulnerability actually exploitable in our product?
- Can it be reproduced in our production environment?
- How much impact does it have on customers?
- Could patching it cause service disruptions?
- How far does the impact extend from a supply-chain perspective?
- How should legal and ethical responsibilities be handled?
- How should this be explained to executives and customers?

AI can suggest technical candidates. However, connecting those results to a company’s real-world risks, deciding priorities, and turning them into a response strategy still requires human judgment.

For that reason, security work is likely to move away from simple vulnerability discovery and toward the following roles: verifying vulnerability candidates found by AI, assessing their real-world impact, and prioritizing patches and mitigation measures. Designing AI-based security automation workflows will also become important, along with evaluating the reliability and security of LLM-generated outputs and connecting attack results to detection rules, patch strategies, and threat modeling.

From this perspective, security professionals will not disappear. Instead, they will be expected to make higher-level judgments.

Rather than people who simply find vulnerabilities, those who can use AI and automation systems to manage and operate security problems will become more important.

5.2 Junior Roles and Repetitive Security Work Are Under Threat

From the pessimistic perspective, the security industry may go through a transformation similar to the Industrial Revolution.

After the Industrial Revolution, the introduction of machines increased productivity, but the roles of some workers changed significantly. The value of manual work itself decreased, while the value of operating, managing, and designing machines increased.

AI may change the security industry in a similar way.

Repetitive and standardized tasks will be automated quickly by AI. Log analysis, alert classification, summarizing vulnerability scan results, report writing, and basic code analysis are increasingly likely to be handled by AI.

From a company’s perspective, it may become more efficient to have a small number of highly skilled people operate multiple AI agents than to hire many junior analysts for repetitive tasks.

Security workers will not disappear completely. However, the number of people needed and the roles they perform may change significantly.

What is especially dangerous is the weakening of the ladder for gaining experience.

Traditionally, junior analysts performed simple tasks while learning how systems work, how to read logs, how to recognize vulnerability patterns, and how to gradually move on to more difficult analysis. If those simple tasks are replaced by AI, juniors will have fewer opportunities to grow.

As a result, seniors who already have expertise may become even stronger with AI, while newcomers may face a much higher bar to entry.

The OECD’s analysis of the Korean labor market is not a document that directly discusses the security industry. Still, it shows that the benefits and risks of AI may not be distributed equally. The report analyzes that some forms of AI in Korea may be associated with slower regular-employment growth among young workers, low- and middle-educated workers, and workers in manufacturing, and explains that the benefits and risks of AI may not be distributed evenly.[15]

If we carefully apply this to the security industry, AI may not provide the same opportunity to everyone. Instead, it may amplify the productivity of people who already have strong fundamentals and judgment to a much greater degree.

It is difficult to assume that the future security industry will change only in a positive direction for everyone.

For people with expertise, it is a major opportunity. For people who lack fundamentals or only perform standardized tasks, it may become a threat.

5.3 Vulnerability Reporting and the Bug Bounty Ecosystem Are Also Changing

The impact of LLMs is not limited to vulnerability analysis capability. The vulnerability reporting ecosystem is already being affected as well.

A representative example is curl.

In a July 2025 post, curl maintainer Daniel Stenberg stated that low-quality vulnerability reports generated with LLMs, often called AI slop, were increasing. The percentage of reports confirmed as real vulnerabilities had dropped sharply. At the time, he explained that around 20% of the security reports submitted to curl in early 2025 were AI slop, while only about 5% were confirmed as real vulnerabilities.[10]

curl ended its monetary bug bounty program on January 31, 2026. Stenberg explained that the decision was driven by a combination of AI slop, lower report quality, bad-faith reports, and the burden placed on reviewers.[11]

This case shows an important point.

If AI produces many vulnerability candidates, some of them may be real. At the same time, reports that look plausible but are actually wrong can also increase sharply. Project maintainers must spend more time reviewing reports, leaving them with less time to respond to truly important vulnerabilities.

Google’s changes to the Android and Chrome VRPs can be understood in a similar context.

Google Bug Hunters announced changes to the Android and Chrome VRPs for the AI era. This can be seen as a move toward placing more value on real impact, reproducibility, actionable reports, and high-impact vulnerabilities, rather than simply submitting large numbers of things that “look like vulnerabilities.”[12]

In the future, bug bounty and vulnerability reporting ecosystems are likely to place greater importance on the ability to create reproducible PoCs, accurately explain vulnerability impact, and reduce false positives. The ability to explain patchability and operational risk together will also become important, as will the ability to turn AI-generated analysis into evidence that humans can verify.

Vulnerability reporting in the AI era is likely to become a battle of quality over quantity.

6. What Several Real-World Cases Show

6.1 Firefox: AI Can Contribute to Security Improvements in Large Real-World Codebases

The Firefox case from Anthropic and Mozilla clearly shows the defensive potential of LLMs.

Claude Opus 4.6 found 22 vulnerabilities in Firefox, 14 of which were classified as high-severity vulnerabilities. Mozilla fixed the related security-sensitive bugs in Firefox 148. Later, Mozilla stated that early evaluation with Mythos Preview helped fix 271 vulnerabilities in Firefox 150.[5][6]

This case gives a fairly strong answer to the question, “Is AI actually useful?”

It is.

But there is a condition.

The results produced by AI were not trusted blindly. Researchers from Mozilla and Anthropic verified them, triaged them, and connected them to patches. AI is a powerful engine for quickly finding vulnerability candidates, but humans are still needed to turn those results into actual product security improvements.

The Firefox case shows four things.

1. AI can find meaningful vulnerabilities in large real-world software projects.
2. It can also increase the speed of vulnerability discovery for defenders.
3. Final judgment, verification, patching, and release still depend on the capabilities of people and organizations.
4. Workflows that turn AI-generated vulnerability candidates into trustworthy security improvements will become increasingly important.

6.2 Copy Fail: Human Insight and AI-Scale Exploration Come Together

Copy Fail is a Linux kernel vulnerability case that shows how the roles of AI and human researchers can be combined.

It would be inaccurate to describe this case simply as “AI found a vulnerability.”

According to Xint’s explanation, Copy Fail started from an attack-surface insight by Theori researcher Taeyoung Lee. A human researcher pointed to the interaction between the Linux crypto subsystem and page-cache-backed data. Based on that direction, Xint Code scanned the entire crypto subsystem and identified an authencesn scratch-write bug as a high-severity finding.[13]

This case is important because it shows that the decisive starting point of vulnerability discovery can still come from human insight.

Even when AI is powerful, deciding what to look at, where to be suspicious, and which direction to explore may still begin with human intuition and expertise.

Of course, after that, AI can perform large-scale code exploration and candidate extraction much faster. Future vulnerability research is likely to evolve not as “human vs. AI,” but as a structure of human insight + AI-scale exploration.

Copy Fail shows a workflow where humans provide direction about the attack surface and bug class, AI performs large-scale exploration based on that direction, meaningful vulnerability candidates are found, and humans then verify them and assess exploitability. In the future, strong researchers may not be those who avoid AI, but those who can design problem spaces where AI can search effectively.

6.3 Dirty Frag: Disclosure and Patch Timing Become More Important

Dirty Frag is a Linux kernel LPE case that shows the importance of vulnerability disclosure and patch timing.

According to the Dirty Frag document, this vulnerability class is a Linux local privilege escalation case that chains xfrm-ESP Page-Cache Write and RxRPC Page-Cache Write. Based on the public document, the two vulnerabilities were assigned CVE-2026-43284 and CVE-2026-43500, respectively, and the mainline patches were also organized.[14]

The especially important part of this case is the order of disclosure.

The Dirty Frag document explains that on May 7, 2026, the initial disclosure date, the embargo was broken due to external factors before the patches and CVEs were ready. The mainline patches and CVEs were organized afterward.[14]

The key point here is not simply that “the vulnerability is powerful.”

The more important issue is the response gap that appears when vulnerability information is disclosed before a patch is ready. Until an accurate fix is available, defenders must judge the scope of impact with limited information, apply temporary mitigations, and check whether their internal systems are exposed.

If AI accelerates vulnerability analysis and reproduction, this gap becomes an even more dangerous period. In the past, it often took time to turn publicly disclosed vulnerability information into practical attackability. In the future, that time may become much shorter.

However, Dirty Frag should not be described as a vulnerability where “anyone can instantly get root.” The actual impact may vary depending on local access, module loading state, namespace policy, distribution configuration, kernel version, and whether patches have been backported.

Still, the message from Dirty Frag is clear. In the AI era, it is not enough to focus only on how quickly vulnerabilities can be found. It also becomes critical to decide when information should be disclosed, how quickly patches can be prepared, and how defenders can be given enough time to respond.

7. How Can We Survive If the Future We Predicted Arrives?

Based on our team’s discussion, the kind of security talent that survives in the AI era is not simply “someone who uses AI.”

Nor is it “someone who is better than AI at everything.”

The core idea is closer to this:

Someone who can understand, question, verify, and connect AI-generated results to real security decisions.

The required capabilities can be summarized into five areas.

7.1 We Need to Build Deeper CS Fundamentals

As AI advances, fundamentals become even more important.

Even if AI suggests a possible race condition, it is difficult to judge whether the analysis is correct without understanding threads, locks, synchronization, and TOCTOU issues.

Even if AI suggests SSRF, it is difficult to judge actual reproducibility and impact without understanding DNS resolution, internal networks, redirects, metadata endpoints, and parser differentials.

Even if AI points to a UAF or OOB issue, exploitability assessment becomes unstable without an understanding of memory layout and object lifecycles.

Security is not a field where we simply memorize vulnerability names.

Vulnerabilities emerge on top of fundamentals such as operating systems, networks, computer architecture, web browsers, databases, authentication structures, and cryptography.

Security talent in the AI era must go beyond understanding what AI explains. They must also be able to challenge AI when it is wrong.

7.2 We Need to Be Able to Verify AI-Generated Results

In security, what matters is not a plausible answer, but evidence.

Even if AI says, “This may lead to RCE,” it is not yet a vulnerability unless the actual input, execution path, crash, control flow, permissions, and environmental conditions are confirmed.

Even if AI says, “This is XSS,” we still need to check whether it actually executes in the browser, whether it is blocked by CSP, and whether it reaches the sink.

Even if AI says, “This code may have a race condition,” we need to verify whether the race condition can actually occur, whether the attacker can control the timing, and whether the result leads to a security impact.

In the future, the amount of analysis produced by AI is likely to explode. Some of it will be real, but much of it may be plausible-looking wrong answers. As seen in the curl case, open-source projects and bug bounty operators are already experiencing noise from the increase in LLM-generated vulnerability reports.[10][11]

The important capability is not the ability to produce many answers, but the ability to identify the real ones.

To do this, we should not blindly trust what AI says. We should always demand reproducible evidence. Logs, crashes, PoCs, patch diffs, requests, and responses need to be checked, and the actual impact of the vulnerability must be evaluated. Attack feasibility and operational risk should also be judged separately.

Security professionals in the AI era must be able to take responsibility for AI-generated results.

7.3 We Need to Be Able to Design LLM Agents and Automation Workflows

In the future, being good may not simply mean being good at asking questions to an LLM.

What will matter more is the ability to design environments, tools, and verification loops that allow LLMs to solve complex problems.

Recently, this trend is sometimes called harness engineering or scaffolding.

Harness engineering, or scaffolding, is the practice of designing an environment where a model does more than just generate answers. The model solves problems by using an execution environment, tools, sandboxes, test loops, success criteria, and verification procedures together.

When applied to security work, this can create a workflow like the following:

1. Recon
2. Hypothesis
3. Code/Target Analysis
4. Test
5. Evidence Collection
6. Reproduction or Exploitability Validation
7. Patch Suggestion
8. Report

Instead of having a human repeat every step manually, LLM agents and tools can perform the process together.

However, designing this structure requires the human to understand the problem accurately. They need to know what information should be collected, what tools should be connected, and what criteria should be used to judge success or failure.

In the future, the value of people who can design AI systems that solve security problems well will grow, in addition to people who directly solve security problems themselves.

7.4 We Need the Ability to Discover Problems and Plan Solutions

LLMs are greatly lowering the barrier to implementation.

In the past, even if we had an idea, it was difficult to build a tool without enough implementation skill. Now, with LLMs, we can build automation scripts, analysis tools, report generators, and agent workflows much faster.

This means that simple implementation skill is no longer the only thing that matters.

What becomes more important is the ability to discover which problems matter, define them in a solvable form, and design a solution structure by combining LLMs and tools. It is also important to verify whether the result is actually valid and organize it into a form that a team or organization can use.

The value of problem definers, designers, and verifiers may become greater than that of simple coders.

The same applies to security.

Finding one more vulnerability is important, but the ability to build tools, agents, skills, and verification pipelines that help find vulnerabilities better will become increasingly important.

To use an analogy, the ability to mine gold is still important. But in the future, the ability to build a better pickaxe for mining gold may become even more valuable.

7.5 Code Generated by LLMs Must Also Be Verified

Using LLMs in security work does not mean that LLM-generated code automatically becomes safe.

Studies on the security of LLM-generated code show that LLMs can produce code that looks functionally plausible, but that code may still contain vulnerabilities. The study by He and Vechev discusses security hardening and adversarial testing in LLM code generation, while SALLM proposes a benchmark framework for evaluating the security of code generated by LLMs.[16][17]

This point is also important in security automation.

PoCs, analysis scripts, patches, detection rules, and report-generation code created by AI are all targets for verification. Just because AI-generated output helps security work does not mean the output is immediately trustworthy.

Security workflows in the AI era should always include questions such as:

- Is this result reproducible?
- Can this code be executed safely?
- Does this patch break existing functionality?
- Does this detection rule create too many false positives?
- Does this automation tool itself introduce new security risks?

Just as important as using AI is building a system that verifies AI-generated outputs.

8. Conclusion

The emergence of GPT-5.5-Cyber and Claude Mythos Preview shows that the security industry is moving toward a faster and more automated future.

From an optimistic perspective, LLMs are powerful tools that can greatly improve the productivity of security professionals and help them solve harder problems. The Firefox collaboration between Mozilla and Anthropic, as well as Google Project Zero’s Big Sleep case, show that AI can provide practical help to defenders as well.[5][6][9]

From a pessimistic perspective, LLMs can lower the barrier to entry for attacks, put pressure on repetitive security work, and reduce opportunities for juniors to grow. As NCSC and Fortinet point out, as the cost of vulnerability discovery and exploitation decreases, patch pressure on defenders will inevitably increase.[7][8]

One thing seems clear.

The security industry will not continue operating in the same way as before.

The important people in the future will not be those who blindly reject AI, nor those who blindly trust AI’s answers.

What will matter is the ability to actively use AI while verifying its results with one’s own CS knowledge and practical experience, then connecting those results to real security decisions.

Security work will also no longer end with simply finding vulnerabilities. It will expand toward building systems that help AI find, verify, report, and connect vulnerabilities to patches more effectively.

In that process, “pickaxes” such as LLM agents, analysis automation tools, and skills and workflows for better CTF practice and vulnerability research will become increasingly important. A good pickaxe is not just a tool that does the work for us. It is a foundation that helps us form better hypotheses, verify them faster, and produce more accurate evidence.