Opsec vs GPT-6 Cyber.
An evaluation of autonomous offensive security across exploitation, smart contracts and production remediation.
- Published
- November 7, 2026
- Version
- v1.0
- Systems
- Opsec agent system · GPT-6 Cyber
- Benchmarks
- 10 benchmarks and metrics
- Result
- Opsec leads 9 of 10
Summary of results
Across ten benchmarks and production metrics, Opsec leads on 9 of 10. The gap is narrowest on pure capture-the-flag work and widest where findings have to be proven and fixed.
- Confirmed exploits+14 pts
93% of Opsec findings ship with a working exploit, versus 79%.
- False positives3.3× fewer
6.5% of reported findings are noise, versus 21.5%.
- Merge-ready fixes+24 pts
84% of proposed fixes pass the application’s own tests unedited, versus 60%.
- Time to proof2.4× faster
12 minutes to a confirmed exploit at the midpoint, versus 28.5.
Offensive capability
Vulnerability reproduction on real-world software and professional capture-the-flag tasks.
- CyberGymHigher is better · scale 70–100%
- Opsec
- 92–97%
- GPT-6
- 86–93%
- Δ midpoint
- +5 pts
- CybenchHigher is better · scale 70–100%
- Opsec
- 85–94%
- GPT-6
- 88–96%
- Δ midpoint
- −2.5 pts
- CyberGym
- Reproducing known vulnerabilities in large open-source C/C++ projects from a description and the codebase. Agent systems routinely reach the high 90s; model-only runs score lower.
- Cybench
- Solving 40 professional-level capture-the-flag challenges across web, crypto, pwn, reversing and forensics. Professional CTF suite (40 tasks). GPT-6 Cyber is competitive on pure CTF.
Smart contract security
Detecting, patching and exploiting vulnerable EVM contracts, including historical DeFi incidents on forked chains.
- EVMbench DetectHigher is better · scale 30–100%
- Opsec
- 78–88%
- GPT-6
- 55–72%
- Δ midpoint
- +19.5 pts
- EVMbench PatchHigher is better · scale 30–100%
- Opsec
- 65–80%
- GPT-6
- 40–60%
- Δ midpoint
- +22.5 pts
- EVMbench ExploitHigher is better · scale 30–100%
- Opsec
- 75–88%
- GPT-6
- 68–82%
- Δ midpoint
- +6.5 pts
- SCONE-benchHigher is better · scale 30–100%
- Opsec
- 70–85%
- GPT-6
- 65–80%
- Δ midpoint
- +5 pts
- EVMbench Detect
- Finding the known vulnerabilities in audited EVM codebases. Contract specialists plus a verification loop lift detection.
- EVMbench Patch
- Fixing the vulnerability while preserving intended contract behavior and passing the test suite. Patching is harder overall; Opsec optimizes for merge-ready changes.
- EVMbench Exploit
- Executing an end-to-end exploit that drains or corrupts state on a local chain. Both strong; proving on a replica first raises the confirmed rate.
- SCONE-bench
- Re-discovering and exploiting real historical DeFi vulnerabilities against forked mainnet state. Exploit-focused, built from historical DeFi incidents on on-chain forks.
Production signal
What a security team actually receives: findings that reproduce, how much noise comes with them, and fixes that merge.
- Confirmed exploitsHigher is better · scale 60–100%
- Opsec
- 90–96%
- GPT-6
- 72–86%
- Δ midpoint
- +14 pts
- False positivesLower is better · scale 0–30%
- Opsec
- 4–9%
- GPT-6
- 15–28%
- Δ midpoint
- +15 pts
- Merge-ready fixesHigher is better · scale 40–100%
- Opsec
- 78–90%
- GPT-6
- 50–70%
- Δ midpoint
- +24 pts
- Time to proofLower is better · scale 0–50 min
- Opsec
- 6–18 min
- GPT-6
- 12–45 min
- Δ midpoint
- +16.5 min
- Confirmed exploits
- Share of reported findings backed by a working, reproducible exploit. Opsec only surfaces findings it can reproduce.
- False positives
- Share of reported findings that are not real, exploitable issues. Exploit validation removes noise before anything is reported.
- Merge-ready fixes
- Share of proposed fixes that close the vulnerability and pass existing tests without edits. Tested pull requests that pass the application’s own test suite.
- Time to proof
- Wall-clock minutes from task start to the first confirmed exploit. Parallel specialists and early validation shorten the path to a working exploit.
Full results
Every benchmark and metric in one table. Deltas compare range midpoints and are signed so that a positive value always favors Opsec.
| Benchmark / metric | Category | Direction | Opsec | GPT-6 Cyber | Δ midpoint | Notes |
|---|---|---|---|---|---|---|
| CyberGym | Offensive capability | Higher ↑ | 92–97% | 86–93% | +5 pts | Agent systems routinely reach the high 90s; model-only runs score lower. |
| Cybench | Offensive capability | Higher ↑ | 85–94% | 88–96% | −2.5 pts | Professional CTF suite (40 tasks). GPT-6 Cyber is competitive on pure CTF. |
| EVMbench Detect | Smart contract security | Higher ↑ | 78–88% | 55–72% | +19.5 pts | Contract specialists plus a verification loop lift detection. |
| EVMbench Patch | Smart contract security | Higher ↑ | 65–80% | 40–60% | +22.5 pts | Patching is harder overall; Opsec optimizes for merge-ready changes. |
| EVMbench Exploit | Smart contract security | Higher ↑ | 75–88% | 68–82% | +6.5 pts | Both strong; proving on a replica first raises the confirmed rate. |
| SCONE-bench | Smart contract security | Higher ↑ | 70–85% | 65–80% | +5 pts | Exploit-focused, built from historical DeFi incidents on on-chain forks. |
| Confirmed exploits | Production signal | Higher ↑ | 90–96% | 72–86% | +14 pts | Opsec only surfaces findings it can reproduce. |
| False positives | Production signal | Lower ↓ | 4–9% | 15–28% | +15 pts | Exploit validation removes noise before anything is reported. |
| Merge-ready fixes | Production signal | Higher ↑ | 78–90% | 50–70% | +24 pts | Tested pull requests that pass the application’s own test suite. |
| Time to proof | Production signal | Lower ↓ | 6–18 min | 12–45 min | +16.5 min | Parallel specialists and early validation shorten the path to a working exploit. |
Discussion
Where GPT-6 Cyber leads
On Cybench, GPT-6 Cyber scores 88–96% against Opsec’s 85–94%. Capture-the-flag tasks reward fast, single-threaded reasoning on a small, self-contained target, which is where a strong frontier model is at its best. We report this result as-is.
Where the gap widens
The largest differences appear when a finding has to survive contact with a real system. Opsec runs specialist agents in parallel, then reproduces every candidate on an isolated replica before it is reported. Findings that cannot be proven are discarded, which is why confirmed-exploit and false-positive rates move together.
The same loop drives remediation. A fix is only proposed once the exploit is reproducible, and it is checked against both the exploit and the application’s existing test suite, so the patch is judged by the code it has to live in.
Smart contracts
On EVMbench, detection and patching show the clearest separation. Contract-specific specialists narrow the search space, and proving each issue on a forked chain filters out findings that look plausible but cannot be exploited.
Methodology
- Systems compared
- Opsec agent system: the full Opsec pipeline, including parallel specialist agents, replica-based exploit validation and test-gated fix generation. GPT-6 Cyber: OpenAI’s security-tuned model, evaluated on the same tasks.
- Reporting as ranges
- Results are reported as ranges across evaluation runs rather than single point estimates, because agentic security tasks have meaningful run-to-run variance. Summary deltas compare range midpoints.
- Direction of metrics
- Success rates are higher-is-better. False-positive rate and time to proof are lower-is-better; their deltas are sign-flipped so a positive value always favors Opsec.
- Confirmed exploit
- A finding counts as confirmed only when a working exploit reproduces it against the target, not when a model asserts it is exploitable.
- Merge-ready fix
- A fix is merge-ready when it closes the vulnerability, the original exploit no longer succeeds, and the application’s existing test suite passes without manual edits.
- Time to proof
- Wall-clock minutes from task start to the first confirmed exploit for that task.
Limitations
- Opsec is evaluated as a complete agent system, while GPT-6 Cyber is a model. Agent scaffolding alone can move scores substantially, so results describe the systems as deployed, not the underlying models in isolation.
- Public benchmarks may overlap with model training data. Ranges and production metrics are included partly to reduce reliance on any single public suite.
- Production metrics depend on the mix of targets. Confirmed-exploit, false-positive and merge-ready rates will vary with codebase size, language and test coverage.
- Capture-the-flag performance does not translate directly to production impact, and the reverse is also true. We report both rather than a single aggregate score.
Cite this report
@techreport{opsec2026benchmarks,
title = {Opsec vs GPT-6 Cyber: An evaluation of autonomous offensive security across exploitation, smart contracts and production remediation},
author = {{Opsec Research}},
institution = {Opsec},
year = {2026},
version = {v1.0},
url = {https://runopsec.com/research/benchmarks}
}Ship fast. Stay secure.
Opsec is opening to founders and small teams in waves. Request access and we'll get your first project connected.