A hybrid scanner for the OWASP Top 10 (2025): static analysis (Opengrep with open
rules) finds candidate vulnerabilities, and an LLM gives a second opinion on each to
cut false positives. It is benchmarked on the OWASP Benchmark (Java) with known
answers.
PRELIMINARY results.
The LLM triage numbers below come from 1 of 3 planned runs, so
there is no run-to-run spread yet. Differences of a few points may be noise.
This page is a static, read-only demo. The live
scanner (paste code, get findings) is not hosted here. It runs locally with Docker:
see the README for the two commands.
What I built
Custom rules: a Java rule pack for SQL injection, trust boundary,
weak randomness and XPath injection, the categories the open rules miss.
LLM triage pipeline: each finding and its code context goes to an LLM
for a true/false-positive verdict. Only confident false positives are dropped, every call
is cached, and fixes are checked by re-scanning.
Reproducible benchmark: pinned OWASP Benchmark commit, fixed-seed
450-case sample, scored against the official answers; categories without ground truth
are marked "Not covered".
In plain English:precision is how many flagged issues are real;
recall is how many real issues get found; false-positive rate is how many
safe code samples are wrongly flagged. Higher precision and recall are better; lower
false-positive rate is better.
Configuration
Precision
Recall
False-positive rate
A. Opengrep (sast-rules)
0.675
0.600
0.289
B. A + custom rules
0.676
0.889
0.427
C. B + LLM triage (1 run)
0.712
0.889
0.360
A. Opengrep (sast-rules)B. A + custom rulesC. B + LLM triage (1 run)
Custom rules (B) raise recall but also the false-positive rate; they are broad and untuned.
LLM triage (C) removed 19 of 388 findings in the single run, lowering the false-positive rate without losing recall.
The model may have seen the public Benchmark in training, which could flatter C.