WinnowSec puts AI agents to work for open-source maintainers. Every finding arrives with a reachable path, a blind re-check by a different model and a person's sign-off.
An attacker's agent can try thousands of attacks; a miss costs it nothing, and one hit is enough. A maintainer has to check every report by hand, including the false ones, and each check takes time.
Automated. Each red flash is one attempt; misses cost nothing, and one hit is enough.
Still probing, but the files it needs were already fixed.
By hand. Reports arrive faster than one person can check them, false alarms included.
Checked first. Only verified findings arrive, each with a path and a patch.
Illustrative. Switch to see what reaches the maintainer when the checking is done first: the real bug gets fixed before the attacker lands a hit.
Every scanner hit passes four gates before it reaches you. Anything that fails one is listed in the report, with the reason.
VTM, a deliberately vulnerable Django app: 30 confirmed at validation; 27 upheld on blind re-read, 2 refuted, 1 judge answer unreadable (a tool failure, not an abstention). Reviewer and judge were the same model, so read 27 as optimistic; reviews sent to maintainers use a different judge model. One target, and an unsubtle one. The particle flow is illustrative.
Most open-source projects have none set up. We host open-source scanners, including OpenGrep, OSV-Scanner and DefectDojo, run them on your code and review what they find. Nothing to install on your side.
In a study of over a million npm and PyPI packages, 94–97% had no automated dependency updates. Zahan et al., 2023
Agents that find vulnerabilities are available to defenders now, before attackers use them at scale. OpenAI calls that gap the defender's window, and says it is closing.
A fix merged upstream can protect every project built on it.
Open-source code is public to attackers too. The advantage is going first.
Every finding carries its evidence, tied to the file, line and commit. You decide what to fix.
What each number means for the people who maintain open source.
In DARPA's AI Cyber Challenge, autonomous systems analysed more than 54 million lines of code from open-source projects. Besides the bugs planted for the contest, they found 18 real ones, which are being reported to the projects' maintainers.
Vendors of mainstream static-analysis tools report that 40 to 60 of every 100 alerts turn out to be false. Someone still has to check each one.
Vendor-reported range, mainstream SAST tools
In the final week of curl's bug bounty, 7 reports came in and none described a real vulnerability. The project ended the program in January 2026 to remove the incentive for AI-generated reports.
In July 2026, an autonomous agent framework broke into Hugging Face over a weekend. It started from a malicious dataset and ran thousands of actions from a swarm of short-lived sandboxes. The response used AI too: an open-weight model analysed 17,000+ attack events in hours.
We are a team of cybersecurity and applied AI researchers, committed to making the digital world a safer place. We run every review ourselves, read every report before it goes out, and publish what we learn, including whether agents, deterministic tools and independent review really cut false positives.
It starts with one repository: free for public GitHub repositories, delivered privately. We publish what we learn along the way, including what went wrong.
One public GitHub URL, by email.
A draft security advisory with our account added. About two minutes.
Inside that advisory, read by a person, with a checked patch for each confirmed finding, or the reason there is none.
Next on the roadmap: private repositories and reviews on demand.
Deterministic tools gather context. Four agent stages investigate it, and the runner sends back whatever they left unread or unanswered. A blind re-check and a person stand between the pipeline and delivery.
Today, one turn per request, for public repositories. The goal: a turn whenever you need one, private code included.
Cloned at a pinned commit. Every finding links back to that exact commit.
Symbols, calls and imports from the syntax tree, with zero model tokens. The agents query it to navigate.
Rule-based SAST, run once in a sandbox and queried by every step. A scan with errors is reported as incomplete, never as clean.
Already run scanners? Send the consolidated DefectDojo report. No stack yet? WinnowSec AIeSCR runs Semgrep, SonarQube and OSV-Scanner itself, OSV against an offline database, and builds the same report. Either way, each finding gets its own verdict: confirmed, rejected or inconclusive.
Behaviour, stack, routes and access controls.
The checklist, domain by domain. Every scanner hit ends as a finding or a dismissal with a reason.
The runner lists the code no step opened and sends it to bounded readers, more of them for a larger repository. What they flag goes to Validate like any other candidate.
Re-reads every cited line, then confirms, downgrades or dismisses. Any scanner hit or candidate it leaves without a verdict is asked again; what still has none is marked open.
Counts computed in code. Permalinks built by the runner. Every file you receive shows the same findings, and the runner checks that it does.
Traced from entry point to impact. The runner checks every hop against the code and reads the severity off a fixed table; the model never sets it, and nothing is removed.
A diff checked against your commit with git apply --check. When the finding was reproduced, the patch is re-run in the sandbox: the exploit, the build, a normal request, your tests. A committed secret gets rotation steps instead of a diff.
A different model gets the file and line of each finding, never the argument. It runs separately, started by hand.
Including whether each step actually read code before it answered.
@login_requireddef ping(request): host = request.POST['ip'] cmd = "ping -c 1 " + host subprocess.Popen(cmd, shell=True) return render(request, 'ok.html')
A dangerous call is reported only if attacker-controlled input can reach it with nothing in the way.
A different model gets the file and line, never the argument, and has to rebuild the finding from the code.
Totals and citations are computed by the runner. No model is asked to count.
Pass marks are set in advance and results are recorded regardless, including a change that shipped as the default without passing its gate.
Every report is read by a person before it is sent.
Swapping the model doesn't touch the skills or the orchestration: it runs on open-weight models or hosted ones. The test runs on the Results page used a hosted frontier model. Every result we publish names the model behind it.
It happened in this project: the scanner had flagged real code, yet the report said “no security findings”.
Opengrep had flagged the repository. The code under review was real.
An agent's answer had been cut off midway. It wasn't empty and carried no error marker, so it counted as finished. Now every output has to pass the next step's parser. A parser can't tell whether any code was actually read, so a person checks that before anything is sent.
Some findings are confirmed by reproducing them, and that happens only in the reproduction sandbox.
| bound | review runner | SAST scan | reproduction sandbox | code graph |
|---|---|---|---|---|
| memory | ● | ● | ● | ● |
| swap disabled | ● | ● | ● | ● |
| cpu | ● | ● | ● | — |
| process count | ● | ● | ● | ● |
| single-file size | ● | ● | ● | — |
| core dumps · /dev/shm | ● | ● | ● | — |
| log volume capped | ● | ● | ● | — |
| killed on timeout | ● | ● | ● | ● |
| seccomp · AppArmor | ● | ● | ● | n/a |
| caps dropped · no-new-privs | ● | ● | ● | n/a |
| read-only filesystem · checkout | ● | ● | copy | n/a |
— not bounded on that component · n/a does not apply to it · copy: the sandbox works on a disposable copy of the checkout, which it may change. Bounds are read back from the live cgroup for the review runner, the reproduction sandbox and the code graph, because this project has shipped limits that were present in writing and absent in effect; the scan container's are checked in its command line. Those tests are run by hand for now.
Static analysis covers every file and gives the same answer twice, but it can't tell which hits are real. Agents investigate each one; you get the verdict, the reasoning and the evidence to check it.
A location for every hit, with no verdict and no fix. Thousands of lines on a large codebase.
A verdict, the reasoning and a patch for each finding that survives. Dismissed hits are listed with the reason.
[HIGH] SQL Injection UserDAO.java:142
[HIGH] SQL Injection UserDAO.java:158
[HIGH] SQL Injection UserDAO.java:174
[HIGH] Path Traversal FileCtrl.java:88
[MED] Weak Hash CryptoUtil.java:31
… and thousands more
Where, and nothing else.
filter (LoginController:88) → UserService.search() → UserDAO.query(). No sanitisation on the way. Public endpoint.- String q = "SELECT … WHERE name='" + filter + "'"; + PreparedStatement ps = con.prepareStatement( + "SELECT … WHERE name = ?"); + ps.setString(1, filter);
git apply --check) · permalink to the exact line · re-checked blindUsing language models to triage static-analysis output has been studied since 2024. WinnowSec AIeSCR applies the idea to open-source repositories.
The best agent setup cut residual false positives on OWASP Benchmark to 6.3%. Results depended heavily on the model.
Read the paper → 2025 · arXiv 2511.0402343.7% fewer false positives and 11.2% better recall than CodeQL, Joern and LLM-only pipelines.
Read the paper → 2025 · arXiv 2510.02534Improving the precision of static analysis with language models.
Read the paper →Reviews start from scanner hits and a fixed checklist. A vulnerability neither leads to can be missed.
On the four largest test repositories the review opened 1 to 13% of the code, and two of them got no confirmed finding from it. The report states the share it read. The reading budget now grows with the repository; that change is not yet measured.
The weakest area. Whether an access rule is wrong depends on what the application is meant to allow, which the code rarely says.
Published studies find agent triage least reliable here, Sifting the Noise included.
Code under review can contain comments aimed at the agents. A person reads every report before it is sent.
A patch that applies cleanly can still fail to build, break tests or miss the bug. It is re-run only when the finding was reproduced in the sandbox; review it like any other contribution.
The test runs on the Results page used a hosted frontier model. A self-hosted open-weight model keeps your code on our machines, with less raw capability. We tell you which one reviewed your code.
A review is a map of open vulnerabilities, with file names and line numbers. The public summary shows none of that.
Seven were written for security training, so their flaws are public already, and they are named below. Seven are production projects whose maintainers have not seen the results: they appear only in totals, and their findings go to the maintainers first. All ran in September 2026 on a hosted frontier model.
Scanners: Semgrep, SonarQube and OpenGrep. Reached means a scanner row near the vulnerability whose rule describes the same weakness; 3 of their 17 were only partial. The review confirmed 17 of the 32, and 17 of its 19 scored confirmations match a known vulnerability. The known set is ours, revised four times, twice by an auditor who saw no review output. One run, one target, an unsubtle one.
| built for training | stack | scanner rows | rejected | review confirmed | no scanner nearby | code read |
|---|---|---|---|---|---|---|
| VTM | Django | 132 | 16% | 22 | 7 | 99% |
| NodeGoat | Node.js | 350 | 67% | 11 | 8 | 78% |
| RailsGoat | Rails | 125 | 50% | 16 | 2 | 36% |
| SKEA Django | Django | 5 | 0% | 1 | 1 | 86% |
| SKEA Node | Node.js | 73 | 82% | 1 | 1 | 100% |
| SKEA Rails | Rails | 138 | 62% | 2 | 0 | 37% |
| Handouts | templates | 552 | 48% | 0 | 0 | 100% |
| 7 production projects, combined | — | 3,620 | 55% | 35 | 24 | 1–69% |
Scanner rows: Semgrep, SonarQube and OSV-Scanner, each adjudicated on its own. No scanner nearby: no scanner row or OpenGrep hit within five lines of the finding. Code read: the share of the code the review opened, measured by the runner from its reads, not reported by the model.
Targets, not results: goals the project set before its first review. Each is published here when it's measured, pass or fail. The first readings come from VTM alone.
Free reviews for public GitHub repositories you maintain. Send a link, not code. We confirm you maintain it before any review begins, and the report comes back privately, inside a draft security advisory.
Prepared in your browser. Nothing is sent until you send it from your own mail app.
One public GitHub URL. Don't send code, attachments or access tokens.
Open a draft security advisory and add our account to it. Only admins can create one, and it stays private.
At a pinned commit. A different model re-checks each finding blind; a person reads the report before it's sent.
The report is posted inside that draft advisory. It isn't public.
About two minutes. You need admin access to the repository.
GitHub's own guides: Creating a repository security advisory · Adding a collaborator to an advisory
Not an admin? Commit the one-time token we email you to the default branch as .winnowsec-verify, and delete it once we confirm: the file is public. The report then comes encrypted to the address you wrote from.
Cloned read-only at one commit, and deleted when the review closes.
Only to confirm a finding, inside the bounded container shown on the Method page.
No commits, branches, issues or pull requests. The only thing we write is inside your private advisory.
A public summary appears only after the maintainer has read the report and agreed in writing.
Without proof of ownership, nothing runs. There is no way around step 02.
A fork or mirror makes you admin of a copy, not maintainer of the original.
Coming with the GitHub App on the roadmap below.
This reads and reproduces from source. It does not test running services.
Public GitHub repositories, free. Ownership proved through a draft advisory, and every report read by a person.
Private repositories, through an app that can only read file contents. GitHub has no read-only collaborator role on personal repositories, so the app replaces invites entirely.
Request a review of any repository you maintain, whenever you need one, including results from the scanners you already run through DefectDojo. Reviewing a single change instead of the whole repository is already built.