All posts
Concept ~14 min read

A CVE being ‘fixed’ doesn’t mean you’re safe: incomplete-fix variant analysis

How to read a fresh security patch for the instances it missed: the four questions I ask, real fix diffs from flyto-core and PhotoBlocks, Semgrep and CodeQL sketches, advisory feeds, and where the method stops working.

The assumption worth attacking

A security advisory goes out. It says “Fixed in version X. Upgrade.” Everyone upgrades and moves on, and the shared assumption is that the class of bug described in that advisory is now closed for this project.

Often it isn’t, and the reason is structural rather than careless.

When a maintainer gets a report, they fix the instance that was reported. They rarely re-audit the whole codebase for every reachable copy of the same mistake, because the report is specific, the deadline is short, and the reporter’s proof of concept is the test they have. So the fix lands on one function, one endpoint or one encoding, and the sibling component with the same sink, or the other input format, keeps the bug. The advisory also tells everyone what to look for. A freshly published fix is a labelled map of where a developer was recently thinking about a vulnerability class and may not have finished.

That is why patch diffing and incomplete-fix variant analysis work. You are not searching the whole codebase blind; you are checking whether a known bug was removed everywhere it lives.

This post is the long version of how I do it. It covers the method, three public cases with the actual fix diffs, the tooling I use to find the missed spot, how I follow advisory feeds, what my own sweeps turned up, and where the method stops paying.

The method

For a recently published fix (ideally days old), I ask four questions:

  1. Did the patch cover every reachable instance, or just the reported one? Grep the codebase for the sink that was fixed. If the fix touched file_a.py but the same pickle.loads(request_body), send_file(user_path) or unsafe deserialize exists in file_b.py, the fix is incomplete by omission.
  2. Is there a sibling component with the same pattern? Many projects have parallel code: a loader and a result_storage, a config server and a worker, an http client and an agent client. Fixes land on one and skip the twin.
  3. Does the fix handle every encoding or variant of the input, or one? A blocklist that now decodes one representation of a forbidden value invites the other representations. SSRF guards that learned to decode IPv4-mapped IPv6 addresses and stopped there are a common example.
  4. Does the “patched” release behave as claimed? Don’t trust the changelog. Install the fixed version and run the original PoC plus variants. “Fixed in X” is a claim to test.

Then you report the gap as what it is, an incomplete remediation of CVE-YYYY-NNNNN, with the exact file and line the fix missed. Maintainers tend to take these seriously, because they already published an advisory that promised more than the code delivers.

A public case first: Log4Shell

CVE-2021-44228 in Apache Log4j was fixed in 2.15.0. Within days, CVE-2021-45046 showed the fix was incomplete: in certain non-default configurations (lookups reached through Thread Context data in a layout pattern) JNDI lookups still fired. It was first rated as DoS and later upgraded, because RCE was possible in some environments. 2.16.0 removed message lookups and disabled JNDI by default. Then CVE-2021-45105 (a recursive-lookup DoS) needed 2.17.0, and CVE-2021-44832 (JDBC appender with attacker-controlled configuration) needed 2.17.1.

Four releases for one bug class, because each fix closed the path that had been reported. Log4j had a large, experienced team and the whole industry watching. The pattern is not about competence.

Case 1: flyto-core, a guard added to every module but one

flyto-core is a Python framework for building LLM workflows. Its modules make outbound HTTP requests to URLs that workflow authors supply, and the project has a real SSRF guard, validate_url_with_env_config, that each module is supposed to call. There is no global egress filter, so coverage depends on every module remembering to call it.

The parent advisory, GHSA-pgwh-4jj4-qm8v (CVE-2026-67428), listed more than a dozen modules that skipped the guard. Its text ended the list with “the agent/llm inline base_url branch”. The fix shipped in 2.26.7. Comparing the tags shows what it touched:

What the 2.26.7 fix changedShell
gh api 'repos/flytohub/flyto-core/compare/v2.26.6...v2.26.7' \
  --jq '.files[] | .filename'

The result is fourteen module files (graphql/query.py, graphql/mutation.py, monitor/http_check.py, slack_send.py, the Discord/Slack/Teams notifiers, developer/http/requests.py, vision_analyze.py, visual_diff.py, proxy_rotate.py and the http.* modules), plus utils.py and the verification service. src/core/modules/atomic/llm/agent.py is not in the list. Each sibling got the same few lines:

src/core/modules/atomic/graphql/query.py, 2.26.7diff
+from ....utils import enforce_outbound_url, SSRFError
 
 logger = logging.getLogger(__name__)
 
@@ -32,6 +33,13 @@ def _prepare_graphql_request(params: Dict[str, Any], operation_key: str = 'query
+    # SECURITY: gate the client-controlled URL through the SSRF guard, like the
+    # guarded sibling http.get (GHSA-pgwh-4jj4-qm8v).
+    try:
+        enforce_outbound_url(url)
+    except SSRFError as e:
+        raise ModuleError("SSRF protection blocked request: {}".format(e))

enforce_outbound_url is a new wrapper in utils.py that calls validate_url_with_env_config when the guard is enabled. So after 2.26.7, the question “which modules reach a URL without the guard?” has a mechanical answer. Grepping the 2.26.7 tree for the LLM modules shows the asymmetry:

git grep on the v2.26.7 tagtext
src/core/modules/atomic/llm/agent.py:27:from ....utils import assert_env_credential_endpoint_allowed, CredentialEndpointError
src/core/modules/atomic/llm/agent.py:34:from ._chat_models import create_chat_model
src/core/modules/atomic/llm/agent.py:877:            return create_chat_model(**config)
src/core/modules/atomic/llm/agent.py:902:    assert_env_credential_endpoint_allowed(caller_base_url, key_from_env)
src/core/modules/atomic/llm/agent.py:919:        return create_chat_model(
src/core/modules/atomic/llm/chat.py:17:    validate_url_with_env_config,
src/core/modules/atomic/llm/chat.py:19:    assert_env_credential_endpoint_allowed,
src/core/modules/atomic/llm/chat.py:157:            validate_url_with_env_config(base_url)
src/core/modules/atomic/llm/chat.py:182:        assert_env_credential_endpoint_allowed(base_url, key_from_env)

chat.py calls both the SSRF guard and the credential check. agent.py calls only the credential check, which exists to stop the operator’s environment API key from being sent to an arbitrary host. It returns immediately when the caller supplies their own key, so it does nothing against SSRF. The project’s automated coverage test listed it as an accepted guard anyway, which is why the agent module looked protected. The full story is in The guard that wasn’t.

I found this by doing exactly what the advisory invited: it named the agent branch as fixed, so I opened the agent branch in the patched release. It was the same code as before the fix, and still the same in 2.28.1, the latest release when I reported (GHSA-f9q4-fp8j-r5h7). The maintainer shipped 2.29.0 five days after the report:

src/core/modules/atomic/llm/agent.py, 2.29.0 (trimmed)diff
+def _guard_inline_base_url(base_url: Optional[str]) -> None:
+    """SSRF-validate a caller-supplied model endpoint before the agent fetches it.
+    ..."""
+    if base_url:
+        validate_url_with_env_config(base_url)
 
@@ -874,6 +898,9 @@ def _resolve_chat_model(context: Dict) -> Optional[ChatModel]:
             logger.info(f"Using sub-node config: {config.get('provider')}/{config.get('model')}")
+            # The backward-compat dict reaches the same builder as the inline
+            # path, so it gets the same check rather than being a way around it.
+            _guard_inline_base_url(config.get('base_url'))
             return create_chat_model(**config)
 
@@ -884,6 +911,8 @@ def _resolve_chat_model(context: Dict) -> Optional[ChatModel]:
     caller_base_url = base_url
+    _guard_inline_base_url(caller_base_url)

The same release removed the credential check from the coverage test’s list of guards, with a comment that names both reports. That second change matters more than the first: it fixes the reason the gap was invisible, so the next module that forgets the guard fails the test.

What to take from the diff: when a fix adds the same guard to many files, the guard is opt-in per file. Your job is to list every file that reaches the sink and subtract the ones that call the guard. The remainder is either safe for a reason you can name, or it is the variant.

Case 2: PhotoBlocks, three attributes escaped and one left raw

Gallery PhotoBlocks is a WordPress gallery plugin. CVE-2026-66448 was a Contributor-level stored XSS fixed in 1.3.4. Before reporting what I had found in the same template, I pulled both tags from the plugin SVN to make sure I wasn’t reporting the same bug twice. The diff answered both questions at once. The admin code and class-photoblock.php were byte-identical, and the only change was this:

public/partials/photoblocks-public-display.php, 1.3.3 to 1.3.4diff
                 rel="<?php 
-        echo $block->click->rel;
+        echo esc_attr( $block->click->rel );
         ?>"
                 <?php 
         if ( $block->has_link_or_lightbox() ) {
             ?>
                     target="<?php 
-            echo $block->click->target;
+            echo esc_attr( $block->click->target );
             ?>"
                 <?php 
         }
         ?>
                     href="<?php 
-        echo $block->get_link();
+        echo esc_attr( $block->get_link() );
         ?>">

Three attributes on the gallery link, in the reported spot. About thirty lines higher in the same file, the <img> tag still echoed a stored gallery setting into a quoted attribute with no escaping:

photoblocks-public-display.php, 1.3.4PHP
                <img
                    class="pb-image skip-lazy <?php 
        echo $block->get_image_class();
        ?>"

image_class is saved through sanitize_text_field(), which strips tags but keeps double quotes, so x" onload="alert(document.domain) closes the attribute and adds an event handler. That became CVE-2026-107323. The vendor’s fix in 1.3.6 escaped that line and ten other echoes in the same template:

photoblocks-public-display.php, 1.3.4 to 1.3.6 (one hunk of eleven)diff
                 <img
                     class="pb-image skip-lazy <?php 
-        echo $block->get_image_class();
+        echo esc_attr( $block->get_image_class() );
         ?>"

The 1.3.6 diff is what a whole-file pass looks like: style, data-valign, data-halign, the link class, the overlay colour, caption positions and custom styles all gained esc_attr(). The 1.3.4 diff is what a fix for the report looks like. When you see a fix of the second shape, open the file and read every other echo. The write-up with the full data path is One unescaped class name.

Case 3: Salon Booking System, one controller fixed and its siblings not

The third shape is inheritance. In Salon Booking System, CVE-2026-110012 let any Subscriber read every customer’s bookings through REST routes whose shared permission check, inherited from a base controller, resolved to the core read capability that every role holds. The first fix overrode the check in Bookings_Controller only:

Bookings_Controller.php, after the booking fixPHP
protected function current_user_can_list_bookings()
{
    return current_user_can('manage_options')
        || current_user_can('manage_salon')
        || $this->is_shop_manager();
}

The weak logic lived in the base class, and three other controllers inherited it. Discounts_Controller was byte-identical before and after the booking fix, and it returned every coupon code to any Subscriber. That was CVE-2026-110035. The later fix moved the stricter check into the base class, which covers every subclass at once. The full analysis is in Logged in is not the same as allowed.

The review habit: when a fix overrides behaviour in one child class, list every class that inherits the same parent and test each one.

Tooling for finding the missed spot

None of this needs special tools. Most of my time goes into git, grep and reading. Static analysis helps when the pattern is clear enough to write down.

Diffing a fix

Start from the tags on either side of the fix, not from the commit message.

Reading a fixShell
# Which files did the fix touch?
git diff --stat v2.26.6..v2.26.7

# The fix itself, limited to the code that matters
git diff v2.26.6..v2.26.7 -- src/core/modules/

# When was a guard introduced, and where did it spread?
git log -S'enforce_outbound_url' --oneline -- src/

# Who else reaches the sink? Files with the sink, minus files with the guard
git grep -l -E 'create_chat_model\(|ClientSession\(' v2.26.7 -- src/core/modules | sort > sinks
git grep -l -E 'enforce_outbound_url|validate_url_with_env_config|guarded_aiohttp_request' v2.26.7 -- src/core/modules | sort > guards
comm -23 sinks guards

The last command, run against flyto-core 2.26.7, returns 27 files, and agent.py is one of them. Most of the rest call fixed vendor endpoints (Stripe, Gmail, GitHub and so on), and a look at where each URL comes from removes them quickly. Two details made the difference here. The sink pattern has to include the client factory (create_chat_model) and not just ClientSession, because the agent module never touched aiohttp directly. And the guard list has to be one you checked yourself, not the project’s own list, because the project’s list was the bug.

For WordPress plugins there is no git history, but the plugin SVN has every tag. svn diff between two tag URLs works, and so does fetching the same file from https://plugins.svn.wordpress.org/<slug>/tags/<version>/<path> for both versions and running diff -u. That is how the PhotoBlocks diff above was produced.

A Semgrep rule for “pattern without its sibling”

Many incomplete fixes have a recognisable signature: the code handles one variant and lacks the code for its siblings. Semgrep can express that with pattern-not-inside. This rule flags Python functions that decode IPv4-mapped IPv6 addresses but never touch 6to4 or the NAT64 prefix:

ipv6-guard-only-ipv4-mapped.yamlYAML
rules:
  - id: ipv6-ssrf-guard-decodes-only-ipv4-mapped
    languages: [python]
    severity: WARNING
    message: >
      This address check unwraps IPv4-mapped IPv6 (::ffff:a.b.c.d) but no other
      transition form. NAT64 (64:ff9b::/96), 6to4 (2002::/16) and IPv4-compatible
      addresses reach is_global / is_private unchanged.
    patterns:
      - pattern-inside: |
          def $FUNC(...):
            ...
      - pattern: $ADDR.ipv4_mapped
      - pattern-not-inside: |
          def $FUNC(...):
            ...
            <... $X.sixtofour ...>
            ...
      - pattern-not-inside: |
          def $FUNC(...):
            ...
            <... "64:ff9b::/96" ...>
            ...

The public instance of this exact shape is the SSRF classifier in mcp-atlassian (GHSA-85p8-f4j5-j5jp), which decoded addr.ipv4_mapped before an is_global test and nothing else. Treat the rule as a starting point. It will miss guards that handle NAT64 in a helper outside the function, and it will flag functions that do handle it with a different spelling. Both are cheap to check by hand, which is the point: the rule narrows a large codebase to a short reading list.

The same structure works for other pairs: esc_attr on some echoes in a template but not others, yaml.safe_load in one loader and yaml.load in its twin, a path check on upload but not on download.

A CodeQL sketch for “flow that skips the guard”

When the guard is a function call rather than a syntactic pattern, a taint query is the better fit. This sketch for Python reports remote input that reaches an HTTP client inside a function that never calls the project’s guard:

MissingProjectSsrfGuard.qlql
/**
 * @kind path-problem
 * @id py/ssrf-missing-project-guard
 */
import python
import semmle.python.dataflow.new.DataFlow
import semmle.python.dataflow.new.TaintTracking
import semmle.python.dataflow.new.RemoteFlowSources
import semmle.python.Concepts

predicate callsGuard(Function f) {
  exists(Call c | c.getScope() = f |
    c.getFunc().(Name).getId() = ["validate_url_with_env_config", "enforce_outbound_url"]
    or
    c.getFunc().(Attribute).getName() = ["validate_url_with_env_config", "enforce_outbound_url"]
  )
}

module MissingGuardConfig implements DataFlow::ConfigSig {
  predicate isSource(DataFlow::Node n) { n instanceof RemoteFlowSource }

  predicate isSink(DataFlow::Node n) {
    n = any(Http::Client::Request r).getAUrlPart() and
    not exists(Function f | f = n.getScope() and callsGuard(f))
  }
}

module MissingGuardFlow = TaintTracking::Global<MissingGuardConfig>;
import MissingGuardFlow::PathGraph

from MissingGuardFlow::PathNode src, MissingGuardFlow::PathNode sink
where MissingGuardFlow::flowPath(src, sink)
select sink.getNode(), src, sink, "URL from $@ reaches an HTTP client without the project's SSRF guard.",
  src.getNode(), "remote input"

Two caveats keep this honest. In a framework like flyto-core the input does not come from a recognised web framework, so RemoteFlowSource will not fire; you replace it with your own source, such as reads from the module’s params dictionary. And the guard check here is per function. In flyto-core the guard sat in the module and the HTTP call sat in a shared builder one call away, so a useful version needs a barrier that follows the call path, for example by treating values that flowed through a guarded call site as sanitised. The sketch shows the shape of the question; the project decides how much modelling it needs.

Following advisory feeds

Variant analysis depends on seeing fixes while they are fresh, before the maintainer does a second pass or someone else reads the same diff.

The GitHub Advisory Database has a REST endpoint that filters by ecosystem and sorts by publication date:

Newest reviewed advisories for PyPIShell
gh api 'advisories?ecosystem=pip&per_page=50&sort=published&direction=desc&type=reviewed' \
  --jq '.[] | [.ghsa_id, .published_at, .severity, .summary] | @tsv'

The ecosystem values include pip, npm, composer, go, maven, rubygems and rust. Per-repository advisories, including the full text and references, come from gh api repos/OWNER/REPO/security-advisories. If you prefer GraphQL, securityVulnerabilities(ecosystem: PIP, first: 50, orderBy: {field: UPDATED_AT, direction: DESC}) returns the package, the vulnerable range and the first patched version in one query.

OSV aggregates GitHub, PyPA, RustSec, Go and others. For a single package, POST https://api.osv.dev/v1/query with {"package": {"name": "...", "ecosystem": "PyPI"}} returns its history. For sweeping, the bulk exports at https://osv-vulnerabilities.storage.googleapis.com/<ecosystem>/all.zip are easier than paging an API, and each ecosystem folder has a modified_id.csv for incremental updates.

The CVE List is on GitHub as CVEProject/cvelistV5. Pulling it and reading cves/deltaLog.json shows what was added or changed recently, including CVEs from CNAs that never touch the GitHub database.

For WordPress, the useful sources are the WPScan vulnerability database, Patchstack’s database and Wordfence Intelligence, plus the plugin repository itself: the SVN log and the “Changelog” tab often say “security fix” before any advisory exists. Maintainers’ own release notes and CHANGELOG.md “Security” sections are worth watching for the projects you care about, because a fix often ships days before the advisory is published.

Which advisories are worth reading

The feeds produce far more than anyone can read. I triage on the shape of the fix more than on severity:

  • A fix that adds the same guard to many files means the guard is opt-in. Count the files that reach the sink and compare. flyto-core 2.26.7 is the model case.
  • A fix that adds an entry to a blocklist, a regex or an enumeration of “bad” values is a strong signal. Blocklists are incomplete by nature, and the question “what else belongs on this list?” usually has an answer.
  • A small fix to a large class: one esc_attr() call in a template full of echoes, one controller override in a class hierarchy, one decoded encoding in an address check.
  • A fix that changes a child class or a single call site when the advisory text describes a shared helper.

What I skip: fixes that move validation to a central choke point, fixes that switch from a blocklist to an allowlist, and fixes in projects whose security team publishes a design note with the patch. Those are usually complete.

What my own sweeps turned up

I started with WordPress in August. My early notes from that month already record reading fresh security fixes as the approach that worked best for me, and the PhotoBlocks and Salon Booking findings above both came out of that month’s diffs. Both plugins are maintained by small teams, and both fixes were the “report-shaped” kind.

In late October I tried the method at scale on fresh GitHub advisories. It found nothing:

  • On 26 October I audited seven fresh high and critical advisories from a four-day window, end to end, looking for siblings of each fix. None of the seven turned out to be incomplete. GitPython validated paths before any mutation, mistune switched to realpath plus commonpath, and the langgraph fix covered every store backend across both the Python and JavaScript packages.
  • On 28 October I triaged about 100 high and critical advisories from the previous three weeks across Composer, npm and PyPI, and read five of the fixes in depth. All five were thorough. The litellm fix, for one, removed every "https" in model substring check in the codebase, not only the one in the report; a grep of the patched release found none left.
  • In the same session I checked four fresh advisories in widely used libraries with dedicated security teams. All four were closed properly.
  • Earlier that month, a pass over roughly 65 recently patched or popular WordPress plugins, looking for variants and access-control gaps, found nothing inside the scope of the bounty program I was targeting.

My conclusion from those numbers: in late 2026, large and well-run projects fix the class, often with defence in depth, and spraying their advisories is a poor use of time. The variants I have found came from smaller projects, from fixes built on blocklists or per-file opt-in guards, and from fixes that clearly matched the reporter’s PoC line for line. flyto-core, PhotoBlocks and Salon Booking all fit that description. I now pick advisories by project size and fix shape first.

Limits

Most fixes are complete. If you go in expecting a variant behind every advisory, you will waste days and start seeing bugs that are not there. The useful output of reading a complete fix is a note saying so, so you don’t read it again.

Someone else is reading the same diff. A fresh advisory is public, and so is its patch. flyto-core’s fix credited an independent report of the same parameter (GHSA-pp5w-w9c3-qfv2) alongside mine. My report on the mcp-atlassian classifier was closed as a duplicate of an earlier one, and I am listed as a finder rather than the primary reporter. Speed matters, and so does checking the project’s open advisories before you write anything up.

The layer you didn’t read can close the gap. Of roughly 45 findings I have reported, two were closed as not a vulnerability. One was a variant of a published authorization advisory: the handler looked as weak as the original, but a check two layers down, in the service layer, already refused the change. I had stopped reading at the handler. A live PoC against the patched release would have shown it.

Severity is negotiable. Variants often need more privilege or a less common configuration than the original, and maintainers and programs say so. CVE-2021-45046 was rated as DoS before it was upgraded. The PhotoBlocks variant needs a Contributor account, which put it outside Patchstack’s bounty scope; I reported it through WPScan instead, which accepts Contributor-level XSS and assigns CVEs. State the precondition plainly in the report. A maintainer who has to discover it themselves will trust the rest of the report less.

Not every gap is a bug. Projects document known limitations in the README or security policy and treat them as out of scope, and a report that only restates one of them gets closed. Read the project’s security policy before you claim the fix was incomplete.

Takeaway

A fresh security fix is a map of where a developer was just thinking about a bug class.

Read the patch and check every sink, sibling component and input encoding it did not touch.