Dependency Upgrade Agent With Regression Test Gating

A dependency upgrade agent with regression test gating is trustworthy only when the gate controls its decisions. The silent change in behavior that follows a version bump poses the real risk. Building around that risk means giving the agent evidence to approve, repair, or roll back an upgrade, and clear limits on when it must hand control to a human.

Why dependency upgrades keep failing despite existing automation

A version bump can remove a vulnerable package while breaking the application that uses it. Compilation may fail immediately, or an API change may ripple through application code. PR-opening bots propose the update, but a developer still has to repair what breaks. Their responsibility ends at precisely the point where the difficult work begins. Compilation, test interpretation, patch generation, and rollback fall back to a developer.

Enabling automation across an entire repository set can produce a flood of pull requests that developers learn to ignore, leaving security debt unresolved. Less visibly, unit tests can pass while edge behavior changes under real traffic. CI settings made for bots can also make integration tests that need infrastructure credentials fail or skip, so only lint is green. The missing capability is therefore not more proposals but a reliable repair-and-verify loop.

What makes an upgrade agent different from an upgrade bot

An upgrade agent examines manifests, applies changes, launches regression suites, interprets failures, and patches what broke within one coordinated run. It also generates a human-readable changelog explaining the updated packages and reasons. The agent acts on the outcome of its own work instead of treating a proposed change as a completed task.

LLM Agents for Automated Dependency Upgrades provides a concrete research anchor for this architecture. By contrast, ArXiv 2505.19443 describes the broader distinction between vibe coding and agentic coding. The loop needs clear outcomes, which are pass and merge, patch and retry, or escalate to a human. Without those conditions, the agent can keep working with no control over when it stops. Because the regression suite supplies the evidence for those branches, gating becomes a structural requirement.

Regression gating as the decision engine, not the safety net

A downstream gate shows whether a finished change passed its tests. A decision engine must also tell the agent which tests matter before it commits a patch. TDAD, published in March 2026, addresses this through pre-change impact analysis. It builds a dependency map between source code and tests so the agent knows which tests to verify and can self-correct. That map makes targeted repair possible instead of relying on broad reruns after every attempt.

The distinction matters because “run until green” is not a complete policy. LADU runs until unit tests pass, but those tests may miss problems that arise when the application is deployed. An agent can meet its stopping condition even though it has not shown that the application still behaves as it should. The quality of the gate sets the ceiling on safe autonomy, regardless of how capable the repair model is. Teams must audit coverage before granting more authority, so approval, repair, and rollback rest on evidence strong enough to support them.

How the LADU loop structures the three-agent upgrade cycle

LADU, from J.P. Morgan AI Research, separates the work among a Summary Agent, a Control Agent, and a Code Agent. Its Meta-RAG mechanism gives the Summary Agent a compact representation of the codebase through one-line natural language summaries of functions and classes. The summaries are aligned with the code’s Abstract Syntax Tree, or AST, for structural matching and storage. Separating summary generation from the AST’s structural matching and storage reduces the amount of context needed to work with a large codebase.

After a change, LADU attempts compilation and runs unit tests. When the build fails, error logs return to the Control Agent to refine the fix. A threshold of three consecutive attempts with the same error bounds the repair loop by handing the work to a human developer. It is also a tunable judgment about when the available signal no longer supports self-correction.

Transferable repairs and what BigBag adds to cross-repo upgrade agents

Once the inner loop works, the next problem is repeating it across repositories. The same library update may break several projects, but making a separate patch for each wastes repair work that could be reused. BigBag, published in June 2026, addresses this by generating structured, executable transformation programs that encode repair logic at the API level. Its aim is to apply that logic to other clients broken by the same update.

Transferability depends on how those clients use the affected API. When usage diverges, a shared transformation becomes less useful and project-specific repair becomes necessary. BigBag’s evaluation on the BUMP benchmark distinguishes compilable transformation rate from full fix rate, preserving an important boundary between code that compiles and an update that is repaired. Neither compilation nor success in one project proves behavioral correctness elsewhere. A multi-repo agent should try a transferable transformation first, fall back to local repair when usage differs, and gate every target repository independently.

What the regression gate must actually test to be trustworthy

The gate must test both the upgraded application and the agent that changes it. Unit tests alone do not establish that the orchestration layer preserves state, handles repeated tool calls, or obeys output contracts. Context persistence needs a deliberate test, such as filling a synthetic history to ~80% of the prompt budget and checking retrieval of a fact from the first turn. The result must distinguish retrieval from summarization, because those are different capabilities.

Write operations need an equally concrete check. The gate should send the same tool-call payload three times and check that exactly one write occurs. Idempotency keys should follow the logical identity of the operation. Otherwise, a repeated operation can appear new merely because it arrived through a different part of the run.

Structured output tests must go beyond successful parsing to detect truncation and semantically wrong values, including checks on finish_reason. Prompt injection tests must check tool-call traces and side effects, because a polite refusal does not prove that no harm was done. Because agent behavior is stochastic, pin the model snapshot, set temperature to zero where supported, and use enough trials to establish a confidence-bounded pass rate. A fast gate that retries flaky failures into silence is not a trustworthy gate.

Building the agent loop: components, sequencing, and escalation policy

The practical design is a deterministic backbone with AI-assisted repair where inference is needed. Control flow stays fixed and auditable, while the model interprets failures and proposes patches. Begin by scanning manifests for outdated or vulnerable packages across the target repositories. Then build the source-to-test dependency map before changing the manifest and lock file. Run the scoped regression set and gather the results before deciding what to do next.

When all required checks pass, open a pull request with an auto-generated changelog and flag it for lightweight human review before merge. That review is a gate, while autonomous merging can be introduced later within an approved scope. On compilation or test failure, return the error log to the Control Agent, apply a patch, and rerun the gate. Use a configurable retry budget, with three attempts as the LADU-pattern default. When that budget is exhausted, escalate with a structured failure summary, attempted patches, and test diff.

After merge, run the full suite nightly to catch cross-package interactions missed by the scoped checks. Impact analysis keeps per-PR testing focused without making the scoped set the only source of evidence. For transferable repairs, run version changes, gates, and repair branches concurrently across repositories. Each changelog should record updated packages, previous and new versions, the reason for the change, and test outcomes. This artifact makes decisions reviewable without requiring someone to replay every log.

Persistent state and memory as prerequisites for a multi-session upgrade agent

In-flight state keeps track of the packages being repaired, the last error, and how much of the retry budget is left. Repository history includes previous changes, policy-pinned packages, and subsystem reviewers. Test knowledge distinguishes load-bearing checks from informational ones and records flaky suites that should be quarantined. An escalation queue prevents the agent from starting again on work it has already handed to a human.

This memory can also support changes to operating rules. Procedural memory lets the agent use feedback gathered over time to change escalation thresholds for package ecosystems that keep going over the retry budget. A shared runtime can preserve identity and state across CLI use, Slack interactions, and scheduled cron jobs. Without shared persistence, each surface risks reconstructing the same work from CI logs.

Scheduling makes that continuity operationally useful. The agent should follow a defined cadence, including nightly scans and an immediate response to a new CVE disclosure. Status should appear where engineers already work, through escalation messages, retry-queue inspection, and gate-outcome dashboards that consistently show what is running, what is blocked, and what needs attention.

Where the gate fails silently and how to close the remaining gaps

Even a well-built gate can miss a semantic regression that no test was written to catch. Unit and integration tests may pass while production traffic takes a changed edge path that is missing from the test environment. Supply chain reachability creates a related problem in the other direction. A high-severity dependency may be unreachable from production code, so an upgrade adds activity without materially changing risk.

Within the upgrade loop, span-level evaluation follows tool calls, patch decisions, and test interpretations back to the decision that caused a downstream failure. That yields a bug report with a failure that can be reproduced.

The gate also needs scrutiny of its own. A flaky test must be quarantined before nondeterministic results become accepted evidence of safety. More coverage does not help if the pass signal itself is unreliable. Start with one ecosystem or repository cluster where coverage is known to be strong, then compare gate outcomes with actual behavioral correctness. Widen scope only after that relationship has been established.

Phasing autonomy: how to roll out the agent without recreating the PR flood

The first phase should put the gate ahead of merge authority. Choose one repository with high coverage and low flakiness, then run the agent in observe-only mode for two to four weeks. Check whether a pass predicts a clean production merge. This period also builds an escalation history that reveals which ecosystems should remain under human control.

The next phase grants autonomous merge only for patch-version bumps in dependencies with no transitive impact on public APIs. Minor and major upgrades stay in human-review mode. Multi-repo expansion follows through transferable transformations where API usage is uniform, with independent regression gating in every target repository. The agent gains more autonomy later, as production failures that have been confirmed become permanent regression cases. Authority grows with demonstrated coverage rather than with the number of repositories available.

Throughout the rollout, the runtime should cap concurrency, enforce repository-level rules, and expose the current phase and escalation status through engineers’ working surfaces. Each time a bump is applied, a patch is generated, or an escalation is opened, the agent should create a structured audit artifact. The changelog explains successful decisions, while the escalation summary explains where the agent reached its limit. Those records let an organization understand, challenge, and trust the autonomy it has granted.