Agent Capability Is Not Operating Authority

Four verified developments expose different gaps between an agent’s apparent capability and the evidence needed to authorize it for consequential work. One research evaluation finds that frontier agents can complete substantial engineering while failing the central judgment demands of open-ended research. A second paper limits tool acquisition according to financial, context, and privacy costs rather than relevance alone. GitHub is making parallel agent sessions, isolated worktrees, and visible subagent activity part of a mainstream development interface. A fourth paper proposes a readiness index that keeps governance evidence visible alongside behavioral evaluation.

The sources do not establish a complete production-governance model. The open-ended research result is based on two case studies. The tool-acquisition method optimizes within author-defined costs and ranked candidate sets. GitHub’s July announcement consolidates features shipped throughout the month, some still in public preview or experimental status. The proposed readiness index is an author-defined framework whose held-out validation claims require independent replication.

Engineering completion does not establish strategic competence

Can AI agents conduct open-ended AI research? Early evidence from two case studies (opens in a new tab) evaluates frontier agents on the central research questions of two unpublished NeurIPS 2026 submissions. The original paper authors graded the resulting work. Agents received six days and thousands of dollars of compute, completed the engineering without human assistance, but did not make substantial progress on the core research questions. Both outputs were rejected by the authors.

The paper identifies five recurring failure modes: poor judgment about the standard for publishable research, uncreative responses to weaknesses in the research design, ineffective backtracking from dead ends, poor resource awareness, and instruction drift. A robustness check using a second model and scaffold reproduced the failures. The authors release expert reviews, survey responses, repositories, and logs.

This creates an important distinction between execution and judgment. Long-running autonomous activity, extensive tool use, and the completion of technically demanding subtasks can all be genuine capabilities without establishing that an agent can recognize whether its strategy is answering the right question. A system may therefore look productive while consuming resources along a weak path.

The evidence is deliberately early and narrow. Two case studies cannot establish a general failure rate for AI research agents, and the grading depends on the original authors’ assessment of unpublished work. The agents were also evaluated on unusually open-ended questions with substantial compute budgets. The result should not be generalized into a claim that agents cannot contribute to research. It supports a narrower governance principle: successful task activity is not sufficient evidence for authority over strategic decisions whose quality is difficult to verify automatically.

Tool access should be bounded by the task’s actual exposure

Scores Are Not Decisions: Cost-Aware Stopping for Tool Acquisition in LLM Agents (opens in a new tab) addresses a pre-execution decision that relevance ranking does not solve: how many external tools should an agent acquire for a task? Too few tools may leave the agent under-informed. Too many can add financial cost, context load, and privacy exposure.

The paper formulates tool selection as cost-aware stopping over ranked tool prefixes. Its CAM-DF method learns whether the expected value of continuing to acquire tools exceeds the value of stopping, while weighting errors by the payoff at stake. The authors evaluate 1,343 tasks across five tool-use domains. In live execution, they report exposing agents to 37% fewer tools than full access while maintaining comparable task success.

The governance signal is not that this particular optimizer should determine authorization. It is that a list of permitted tools defines a maximum boundary, not necessarily the correct task-level grant. An agent authorized to use ten connectors may need only two for a specific job. Reducing unnecessary acquisition can narrow cost, context contamination, and information exposure before execution begins.

The reported results depend on the paper’s payoff functions, cost assumptions, ranking sources, and task domains. A privacy cost represented numerically in an optimization objective is not a substitute for a hard prohibition, consent requirement, or deterministic access policy. Cost-aware stopping is best understood as a decision layer inside an already valid authorization envelope: it can reduce unnecessary exposure but should not price its way around non-negotiable controls.

Isolation and subagent visibility are becoming interface expectations

GitHub Copilot in Visual Studio Code, July 2026 releases (opens in a new tab) consolidates changes shipped across VS Code versions 1.127 through 1.131 during July. The Agents window can start Copilot, Claude, or Codex sessions in separate Git worktrees, allowing multiple agent harnesses to operate on isolated repository copies. It also exposes each running subagent’s model, elapsed time, active tool call, and conversation.

The release adds management for multiple related chats, including peer-chat forks that preserve earlier context, and extends bring-your-own-key model use into the Copilot agent within the Agents window. GitHub also describes faster change review and in-chat handling of failed checks and review comments.

These are interface and workflow capabilities rather than guarantees of governed execution. A worktree isolates filesystem changes; it does not establish identity, authority, secret containment, network restrictions, or approval policy. Visible tool activity improves operator awareness but does not prove that an action was authorized or that the displayed event is a complete audit record. Several Agents window features remain in public preview, while other items are explicitly experimental.

The competitive signal is nevertheless material. Parallel sessions, execution isolation, subagent observability, preserved branching context, and consolidated review are becoming baseline expectations in a widely used development environment. A governance-oriented control plane must make authority and consequential state at least as understandable as mainstream tools make agent activity. The differentiator is not the presence of multiple sessions; it is whether their permissions, approvals, dependencies, and realized effects remain inspectable.

GitHub’s separate managed-device restriction for Copilot remote control (opens in a new tab) reinforces the distinction between capability and authorized operating context. Organizations can constrain remote control through managed settings and organisational sign-in requirements. It is a supporting control, not evidence of a complete authorization architecture.

Readiness evidence should not disappear into a capability score

Stop Shipping AI Agents on Faith: Capability Is Not Production Readiness (opens in a new tab) introduces the ProofAgent Index, an author-defined governance-readiness framework spanning four dimensions: Evaluation, Context, Compliance, and Governance. The Governance dimension asks whether an organization can authorize, monitor, audit, and control an agent during operation.

The paper’s central framing is useful because it resists collapsing deployment evidence into a single capability result. Behavioral performance may improve while the surrounding operating context, compliance obligations, approval structure, or auditability remains inadequate. The canonical listing says the index is implemented in the open-source ProofAgent Harness and reports held-out readiness signal across healthcare and finance configurations. It further argues that governance evidence should remain visible rather than being averaged away.

Those claims require careful qualification. The index is designed and evaluated by the ProofAgent author, and the canonical release record does not by itself establish independent validation, regulatory acceptance, or production effectiveness. The weighting of its dimensions, construction of the held-out tests, definition of higher- and lower-risk configurations, and sensitivity to missing evidence all need scrutiny. A composite readiness index may still conceal a critical failure if organizations treat its aggregate output as authorization.

The durable contribution is the proposed separation of evidence classes. Capability evaluation asks what an agent did under test. Context describes the environment shaping that behavior. Compliance maps applicable obligations and controls. Governance asks who can authorize, observe, interrupt, audit, and remain accountable for operation. Even if the specific index changes, those questions should remain independently inspectable.

Keelbase Signal assessment

Together, the four records support four boundaries that should remain distinct:

• A competence boundary should distinguish completed activity from reliable strategic judgment.
• An acquisition boundary should constrain task-level tool exposure within the larger authorization envelope.
• An execution boundary should isolate concurrent work and make subagent activity visible without confusing visibility with proof of authority.
• A readiness boundary should preserve governance, context, and compliance evidence alongside capability evaluation rather than hiding them inside an aggregate score.

For Keelbase, this reinforces the difference between what a Vessel’s crew can attempt and what it is authorized to do. Capability evidence can inform role and specialist selection, but it should not replace approval policy or deterministic limits on consequential actions. Tool permission should define the valid perimeter, while task-level acquisition can narrow exposure further. Concurrent agent work should be isolated and observable, with consequential outcomes recorded through the coordination contract and AnchorLog rather than inferred from an activity display.

The readiness framing also fits Keelbase’s claim discipline. Structural governance and auditability can be described through visible authorization, monitoring, approval, and record-keeping mechanisms. They should not be presented as proof of runtime behavioral safety. No benchmark score, interface feature, or readiness index removes the need to inspect the operating authority granted to an agent and the evidence retained after it acts.

Agent capability is therefore a necessary input to deployment, not the deployment decision itself. Governed operation requires evidence that the agent is competent for the work, limited to justified tools and context, isolated where parallel execution creates risk, and subject to controls that remain visible before, during, and after consequential action.