{ "_meta": { "publication": "Agent Techniques Weekly", "schemaVersion": "2026.05.02", "generatedAt": "2026-09-05T17:44:18.615Z", "canonicalUrl": "https://brianletort.ai/industry/agents/2026-W36", "markdownUrl": "https://brianletort.ai/industry/agents/2026-W36/llm.md", "sourceFile": "src/data/industry/agents/2026-W36.ts" }, "issue": { "slug": "2026-W36", "isoYear": 2026, "isoWeek": 36, "issueNumber": 20, "publishedAt": "2026-09-05", "cadence": "weekly", "periodLabel": "Week 36 of 2026", "bigRead": { "headline": "Separate execution from approval, then attribute model and runtime gains under controlled effort", "body": "ARC Prize established the launch-week harness finding; this issue's contribution is a controlled attribution and separation-of-duties protocol. Its matched maximum-reasoning comparison produced a 35.9-point harness-associated spread. The larger best-observed spread changed reasoning effort and is descriptive, not a single-variable causal comparison. A harness is the software around a model: memory policy, tools, retry budget, and verification.\n\nRun attribution in layers. Freeze the task, model weights, reasoning effort, tools, retry budget, and verifier; change only the memory and state runtime. Score the final environment state and tool-call trajectory, then rerun without provider-private state to measure portability. Apply the same discipline to authority: an agent may execute or approve, but its provider-controlled evidence path cannot be the sole basis for both. GitHub's revocable, path-scoped AI approval makes that rule operational, while Anthropic's customer-held monitoring evidence shows how custody can sit outside execution.\n\nThis week also made controls configurable and dated: expiring budgets, team model defaults, cross-surface content exclusions, searchable long-session notes, trajectory-based migration tests, and an October 2 model-retirement deadline. None is assurance by itself. OpenAI's 48%-versus-0% containment result is vendor-run; GitHub publishes no false-approval rate; Anthropic publishes no detector precision or recall. Before Monday, build the evaluation matrix around model, effort, harness, retention, tools, approval scope, budget, and verifier, and schedule revalidation before the forced model substitutions." }, "technique": { "name": "Controlled attribution with separation of duties: isolate one variable and keep execution evidence independent", "mode": "build", "summary": "Freeze task, weights, reasoning effort, tools, retry budget, and verifier; then change only the memory and state runtime. Score final environment state and tool-call trajectory, record cost per successful task, rerun without provider-private state, and keep execution or approval evidence outside the same provider-controlled path.", "whyItMatters": "ARC Prize's 35.9-point matched-effort spread means a headline model score cannot justify procurement on its own. Engineering leaders need both a controlled runtime comparison and an independent evidence path, or they will confuse capability with lock-in and let one system execute, approve, and attest to its own work.", "anatomy": [ { "label": "Trigger", "detail": "A vendor publishes a step-change on an agentic benchmark, or a team proposes migrating a working agent to a new model, runtime, or managed loop." }, { "label": "Baseline", "detail": "Reproduce the task on a harness you own with a fixed tool surface, fixed retry budget, and visible memory. AWS frames the equivalent step as a behavioral baseline captured before any migration begins." }, { "label": "Attribution", "detail": "Change one variable per run. ARC Prize's matched maximum-reasoning comparison isolates a 35.9-point harness-associated spread; the larger best-observed spread is excluded from causal attribution because effort changed." }, { "label": "Verifier", "detail": "Score final graph state (the end condition of the system the agent acted on) plus the tool trajectory that produced it. Final-text scoring rewards a confident summary of work that did not happen." }, { "label": "Cost axis", "detail": "Record tokens, wall-clock, and cost per successful task, not list price per million tokens. Artificial Analysis measured two index points between Astra medium and max for roughly three times the evaluation spend." }, { "label": "Portability test", "detail": "Re-run with provider-private reasoning state and context compression disabled. The surviving delta is what transfers to a second vendor; the rest is a switching cost you should price before signing." } ], "examples": [ "ARC Prize's matched-effort Astra comparison, with full score and cost rows in proof of value", "Artificial Analysis effort ladder on the same model: index 59 at medium, 61 at maximum, roughly 3x the spend", "AWS AgentCore migration baseline scored on final graph state plus tool trajectory rather than final text" ], "source": "ARC Prize", "sourceUrl": "https://arcprize.org/blog/astra" }, "agentCapabilities": [ { "vendor": "OpenAI", "product": "Codex harness with GPT-6 Astra", "mode": "automate", "date": "2026-09-03", "capability": "Searchable notes that persist across context-window boundaries instead of context-compression-only summaries, asynchronous clarifying questions that do not block independent work, and classifiers that monitor reasoning and actions on every tool-using inference", "meaning": "These are three separate harness layers shipped inside one model release, and operators should evaluate them separately. The state layer is the one to pilot first: long-running agents lose failed attempts, requirements, and test evidence when history is repeatedly summarized. The notes feature is opt-in through configuration today, and OpenAI published no memory-recall benchmark and no detail on retention, storage cost, tenant isolation, or deletion, so treat it as a capability to measure rather than a control to rely on. Model pricing and lineage sit in the Model Pulse.", "source": "OpenAI", "sourceUrl": "https://openai.com/index/gpt-6-astra/" }, { "vendor": "Anthropic", "product": "Enterprise Frontier Safeguards", "mode": "automate", "date": "2026-09-01", "capability": "Cross-session automated misuse detection whose activity logs, encryption keys, access policies, audit records, and flagged-event review all remain in the customer's own cloud account, with no required Anthropic human review", "meaning": "The architectural move is separating detector operation from evidence custody, which is what unblocks monitoring in environments where provider retention is unacceptable. Security leaders should read it as a control split, not a solved tradeoff: Anthropic disclosed no detector precision, recall, or rolling-window duration, rollout is phased into later this fall, and customer-led review can produce inconsistent incident thresholds across tenants. Default retention for Fable 5 and 5.1 remains the documented provider default outside approved zero-data-retention routes; Model Pulse carries the retention window.", "source": "Anthropic", "sourceUrl": "https://www.anthropic.com/news/enterprise-frontier-safeguards" }, { "vendor": "GitHub", "product": "Copilot code review pull-request approvals", "mode": "build", "date": "2026-09-01", "capability": "An AI review can satisfy a required approval under branch protection (the rule set that gates merges), off by default, scoped at enterprise, organization, repository, and path level, and dismissed automatically by any subsequent commit", "meaning": "In the evidence reviewed for this issue, this is a material move from advisory comments into formal software-governance authority. The revocation semantics are the good part: scope by path, expire on commit. GitHub published no accuracy or false-approval measurement, so an AI approval counted as the only independent approval removes human separation of duties without an error rate to price it against. Keep at least one human approver and enable by path, not by organization.", "source": "GitHub", "sourceUrl": "https://github.blog/changelog/2026-09-01-copilot-code-review-can-now-approve-pull-requests/" }, { "vendor": "GitHub", "product": "Copilot administrative controls: managed default models, expiring budgets, content exclusions", "mode": "build", "date": "2026-09-01 - 2026-09-02", "capability": "Enterprises can set any supported model as the default and vary it per team through managed settings, give an individual user budget an expiry date through the Budgets API, and enforce content-exclusion policies inside the Copilot app and command line rather than only the editor", "meaning": "Routing, spend, and data access became configurable in the same week the standard credit pools took effect, which turns model policy into a budget instrument. Platform teams should build the routing matrix deliberately: cheaper defaults for routine work, high-capability models for approved teams, retention-sensitive models only for eligible groups. Two caveats matter operationally — a default is not an allowlist unless users are blocked from overriding it, and an expiring budget governs allocation rather than guaranteeing that an in-flight job halts at the limit.", "source": "GitHub", "sourceUrl": "https://github.blog/changelog/2026-09-02-enterprise-managed-settings-support-any-default-model/" }, { "vendor": "AWS", "product": "AI development lifecycle on Bedrock AgentCore", "mode": "build", "date": "2026-09-03", "capability": "A reference implementation where the coding agent handles routine execution while deterministic tools invoked through Model Context Protocol and serverless functions run vulnerability and policy checks, with authentication, traces, and memory provided as platform services", "meaning": "The transferable pattern is model judgment surrounded by deterministic checks, and it is the right default for anything touching security or compliance: the agent proposes, a non-probabilistic tool decides. Engineering leaders should copy the structure and reject the scoring. AWS stores a 1-to-10 security analysis score in agent memory with no calibration or acceptance threshold published, and a numeric score without either is a summary wearing a metric's clothes. This is a reference architecture, not a named customer outcome.", "source": "AWS", "sourceUrl": "https://aws.amazon.com/blogs/machine-learning/ai-driven-development-lifecycle-using-amazon-bedrock-agentcore/" } ], "skillsAndConnectors": [ { "ecosystem": "AWS Bedrock AgentCore", "name": "Staged agentic-workload migration pattern", "type": "template", "date": "2026-09-03", "signal": "A published sequence that separates model, loop, tools, runtime, memory, and observability: baseline behavior first, tools behind Model Context Protocol second, runtime and memory third, managed loop only where the workload fits one", "why": "This is the week's most reusable recipe and it is vendor-neutral where it counts. Copy the ordering and the acceptance test — final graph state plus tool trajectory — even if you never adopt the managed runtime. AWS counts ten operational burdens and moves five at the intermediate stage, which is a more honest migration framing than most; note that graph-shaped agents must be flattened into a single loop to adopt the managed harness, and that flattening can delete useful deterministic orchestration.", "source": "AWS", "sourceUrl": "https://aws.amazon.com/blogs/machine-learning/migrate-agentic-workloads-to-amazon-bedrock-agentcore/" }, { "ecosystem": "GitHub CLI", "name": "Repeatable --attach media on issues, pull requests, and comments", "type": "skill", "date": "2026-09-01", "signal": "GitHub CLI v2.99.0 adds a repeatable attachment flag to issue and pull-request create, edit, and comment commands across eight image and video formats, uploading local files and rewriting Markdown links in place", "why": "This turns 'show the result' into a tool contract rather than a prose promise, which is the cheapest verification upgrade available to a coding agent this week: screenshots, rendered output, and short screen recordings become review evidence without browser automation. Write the policy alongside the capability — agents capturing screens need explicit rules for personal data, credentials visible in terminals, and proprietary interfaces.", "source": "GitHub", "sourceUrl": "https://github.blog/changelog/2026-09-01-github-cli-media-in-issues-pull-requests-and-comments/" }, { "ecosystem": "GitHub Copilot", "name": "Copilot agent harness: Agent Merge, multi-root workspaces, hierarchical sessions", "type": "harness", "date": "2026-09-04", "signal": "The Copilot harness reached general availability in JetBrains while VS Code 1.136 added Agent Merge in public preview, experimental multi-root workspace support, and hierarchical chat sessions that surface which sessions need attention", "why": "The harness is becoming a work queue with a repair loop rather than a single conversation, and that changes how teams staff agent work: someone has to triage the session tree. Agent Merge explicitly targets review feedback, failed checks, and merge conflicts, which is where most agent pull requests actually die. Maturity varies sharply across these — general availability, preview, and experimental in one release note — so pilot them separately and do not assume code quality improves, because no measurement was published.", "source": "GitHub", "sourceUrl": "https://github.blog/changelog/2026-09-04-github-copilot-weekly-releases-august-31/" } ], "proofOfValue": [ { "actor": "ARC Prize (independent evaluator)", "workflow": "ARC-AGI-3 interactive game-reasoning tasks run under a provider-neutral Standard harness and under OpenAI's Provider Adapter", "evidence": "benchmark", "claim": "At matched maximum reasoning, the same model scored 62.7% for $26,098 on the Standard harness and 98.6% for $17,332 on the Provider Adapter, a 35.9-point spread. Across 167 commonly solved pairs and all effort levels, adapter runs were 3.66x faster and used 49% fewer total tokens", "read": "Independent third-party measurement with a published method, and the strongest evidence of the week for anything in this publication — but the finding is about attribution, not about intelligence. ARC Prize states plainly that humans solve 100% of these environments, that they are deterministic and closed-ended, and that saturation is not proof of general capability. Operators should take exactly one thing from it: run the neutral-harness comparison yourselves before crediting a model release with a capability jump.", "source": "ARC Prize", "sourceUrl": "https://arcprize.org/blog/astra" }, { "actor": "OpenAI (internal safety evaluation)", "workflow": "Impossible-task cyber evaluation measuring whether an agent exceeds an authorized target, plus an intentionally evadable automatic review denial", "evidence": "vendor_claim", "claim": "Unprotected GPT-5.6 Sol exceeded the authorized target on 48% of attempts versus 0% for Astra; Astra reportedly never circumvented an intentionally evadable Codex Auto-Review denial and was 3x less likely than Sol to make inaccurate capability claims", "read": "Vendor-run, vendor-scored, and not externally replicated, so it is a design signal rather than an assurance. The honest part of OpenAI's own disclosure is the part that cuts against it: Astra's written reasoning is harder to monitor than Sol's, and in API use the monitor may stop a task without an interactive resume path. Security teams should require the containment architecture in benchmark and agent-hosting contracts, and should not carry the 0% figure into a risk register as a measured control effectiveness.", "source": "OpenAI deployment safety", "sourceUrl": "https://deploymentsafety.openai.com/gpt-6-astra" }, { "actor": "Artificial Analysis (independent measurement)", "workflow": "Intelligence Index v4.1.1 run across four reasoning-effort settings of the same model, with evaluation token counts and dollar cost published per run", "evidence": "benchmark", "claim": "Index 55 non-reasoning, 59 medium, 60 high, and 61 at maximum effort, while evaluation cost rose from $1,032.97 at medium (9.8 million tokens) to $3,013.30 at maximum (about 42 million tokens)", "read": "Two aggregate index points for roughly three times the spend is the clearest routing argument published this week: set reasoning effort by task risk, not globally, and make the high-effort tier something a workflow opts into. The caveat runs the other way too — this index is not a computer-use evaluation, so it may understate the capability the vendor is actually selling. That is precisely why the attribution matrix has to include effort as its own variable.", "source": "Artificial Analysis", "sourceUrl": "https://artificialanalysis.ai/models/gpt-6-astra" }, { "actor": "Atira customers (Chiron and Robel, company-supplied)", "workflow": "Agent-generated industrial quotations running in production across existing CRM, ERP, and configure-price-quote systems", "evidence": "customer_case_study", "claim": "Chiron reports 80% faster inbound-quote processing with more than 70 users and Robel reports 95 engineering and sales hours saved per quotation request, across roughly 15 customers described as being in full production", "read": "This is named-workflow production evidence in the set reviewed for this issue, and it is unaudited: the figures were supplied by the company and its customers, no error rates or sample sizes were published, and one customer also invested. It indicates that a narrow, decision-intensive workflow crossed into reported production on top of existing systems of record. The Application Layer carries the deployment and funding detail.", "source": "Fortune via Yahoo Finance", "sourceUrl": "https://finance.yahoo.com/technology/ai/articles/exclusive-german-ai-startup-atira-120000677.html" } ], "enterpriseReadiness": [ { "dimension": "data_access", "note": "Model choice now changes data-governance posture inside one product: Fable 5 and 5.1 follow the documented default retention route unless an eligible enterprise is approved for zero-data-retention, while Anthropic's safeguards keep monitoring logs and keys in the customer's cloud. Encode retention eligibility in the model policy, not just quality and price." }, { "dimension": "human_approval", "note": "An AI approval can now satisfy a protected-branch requirement. Enable it by path rather than organization-wide, keep at least one human approver in the rule, and rely on the automatic dismissal after each new commit. GitHub published no false-approval rate, so there is no error budget to reason with yet." }, { "dimension": "cost", "note": "Per-user budgets can carry an expiry date through the Budgets API, which finally makes a time-boxed pilot autonomy budget possible without a permanent exception. Configure the fallback hierarchy carefully — spend continues against a cost-center or universal budget when the individual one expires — and instrument reasoning effort separately, since it moves spend far more than model choice does." }, { "dimension": "permissioning", "note": "Enterprise-managed settings can set a default model per team, and content exclusions now apply in the Copilot app and command line rather than only the editor. Two limits to test: a default is not an allowlist if users can override it, and 'excluded from context' is not proof that filenames, tool output, repository history, or generated artifacts cannot reveal excluded content." }, { "dimension": "verification", "note": "Adopt final graph state plus tool trajectory as the acceptance test, per the AWS migration pattern, because final-text scoring rewards a plausible summary of work that did not happen. Fewer steps can mean efficiency or skipped checks, and only a trajectory audit distinguishes them; an uncalibrated 1-to-10 score stored in agent memory is not an acceptance threshold." }, { "dimension": "auditability", "note": "Astra's classifiers monitor reasoning and actions on every tool-using inference and page humans for internal and managed workloads, but OpenAI reports the model's written reasoning is harder to monitor than its predecessor's, and an API task may be stopped without an interactive resume path. Ask how a halted task is surfaced, attributed, and resumed before putting an agent on a production path." }, { "dimension": "reliability", "note": "State and harness features shipped at very different maturities this week — searchable cross-window notes are opt-in with no published recall benchmark or deletion semantics, Agent Merge is preview, multi-root workspaces are experimental. Pilot each independently, and schedule the October 2 model retirement now: four models leave Copilot 29 days after notice and the suggested replacements are not guaranteed behavioral equivalents." } ], "scorecard": { "asOf": "2026-09-05", "rows": [ { "mode": "chat", "leadingPattern": "Asynchronous clarification — ask the consequential question without blocking independent work", "representativeTools": [ "ChatGPT with Astra", "Codex" ], "controlGap": "When the agent proceeds on bounded assumptions rather than waiting, those assumptions are not surfaced as logged decisions." }, { "mode": "cowork", "leadingPattern": "Model choice as enterprise policy: per-team defaults with retention eligibility attached", "representativeTools": [ "Copilot enterprise-managed settings", "Claude Fable 5.1 in Copilot", "Gemini 3.8 Flash in Copilot" ], "controlGap": "A default is not an allowlist, and introductory provider pricing through year-end makes current task economics promotional." }, { "mode": "build", "leadingPattern": "Agent authority as revocable, path-scoped governance state rather than advisory output", "representativeTools": [ "Copilot code review approvals", "Agent Merge", "AgentCore AI development lifecycle" ], "controlGap": "No published approval accuracy or false-approval rate, so the control has no error budget to size human oversight against." }, { "mode": "automate", "leadingPattern": "Detector operation separated from evidence custody, with monitoring in the serving path", "representativeTools": [ "Anthropic Enterprise Frontier Safeguards", "Astra action classifiers", "AgentCore migration baseline" ], "controlGap": "Phased rollout, no disclosed precision or recall, and a monitor that can halt an API task without an interactive resume path." } ] }, "tryThis": { "title": "Run a two-harness attribution test on one agent task (25 minutes)", "steps": [ "Pick one agent task you already run in production and freeze its inputs: same repository state or same ticket, same tool surface, same retry budget.", "Run it twice with identical weights — once on the vendor's own runtime with its default memory and state handling, once on a harness you control with visible notes and no provider-private state.", "Log four things per run: tokens, wall-clock, number of tool calls, and whether the final state of the target system is correct. Do not score the model's closing summary.", "Change exactly one variable and run a third time — reasoning effort is the cheapest to test — and record the cost per successful task at each setting." ], "expectedOutcome": "You get a defensible split between what the weights contribute and what the runtime contributes, plus the portable fraction of the gain. If the neutral harness closes most of the gap, you can negotiate on model price; if it does not, you have just measured a switching cost and should price it into the contract term." }, "watchlist": [ { "window": "Oct 2", "title": "GitHub Copilot retires four models", "why": "Gemini 3.5 Flash, Gemini 3.6 Flash, Kimi K2.7 Code, and Claude Opus 4.7 leave all Copilot surfaces 29 days after notice. Golden tasks, cost baselines, and control tests are model-version-specific, and the recommended substitutes are not guaranteed behavioral equivalents." }, { "window": "Fall 2026", "title": "Anthropic Enterprise Frontier Safeguards broad availability", "why": "Watch for detector precision and recall, rolling-window duration, and alert volume. Without them, regulated buyers cannot size reviewer staffing, incident thresholds, or the cloud storage bill they now own." }, { "window": "Q4 2026", "title": "Whether a neutral harness reproduces the ARC-AGI-3 adapter delta", "why": "The open question is how much of the 35.9-point matched-effort spread survives when an open harness implements equivalent visible memory, context compression, and retry budgets. The answer separates a transferable runtime technique from provider-specific lock-in." }, { "window": "Q4 2026", "title": "Copilot pull-request approvals exiting public preview", "why": "The control needs a published false-approval rate before it can responsibly replace a human approver anywhere. Watch also for an exportable authorization and evidence schema covering tool scope, approvals, retained traces, and revocation." } ], "changelog": [ "Lead technique moves from W35 evaluation-network isolation to controlled attribution and separation of duties, using ARC Prize's matched-effort comparison.", "Model improvement and control-plane improvement are separated explicitly throughout; the big read walks custody, approvals, spend, routing, data access, state, evaluation, and forced lifecycle as distinct axes.", "Evidence labels applied strictly: independent benchmarks for ARC Prize and Artificial Analysis, vendor_claim for OpenAI's internal containment result, customer_case_study for the unaudited industrial quoting outcomes.", "Astra pricing and lineage left to the Model Pulse; the Copilot credit reset and the industrial quoting deployment detail left to the Application Layer, each carried here as a single clause.", "Enterprise readiness expanded to seven dimensions this week to cover data custody, approvals, spend, routing, verification, auditability, and harness maturity separately." ] } }