brianletort.ai
← Library

Anonymous Fortune 500 software company

Verified evidenceCreator to judge

Generative AI agent assistance

AI can diffuse expert practice, with value concentrated among novices.

Software engineering · Multiple locations

An editorial scene for Anonymous Fortune 500 software company contrasts reads the incoming chat, searches available knowledge, and drafts a response. with observes the live chat and proposes response text based on patterns from prior interactions. in the generative ai agent assistance workflow.

Executive brief

The operating-model shift, in one view.

Skill distribution is an operating metric.

AI value · Issues resolved per hour

Verified

About 14% average increase; 35% for novice/low-skill agents

Anonymous company; staggered observational rollout; heterogeneous effects.

Before

Reads the incoming chat, searches available knowledge, and drafts a response. → Resolves the issue or escalates when managerial intervention is needed.

After

Observes the live chat and proposes response text based on patterns from prior interactions. → Accepts, edits, or ignores the suggestion and owns the sent response and escalation.

Human boundary

Agent owns every sent response; managers own escalation.

Why it matters

AI can diffuse expert practice, with value concentrated among novices.

How the work changed

Before

How the work ran before the change.

  1. Step 1 of 2

    Customer-support agent

    Reads the incoming chat, searches available knowledge, and drafts a response.

    ControlControl group had no generative assistant; agents could handle concurrent chats.

  2. Step 2 of 2

    Support agent or manager

    Resolves the issue or escalates when managerial intervention is needed.

    ControlHuman resolution and escalation authority.

What changed

AI can diffuse expert practice, with value concentrated among novices.

Decision rightHuman moves from creator to judge

After

How the same work runs now.

  1. Step 1 of 2

    Generative conversational assistant

    Observes the live chat and proposes response text based on patterns from prior interactions.

    ControlSuggestion only; the working paper does not disclose the company or vendor.

  2. Step 2 of 2

    Support agent

    Accepts, edits, or ignores the suggestion and owns the sent response and escalation.

    ControlHuman acceptance control; effects must be monitored by agent skill because top-agent gains were minimal.

Process model built from the published workflow evidence for Anonymous Fortune 500 software company. Every step, actor, and control appears in full below.
Every step, actor, and control

Exception path

Incorrect suggestions are ignored or escalated.

Work removed

  • Some manual search and drafting

Decision authority

Agent owns every sent response; managers own escalation.

Before

  1. 01

    Customer-support agent

    Reads the incoming chat, searches available knowledge, and drafts a response.

    Control: Control group had no generative assistant; agents could handle concurrent chats.

  2. 02

    Support agent or manager

    Resolves the issue or escalates when managerial intervention is needed.

    Control: Human resolution and escalation authority.

After

  1. 01

    Generative conversational assistant

    Observes the live chat and proposes response text based on patterns from prior interactions.

    Control: Suggestion only; the working paper does not disclose the company or vendor.

  2. 02

    Support agent

    Accepts, edits, or ignores the suggestion and owns the sent response and escalation.

    Control: Human acceptance control; effects must be monitored by agent skill because top-agent gains were minimal.

Work that left the path

  • Some manual search and drafting

Human role before

Customer-support agents read incoming chats, searched available knowledge, drafted replies, and escalated cases requiring managerial intervention.

Human role after

Agents accept, edit or ignore suggestions and own resolution.

AI roleObserves chats and suggests responses in real time.

Outcomes

Issues resolved per hour

Verified

Agents without accessAbout 14% average increase; 35% for novice/low-skill agents

2020–2021 · 5,179 agents

Anonymous company; staggered observational rollout; heterogeneous effects.

What leaders can reuse

Anti-pattern

Do not apply the average uniformly.

Questions

  1. 01Where is the operating threshold set and who can override it?
  2. 02What measured result would trigger rollback or retraining?
  3. 03Which residual decisions must remain human-owned?

Portability conditions

  • Acceptance control
  • Quality monitoring
  • Skill stratification

Reputation risk

low

Evidence and authority

What the public record supports.

Current · updated

2 independent; publication outcomes are verified.

Bundle 1.0.0 · reviewed 2026-08-23 · stable ID 6ff380bc84cd5443

Related transformations

More in Software engineering

Sources

Read the evidence, freshness, caveat, and version policy.