---
title: 'GitHub Copilot research experiment: AI pair-programmed HTTP server'
slug: github-copilot-http-server
stable_id: 4900752c0088f2ed
company: GitHub Copilot research experiment
function_code: software_engineering
pattern_codes:
  - creator_to_judge
evidence_strength: verified
publication_tier: showcase
freshness: current
reviewed_at: '2026-08-23'
updated_at: '2026-08-22'
source_quality_summary: 1 independent, 1 primary; publication outcomes are verified.
caveat_summary: Conditioned on completion; narrow task; author affiliations.
collections:
  - human-still-decides
  - embodied-work
bundle_version: 1.0.0
bundle_fingerprint: sha256:c23c6cc2b88153d008ea8fda928f632ce0011fc2d4c5036672a16e5d895bab93
canonical_url: https://brianletort.ai/transformations/github-copilot-http-server
---

# GitHub Copilot research experiment: AI pair-programmed HTTP server

Inline generation can compress a bounded task but not prove lifecycle productivity.

Function: Software engineering. Patterns: Creator to judge. Evidence: verified.

Freshness: current. Reviewed: 2026-08-23. Updated: 2026-08-22.


Source quality: 1 independent, 1 primary; publication outcomes are verified.

## Before

1. **Randomized control developer** — Reads the HTTP-server specification and writes JavaScript without Copilot. (control: Same instructions and JavaScript familiarity requirements as treatment.)
2. **Developer** — Runs tests, debugs, and submits the implementation. (control: Correctness/completeness test suite; no production deployment.)

## After

1. **GitHub Copilot** — Offers inline code completions while the developer implements the same server task. (control: Suggestion only; developer may accept, edit, or reject.)
2. **Developer** — Integrates suggestions, runs tests, debugs, and submits. (control: Developer owns code; automated tests score correctness and completeness.)

## Decision rights

Developer owns code; tests check requirements.

## Exception path

Incorrect suggestions are rejected or corrected.

## Outcomes

- **Completion time** (verified): 160.89 minutes control → 71.17 minutes treatment; 55.8% faster, P=.0017. Conditioned on completion; narrow task; author affiliations.

## Executive lesson

Use for bounded acceleration, not universal delivery productivity.

## Anti-pattern

Do not generalize to maintained systems.

## Questions for leaders

- Where is the operating threshold set and who can override it?
- What measured result would trigger rollback or retraining?
- Which residual decisions must remain human-owned?

## Sources

- [The Impact of AI on Developer Productivity](https://arxiv.org/abs/2302.06590) — arXiv
- [Quantifying GitHub Copilot impact](https://github.blog/news-insights/research/research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/) — GitHub