AI Results

A practitioner guide covering when a workflow needs an agent, how tools and guardrails are arranged, and how handoffs to people work.

A vendor guide: the architecture is sound and the examples are its own platform. No outcome figures to check.

A set of labelled diagrams for the shapes these products take: prompt chaining, routing, parallelisation, orchestrator and workers, evaluator and optimiser, and the autonomous agent.

Diagrams and advice, not measurements. Its most useful line is that most production value came from simple composable patterns rather than autonomous agents.

In its first month the assistant handled two-thirds of Klarna’s customer service chats.

• 2.3 million conversations in the first month
• The equivalent work of 700 full-time agents
• On par with human agents for customer satisfaction
• A 25% drop in repeat inquiries
• Live in 23 markets, in more than 35 languages

The company’s own press release, one month in, with no control group. Satisfaction is reported as on par rather than better, and resolution counts chats closed, not problems solved.

Inside the frontier of what AI does well, consultants completed 12.2% more tasks, 25.1% faster, at higher quality. Outside it, they did worse.

• 758 consultants at BCG
• 25.1% faster on tasks inside the frontier
• Significantly higher quality inside the frontier
• On a task chosen to sit outside it, AI users were more often wrong

The headline gain and the headline harm come from the same experiment. The lesson is the boundary, not the percentage: people could not feel where it was.

Access to an AI assistant raised issues resolved per hour by 14% on average.

• 5,179 customer support agents
• +34% for novice and low-skilled workers
• Minimal impact on experienced, highly skilled workers

The most useful finding is the split: the average hides a large gain for novices and almost none for experts. One firm, one kind of support work.

Developers with an AI pair programmer completed a task 55.8% faster than the control group.

• 95 professional developers, randomly assigned
• Larger gains for less experienced developers

A randomised trial, which is the strong part, but one self-contained task and authors from GitHub and Microsoft. Speed on a fresh task says nothing about maintaining the result.