How Six AWS Engineers Rebuilt Bedrock to Challenge Microsoft

Bedrock’s original architecture appears to have made a category error. It treated inference too much like a conventional web service. But modern AI traffic has bursty workloads, scarce accelerators, long-running agents and jobs with radically different priority levels.

Six senior AWS engineers built Project Mantle around a different abstraction: inference as a scheduling system. Customers can assign task priorities; workloads are isolated so one tenant’s burst does not degrade another’s service; and AWS’s Journal system checkpoints long-running agents so they can resume after failure instead of restarting. Mantle also added OpenAI-compatible APIs.

The Information reports that the team used Amazon’s own Kiro coding agent heavily. A former employee says capacity-expansion waits fell from roughly three weeks to two days. Bedrock now runs much of this workload on Amazon Trainium as well, giving AWS another supply source beyond Nvidia.

“AI infrastructure” is evolving from GPUs behind HTTP endpoints into durable, stateful operating systems for huge numbers of concurrent agents.