No single AI lab wants a disaster. Ask any engineer inside one and they will tell you they care about safety. And yet the industry keeps moving faster, not slower, even as the systems it builds grow harder to predict.

This isn’t a mystery of bad character. It’s a structure of incentives.

Picture a lake shared by a dozen fishing crews. Each crew knows that if everyone pulls back and lets the fish stocks recover, they’ll all eat better in five years. But no single crew can make that happen alone. If one boat holds back while the others keep hauling, the cautious crew just lands a smaller catch and the fish disappear anyway. So every crew fishes as hard as it can, because restraint by one boat only punishes that boat. The lake empties out, everyone knew it would happen, and nobody could stop it by acting alone.

That’s the shape of the AI race. A recent working paper from the University of Chicago modelled exactly this: as more firms compete to reach advanced AI first, each one shifts a larger share of its resources toward speed and a smaller share toward safety testing, because the firm that spends longest checking its work is the firm that ships last. The researchers describe it as a collective-action problem, which is an academic way of naming a trap nobody escapes by being the good actor alone.

This is why appeals to individual conscience mostly fail here. Several labs have hired brilliant, careful safety researchers. But if their caution slows a product by six months while a competitor ships, the market doesn’t reward the caution. It rewards the release date. Reasonable people making individually sensible choices, at every lab at once, add up to an industry-wide outcome that none of them would pick if you asked them directly and took away the pressure.

Fixing a collective-action problem doesn’t mean asking any one company to be braver or more principled. Problems shaped like this get solved by rules that bind everyone at the same moment, so caution stops being a competitive disadvantage and starts being the baseline. Until something changes the shape of the incentives, don’t expect the pace of the race to change either, no matter how many careful people are working inside these companies.