Cornell Study Finds Weak AI Regulation Could Reduce Safety Below No Rules

Weak AI Rules Could Reduce Safety More Than No Rules at All

Policymakers worldwide are hurriedly drafting rules for artificial intelligence, fearing that the technology will become entrenched before oversight catches up. Yet a recent study from Cornell University indicates that a poorly designed regulatory framework might actually make AI products less secure than a completely free market.

The investigation, published in the Proceedings of the National Academy of Sciences, employs a game‑theoretic model to explore interactions between two types of firms: a “generalist” that creates a broad‑capability model and a “specialist” that tailors that model for a specific application such as a chatbot or tutoring system.

According to Benjamin Laufer, the study’s lead author, the model reveals a “free‑riding” dynamic where the upstream provider can shift safety responsibilities onto downstream partners once minimum standards are imposed. “Regulation becomes a lever for the general provider to offload the safety burden onto the downstream specialist,” he explained in a Cornell press release.

In the simulated game, regulators announce a baseline safety threshold before either party invests in performance and safety. Revenue is shared, but the generalist’s earnings depend on the final safety level of the product, not on its own contribution. This incentive structure encourages the upstream firm to minimize its own safety spending, relying on the specialist to meet the mandated level.

When the baseline is low or absent, both parties independently invest in safety because the model assumes higher safety translates into greater revenue. However, once a modest safety rule is imposed on the specialist, the generalist can reduce its own investment, allowing the downstream firm to shoulder the compliance cost alone. The result is a safety outcome that settles at the legal minimum—lower than what would have emerged in an unregulated scenario.

Conversely, the authors found that setting ambitious safety requirements for both the generalist and the specialist can raise overall protection while still boosting profitability for each company. Jon Kleinberg, a co‑author, highlighted that “appropriately designed AI regulation can enable firms throughout the development pipeline to coordinate toward better consumer outcomes, with the rules crafted to support predictable behavior across the chain.”

The study also notes that its two‑player abstraction simplifies a far more intricate supply network, where multiple specialists and base‑model providers operate under diverse jurisdictions. Laufer cautioned, “AI involves a very complicated set of stakeholders and actors that each have their own contributions to the technology. Thoughtful regulation must account for the entire supply chain, not just a single entity.”

These findings suggest that overly simplistic or lightly enforced AI policies risk achieving the opposite of their intended effect, underscoring the need for comprehensive, well‑calibrated regulatory approaches.