Yves here. Rajiv Sethi describes how AI can and almost surely will manipulate markets. We already have thanks to what was once called black box and now algorithmic trading. But differences in degree are difference in kind. What happens to markets when human views have no weight and AI persistently engages in trading games as opposed to having any concern about fundamentals? How can markets like that provide valid input for real economy decisions? It would seem destined to turbo-charge financialization, which is already so significant as to lower growth and greatly increase inequality.
By Rajiv Sethi, Professor of Economics, Barnard College, Columbia University; External Professor, Santa Fe Institute. Originally published at Imperfect Information
Like countless other folks I’ve been trying to grapple with the implications of what happened at OpenAI over the past couple of months—agents broke out of solitary confinement, established communication channels with each other, found ways to access the internet, colluded to breach servers at another company, gained access to credentials and private data, and took active steps to cover their tracks.1
I’ve been meaning to take a break from posting here in order to focus on my book on prediction markets, but there’s something about this incident that seems to have been missed in most of the reporting, so I thought I would add my two cents. In addition, there’s a chapter in the book on the future of markets dominated by AI agents, and this post is a useful way to flesh out my thinking on the topic.2
The agents in the OpenAI incident were assigned tasks that required finding and exploiting a software vulnerability in order to retrieve a hidden piece of data or “flag.” Some of these tasks were impossible to complete given the constraints under which agents were operating, so they found a way to circumvent those constraints. But here is the crucial point—the success of any given agent in completing its assigned task did not inhibit any other agents from completing theirs. Quite the opposite in fact. The path taken by any one agent could, in principle, point the way for other agents to succeed.
Now consider prediction markets, which are zero sum environments in which one trader’s success has to come at someone else’s expense. AI agents are already achieving levels of predictive accuracy that match or exceed those of the most skilled human forecasters, and traders relying on AI agents have achieved spectacular rates of return in asset markets. It’s only a matter of time before trading comes to be dominated by artificially intelligent agents. The capital at risk will belong to a human being or a conventional organization, but real time authority for making transactions will be delegated to agents. Others will simply be too slow to compete.
How will such a market behave? The first thing to note is that agents will be incentivized to pursue profitability rather than accuracy, and these are not the same thing. An agent may have computed the probability of a referenced outcome in a market, but will also try to infer from market data what kinds of estimates other agents have arrived at. Furthermore, each agent will realize that it can influence market data in ways that trigger other agents to react, and doing so may be more profitable than simply trading based on current prices and long term beliefs. Human traders have engaged in spoofing to profit from market reactions; AI agents will be far more adept at doing so.
Agents will also seek out hidden information to gain an edge, even if this involves hacking into systems to extract material non-public information. We already have plausible evidence of auditors trading ahead of earnings calls, and based on the capabilities demonstrated by the OpenAI agents, accessing such information would be a trivial task.
Could one not hold the individual or entity that is delegating trading decisions accountable for violations of the law? Possibly, but this requires identity verificationon prediction market platforms, which is far from universal. There is also the question of intent—even agents constrained to follow the law may violate it and cover their tracks, if they consider themselves incentivized to do so.
What about the role of humans in all of this? One of my most vivid memories from the flash crash of 2010 is a video clip of Jim Cramer watching the price of Proctor & Gamble fall suddenly from 62 to 42 dollars a share. He instantly recognized that this was “not a real price” and that viewers should pounce on the opportunity. Someone did, and the price was back up to above 60 within a minute.
Perhaps this is where we are headed. Humans watching AI agents trade with each other and keeping an eye out for ruptures to exploit. Not the prettiest of pictures, but at least we will not be completely dispensable.
________
1 The key reports are by OpenAI and METR, with very informative summaries by Ajeya Cotraand Zvi Mowshowitz for those (like myself) with limited expertise on such matters.
2 I spent a very productive hour with G. Elliott Morris on his podcast a few days ago; we covered a range of topics related to prediction markets, but didn’t discuss the impending dominance of AI agents in trading.
By
