Trading against Algorithms: Price Dynamics and Risk-sharing in a Market with Q-learners

We study pricing dynamics and risk-sharing in a market with rational investors and a Q-learning trader. The Q-learner’s trading generates a feedback loop in prices: their demand for the risky security depends on their perceived benefit from trading, which in turn, depends on realized returns. We show that this loop generates state-dependent stochastic volatility, predictable returns, and novel price dynamics which depend on the mass and learning rate of the Q-learner. When rational investors have strong risksharing motives for trading, we show that Q-learners can (i) earn trading profits and (ii) improve average investor utility, even though they increase the volatility of prices.