This paper formalizes the idea that more hedging instruments may destabilize markets when traders are heterogeneous and adapt their behavior according to experience based reinforcement learning. We investigate three different economic settings, a simple mean-variance asset pricing model, a general equilibrium two-period overlapping generations model with heterogeneous expectations and a noisy rational expectations asset pricing model with heterogeneous information signals. In each setting the introduction of additional Arrow securities can destabilize the market, causing a bifurcation of the steady state to multiple steady states, periodic orbits or even chaotic fluctuations.