We study the dynamics of an optical mode in a cavity with a movable mirror subject to quantum Brownian motion. We study the phase noise power spectrum of the output light, and we describe the mirror Brownian motion, which is responsible for the thermal noise contribution, using the quantum Langevin approach. We show that the standard quantum Langevin equations, supplemented with the appropriate non-Markovian correlation functions, provide an adequate description of Brownian motion.