In this paper, we introduce the exponential family distribution and give the relevant definition. The recursive formula of higher order moments of geometric distribution is derived by the reciprocal differential operation and integral operation, and the general formula of higher order moments of gamma distribution is derived by the relation between characteristic function and higher order moments.