In this paper, we document a novel machine learning-based numerical framework to solve static and dynamic portfolio optimization problems, with, potentially, an extremely large number of assets. The framework proposed applies to general constrained optimization problems and overcomes many major difficulties arising in current literature. We not only empirically test our methods in U.S. and China A-share equity markets, but also run a horse-race comparison of some optimization schemes documented in (Homescu, 2014). We record significant excess returns, relative to the selected benchmarks, in both U.S. and China equity markets using popular schemes solved by our framework, where the conditional expected returns are obtained via machine learning regression, inspired by (Gu, Kelly & Xiu, 2020) and (Leippold, Wang & Zhou, 2021), of future returns on pricing factors carefully chosen.