N-body simulations that follow only a collisionless dark matter component have failed to produce galaxy halos or substructure within dense environments. We investigate the `over-merging' problem analytically and with numerical simulations, by calculating dissolution timescales of halos due to physical and artificial dynamical effects. The numerical resolution that has recently been attained is such that mass-loss from two-body relaxation is negligible. We demonstrate that substructure is destroyed in present simulations as a result of large force softening combined with the heating sources of tides and encounters with dissolving substructure. In the limit of infinite numerical resolution, whether or not individual halos or substructure can survive depends sensitively on their inner density profiles. Singular isothermal halos will always survive at some level, however, if halos form with large core radii then the over-merging problem will always exist within dissipationless N-body simulations. In this latter case a dissipational component can increase the halos central density enabling galaxies to survive.