This paper reviews three basic methods of non-linear least squares estimation and the best known modifications to the popular Gauss-Newton procedure. With regard to the Gauss-Newton procedure the paper identifies the problem of poor parameterization and gives simple examples to show how it may arise. A simple example is also given to show that one popular method of handling constrained boundaries can lead to erroneous results. A modification, based on stepwise linear regression techniques, to handle both of these problems is presented and discussed and an example is given to illustrate its effectiveness. The required fundamentals of stepwise linear regression are reviewed.