Kernel estimators with a global bandwidth are commonly used to estimate regression functions. On the other hand, it is obvious that the choice of a local bandwidth can lead to better results, because a larger class of kernel estimators is available. Evidently, this may in turn affect variability. The optimal bandwidths depend essentially on the regression function itself and on the residual variance, and it is desirable to estimate them from the data. In this article, a local bandwidth estimator is studied. A comparison with its global bandwidth equivalent is performed both in theory and in simulations. As the main result it is shown that the possible gain in mean integrated squared error of the resulting regression estimator must be paid for by a larger variability of the estimator. This may lead to worse results if the sample size is small. An algorithm has been devised that puts special weight on stability aspects. Our simulation study shows that improvements over a global bandwidth estimator often can be realized even at small or moderate sample sizes.