SUMMARY General expressions are derived for the asymptotic biases in three approximate estimators of regression coefficients and variance component, for small values of the variance component, in generalised linear mixed models with canonical link function and a single source of extraneous variation. The estimators involve first and second order Laplace expansions of the integrated likelihood and a related procedure known as penalised quasi-likelihood. Numerical studies of a series of matched pairs of binary outcomes show that the first order estimators of the variance component are seriously biased. Easily computed correction factors produce satisfactory estimators of small variance components, comparable to those obtained with a second order Laplace expansion, and markedly improve the asymptotic performance for larger values. For a series of matched pairs of binomial observations, the variance correction factors rapidly approach one as the binomial denominators increase. These results greatly extend the range of parameter values for which the approximate estimation procedures have satisfactory asymptotic properties.