We use the exceptional depth of the Ultra Deep Field (UDF) and UDF-Parallel ACS fields to study the sizes of high redshift (z~2-6) galaxies and address long-standing questions about possible biases in the cosmic star formation rate due to surface brightness dimming. Contrasting B, V, and i-dropout samples culled from the deeper data with those obtained from the shallower GOODS fields, we demonstrate that the shallower data are essentially complete at bright magnitudes to z0.4", >3 kpc) low surface brightness galaxies are rare. A simple comparison of the half-light radii of the HDF-N + HDF-S U-dropouts with B, V, and i-dropouts from the UDF shows that the sizes follow a (1+z)^{-1.05+/-0.21} scaling towards high redshift. A more rigorous measurement compares different scalings of our U-dropout sample with the mean profiles for a set of intermediate magnitude (26.0<z_{850,AB}<27.5) i-dropouts from the UDF. The best-fit is found with a (1+z)^{-0.94_{-0.25} ^{+0.19}} size scaling (for fixed luminosity). This result is then verified by repeating this experiment with different size measures, low redshift samples, and magnitude ranges. Very similar scalings are found for all comparisons. A robust measurement of size evolution is thereby demonstrated for galaxies from z~6 to z~2.5 using data from the UDF.