Separating model optimization and model validation in statistical cross-validation as applied to crystallography

Statistical cross-validation has become an integral part of the model-refinement process in macromolecular crystallography. However, the test set of reflections, for which the free R value is calculated, is used both to optimize the parameterization of the structure model and to validate the model itself. This practice could introduce bias and diminish the value of R_free as an independent check of model quality. It is proposed here that by introducing a dormant hold-out set of reflections, any problems with such bias can be avoided. This procedure requires only a small modification of the standard cross-validation protocol.