While the guidance lists three components of validation, executives need to understand that these components are not to be taken as independently important on their own or even mutually exclusive, e.g., initial outcomes analysis provides developmental evidence and, therefore, helps prove conceptual soundness.  There are only about eight pages of guidance related to validation, across the industry those eight pages have spawned thousands and thousands of pages of validation policies and procedures, which are constantly being rewritten.

We like to think of the three main components of validation like Cerebus – validation encompasses the past, present, and future.

1. Evaluation of conceptual soundness, including developmental evidence. 2. Ongoing monitoring, including process verification and benchmarking. 3. Outcome analysis, including back-testing.

Validations and validation reports provide crucial and, frequently, the only feedback to developers. Proper validations provide constructive criticism leading to more relevant and reliable models that improve decision-making and, therefore, profits.

The principals at Spero Risk Associates have been successfully applying the model risk guidance since its publication in 2011. We integrate all three pieces in our loss-forecasting and benchmark framework – but we have also successfully validated existing framework. Sometimes you may have some of the pieces completed and just be looking for an ongoing monitoring package which we are happy to custom tailor to your needs (and possible black box platform model outputs).