Frequently, we see mistakes on two extremes:
- False Positives: validators, who lack industry experience, confuse difficulty with effective challenges for growing. Seemingly important findings that have little or no potential for adverse consequences—for anyone—are over-emphasized. It’s a lack of proportionality that’s usually oriented to easily-verified, non-conceptual issues.
- False Negatives: for short-term benefit, expedient validators provide a “light touch” to appease noisy developers or managers and possibly to hide their lack of experience and understanding. This does nothing to benefit the firm.
Both extremes waste a lot of time and a lot of resources, especially in the latter case, when a poorly-implemented or conceptually-unsound model could lead to failure costs associated with use or regulatory findings.
Bank executives need to understand that true effective challenge leads to cost-effective models and model development procedures, which helps turn development expenses into development investments and assets, partially by holding modelers accountable for their actions and decisions.