Quantitative models often are criticized for being overly complex, opaque mathematical engines that lead to poor business decisions. Are models bad for business? In our view, they are not. These tools provide useful information for management decisions and offer increased efficiency. As such, we believe models are here to stay. However, the use of these tools exposes organizations to model risk and creates the need for effective model governance.
Reliance on model output plays a central role in today’s complex and increasingly sophisticated business environment. At the same time, significant model-related losses continue to occur at an alarmingly high rate. Consider:
- A large financial institution failed to assess the known risks on its derivatives in its models. The firm has since lost more than $30 billion.
- A financial services firm with $13 billion in annual revenues overstated projected earnings by $2.6 billion due to a model error.
- A $2 billion financial firm had to restate earnings by $70 million due to a model error.
When significant model-related losses occur, they frequently are blamed on the model itself. However, models are tools that fit within a broader business context. Models provide information for decision-making. In some instances, models are heavily or overly relied upon, while in others the output is disregarded as being “not realistic.” Those organizations with robust model control frameworks have greater confidence in the reliability of model output.
Challenges and Opportunities
A robust model governance structure can enhance a firm’s overall risk management effectiveness. To be effective, however, model governance should be rooted in management’s strategic thinking and in an institution’s culture. Often times, model governance is viewed as a necessary evil related to compliance with regulatory requirements rather than as a value-added activity.
Understanding the value of model governance and its role in effective risk management provides a way for organizations to be proactive in managing their model risks. This includes identifying high-risk models, and prioritizing model-related risk and compliance initiatives within business practices such as model validations, assumption reviews and data integrity analysis. Proactively and effectively managing model risk can provide management with a significant competitive advantage through proper model usage and confidence in the model’s outputs.
Our Point of View
Transparency is critical for complex financial models in rapidly changing business environments. Risks can be taken and assumptions made, but governance structures should ensure assumptions and model limitations are updated, properly addressed in implementation, communicated, and more importantly, understood by management, business users and stakeholders. A basic, yet critical, first step in this effort is establishing an enterprisewide definition of a model. Additional policies, standards and guidelines can be built out from this definition.
Model governance standards should be applied across the entire enterprise. This ensures consistent practices for model development, documentation, validation, calibration and monitoring in all business units. Defined model standards enable organizations to react quickly and uniformly in response to major market events or systemic crises.
How We Help Companies Succeed
Our Model Risk Management practice helps organizations by assessing, designing and implementing model governance programs and by conducting independent model validations. We also develop customized quantitative models, refine and calibrate existing models, and design stress testing and scenario analysis programs to supplement existing analytics.
We work collaboratively with our clients to develop effective, firmwide model governance structures tailored to their culture and needs. Using our proprietary Model Governance Framework, we work with risk managers to assess a firm’s current governance structure, identify any gaps presenting undue risk to the firm and design solutions to improve the model control environment.
Our professionals have advised organizations ranging from the largest global financial services firms to billion-dollar banks. We offer flexible, scalable solutions to fit the needs of our clients.
A large U.S. financial institution sought to improve its model governance structure due to several model failures that caused the firm to incur significant losses, face increased regulatory scrutiny and experience significant reputational damage. Protiviti worked with the client to create a fully transparent and repeatable model governance process, resulting in decreased auditor findings, deeper understanding of key model risks by client executives and board members, and reduced regulatory commentary. For our client, Protiviti delivered the following:
- A customized model governance structure applicable to all business lines, thus allowing a central point of control and review for the client’s model risks
- Model risk policies to facilitate model identification, prioritization, categorization and ongoing monitoring
- Technical guidance for model validation techniques across a wide variety of model types and asset classes
- Training guidelines for model validation and monitoring, including a communication plan to each of the business lines
- Design specifications for a technology tool to be used in model monitoring