Audit on Model Risk Management 2021 | Protiviti Netherlands

audit_on_model_risk_management_2021

Model Risk Management framework

The extent in which an organization should implement and adhere to Model Risk Management are dependent on the reliance of the organization on the models in decision making or the production process, the number of models used and the level of regulation. Internal audit plays a key role in the assessment of the effectiveness of the Model Risk Management.

Topics

Internal Audit and Corporate Governance

Model Risk Management framework

The extent in which an organization should implement and adhere to Model Risk Management are dependent on the reliance of the organization on the models in decision making or the production process, the number of models used and the level of regulation. Internal audit plays a key role in the assessment of the effectiveness of the Model Risk Management.

Need for Model Risk Management

Increased regulatory requirements

Multiple regulators are making Model Risk Management mandatory, including requirements under Solvency II, Richtlijn IV kapitaalvereisten (Richtlijn2013/36/EU) and Basel III. General requirements are coming up as well, for example in the use of Artificial Intelligence models, to assure these models are fair and do not carry any, intended or unintended bias, such as discrimination. The latter by the way has also proven to possibly lead to large reputational damage.

Stakeholder pressure

Demand of an increased role in Model Risk Management by shareholders and other stakeholders on boards and senior management due to losses caused by inaccurate or misused models.

Internal credibility

Errors in models are causing users to doubt their effectiveness and are creating a drive for improved Model Risk Management within financial institution.

Senior management involvement

Model Risk Management has not previously been seen as a strategic priority and has been implemented disparately with little or no aggregation. Senior management members need to lead enterprise wise Model Risk Management.

The role of Internal Audit in Model Risk Management

Model Risk Management is receiving attention due to increase in more quantitative data analysis models. As regulatory supervision on Model Risk Management increases, the Internal Audit function plays a key role in the assessment of the framework in organizations.

Internal Audit teams find themselves facing the challenge of auditing:

Complex models audits focusing on 1stline of defense (Model Development implementation and use) and 2nd line of defense (model governance including model validation).
Processes under review include data selection and processing, the model conceptual soundness review and performance testing, reporting and model use, and other complex areas.
Assessing the programmatic quality and compliance which includes model identification and tracking and monitoring model performance.

Model Risk Management from an Internal Audit perspective

Internal Audit assesses the effectiveness of Model Risk Management to assist management, the board of directors and other stakeholders in the performance of their duties and to confirm that the organization adheres to the guidelines of supervisors.

Commonly observed audit issues

Model governance

Effective challenge and model validation

Audit trail

Model Inventory fields do not include all the necessary information.
Not all models are included in the enterprise-wide model inventory.
There is not a clear distinction between financial models and other spreadsheet tools or end-user tools.
Some regulatory expectations for model validation activities are not clearly required by policies and guidelines.

Model governance and model validation policy not consistently implemented across departments and practices do not always conform with policy.
Work is not consistently divided between model owners (1st line) and independent model validators (2nd line).
Model documentation standards for developmental evidence and model validation are not consistently applied across internally developed and vendor provided models.
Assumptions and the use of business judgment are not sufficiently explained in the model developmental evidence or tested in the independent model validation.
Lack of sufficient ongoing monitoring for models.

Lack of model history and development decisions in the model owner documentation. Includes failure to document the rationale for selection of a vendor model.
Lack of evidence of effective challenge for review by internal audit and regulators: Detailed model validation report, Model validation committee minutes, issues tracking log.

Leadership

Daphne Manichand

Daphne Manichand

Daphne Manichand is an associate director in the internal audit and financial advisory practice in Protiviti’s Amsterdam office. Over the past fifteen years, Daphne has focused here career in internal audit within various financial organizations. Daphne has significant ...