Wanted: A Cost-Effective Approach to Validating Performance of the Internal Control Structure

Wanted: A Cost-Effective Approach to Validating Performance of the Internal Control Structure

Section 404 of The Sarbanes-Oxley Act of 2002 (SOA) requires public companies to assert annually that internal control over financial reporting (ICFR) is designed and operating effectively. How will companies transition their Section 404 compliance activity from an ad hoc, high-cost project to an ongoing, cost-effective process? In this issue of The Bulletin, we will focus on how your organization can implement a cost-effective approach to validating the operating effectiveness of its ICFR that includes ALL primary sources of evidence, not just independent tests of controls, supporting management’s assertion in the annual internal control report. The focus of the following discussion addresses management’s assessment process, not the external auditor’s audit of ICFR.

Wanted: A repeatable and cost-effective approach

The Section 404 objective of protecting investors byimproving the accuracy and reliability of corporate disclosures made pursuant to the securities laws is an important one. However, there are many opportunities to make the process more efficient. The high-cost status quo, as experienced by first year adopters, requires reevaluation. A balanced approach to support management’s validation of operating effectiveness of internal controls consists of three elements. These elements lead to an approach that is repeatable, sustainable and cost-effective:

  • Self-assessment – In Issue 1 of Volume 2 of The Bulletin, we discussed how companies implement a self-assessment process to reinforce process owner accountability. While not all companies may use   self-assessment, enough companies will use it to warrant careful consideration by every company of the merits of its use. Linked to Section 404 documentation and supported with clear guidelines responsive to management’s expectations of process owners for per forming tests to support the assessments they report, self-assessment is an effective tool for reinforcing process owner accountability. Because many companies will choose to integrate Section 404 compliance with the Section 302 certification process, as discussed in Issue 3 of Volume 2 of The Bulletin, they will look for tools to accentuate that integration. Self-assessment is such a tool.
  • Entity-level monitoring –Effective entity-level monitoring and the transparency it provides is a critical component to achieving a cost-effective compliance process. Monitoring techniques include budgetary controls, exception reports, event reports, “near miss” reports, process metrics, predictive tests, substantive data mining techniques and internal audit reports. If monitoring is taken to the level of real-time reporting and early warning, the process supports leads to more timely and informed decisions. Analytics dashboard reports summarizing key risk indicators (e.g., number of items requiring correction, number of items corrected, number of unreconciled accounts, amounts and items in suspense, etc.) provide entity management with greater transparency as to whether key processes and controls are operating effectively at multiple locations and units.
  • Independent tests of controls – The third element for validating the operating effectiveness of ICFR is independent tests of controls, including an increased emphasis on testing automated controls. The nature, timing and extent of these tests take into consideration the extent and effectiveness of self-assessment and entity-level monitoring.

We discuss below several keys to implementing a repeating and cost-effective validation approach that deploys the above three elements.

Deploy a risk-based approach

Re-evaluate the risks driving your evaluation scope

The cost-effectiveness of Section 404 compliance begins with management’s decisions around (1) selecting significant accounts and disclosures to evaluate, and (2) determining the nature, timing and extent of testing of controls embedded within the processes affecting those accounts and disclosures. The Securities and Exchange Commission (SEC) and Public Company Accounting Oversight Board (PCAOB) have both asserted that the Section 404 process requires a risk based approach when making these decisions. During first year compliance, auditors often required registrants to include within the Section 404 assessment scope ALL accounts above a stated quantitative threshold and to add additional accounts using qualitative factors. The PCAOB staff discarded that approach when it asserted that “quantitative measures alone are not determinative as to whether an account should be identified as significant.” Therefore, qualitative AND quantitative factors comprise the TOTAL mix of information that is available for purposes of determining the significance of an account or disclosure when setting the evaluation scope. The focus should be on risk, not on coverage.
For calendar year accelerated filers, Year Two compliance is already well underway. Thus, management should revise the ICFR assessment scope if it hasn’t been updated with a strong emphasis on relative risk. A substantive risk assessment process will enable management to formulate a conclusion around the key risks related to financial reporting assertions. If the auditor concurs with management’s conclusion early in the evaluation, the ICFR assessment is less likely to meander.

Strengthen your risk assessment process

Management should commit itself to a formal risk assessment process that takes into consideration its cumulative knowledge, experience and judgment. While the frequency of this process is driven by the nature and extent of change, it should be conducted at least annually. The key elements of a formal risk assessment process include setting objectives (for example, relevant financial reporting assertions), identifying potential future events that can lead to objectives not being achieved, and assessing the impact and likelihood of the potential events. While the risk assessment must at least address reliability of financial reporting, management should consider a broader focus on operational and other compliance objectives. This broader focus would enable timely identification of significant changes in external and internal fac tors for possible action and disclosure. An effective enterprise wide risk assessment process will not only enable management to formulate a conclusion around key business risks, it will also provide quality inputs to the formulation of the organization’s business plans.

Deploy a self-assessment process

Make it process-based

The term, “self-assessment,” is often used to describe circumstances where process owners evaluate the controls for which they are responsible and communicate the results of their self-review to management. As discussed in Issue 1 of Volume 2 of The Bulletin, a robust self-assessment approach is always process-based and involves several key components, including the (a) identification of the most important controls, (b) identification of the owners of those controls, (c) predetermination of questions approved by management, (d) rigorous deployment of questions and appropriate follow-up with owners, and (e) resolution of exceptions and open matters on a timely basis.

Supplement it with self-tests

Self-assessment may be enhanced to a higher form of evidence if process or control owners are required to test a minimum sample of items before formulating their assessments of certain controls.

These self-applied tests augment the inquiry, observation and inspection techniques process owners often use as they supervise and monitor the activities for which they are responsible on a day-to-day basis to assess whether controls are functioning properly.

Integrate it with other sources of evidence

Many registrants will use self-assessment as one source of evidence as part of a balanced approach to validate operating effectiveness of internal controls along with entity-level monitoring (“metrics, measures and monitoring”) and independent controls testing. The PCAOB staff has clarified that management may indeed use self-assessment during the Section 404 compliance process. The staff stated that management’s day-to-day involvement with processes and the underlying controls provide more procedures with which to achieve reasonable assurance for purposes of its ICFR assessment than the auditor has available. The staff directed independent auditors to recognize this distinction when evaluating the adequacy of management’s assessment process.

Deploy a top-down validation approach

Strengthen your company-level controls

The Section 404 evaluation was designed to be applied in a top-down manner. Company-level controls are vital to executing a top-down approach. These controls include the control environment, management’s risk assessment process, the period-end financial reporting process, controls over centralized processing and board-approved policies addressing significant business control and risk management practices. In addition, entity-level monitoring that is entity-wide in scope increases certifying officer confidence in Section 302 and 404 compliance processes and makes a strong contribution to the control environment, because it enables management to identify and investigate potential problem areas more timely. Strong company-level controls coupled with an effective self-assessment process (as discussed above) reduce the scope of management’s independent controls testing that would otherwise be required. For example, effective entity-level monitoring reduces the level of testing required for low-risk financial reporting areas or at insignificant locations and units. A point to remember: Management cannot rely on reports used for monitoring purposes without also testing controls over the underlying processes that generate those reports.

Filter your controls down to the vital few

A bottom-up approach to controls testing needlessly adds to cost and reduces the effectiveness of the compliance process. “Filtering” is the risk-based selection of controls for testing that are especially critical to the mitigation of risk and the ultimate achievement of one or more financial reporting assertions. Effective filtering increases the efficiency of testing by narrowing down the population of controls to the ones that really matter. The objective is twofold. First, more sharply link individual controls to significant financial reporting assertions and, second, test only those controls that have the most direct and pervasive impact on mitigating risks of not achieving those assertions. For example, controls that have a pervasive impact on financial reporting include authorization and limit controls in volatile areas, segregation of incompatible duties in significant areas, physical safeguards over significant assets, restricted access to program logic and data, and implementation of process and systems change controls.

If more controls than necessary are tested, significant non value added activity is driven off of activities around understanding the reason for exceptions relating to controls that aren’t important. If evaluation teams decide to ignore test exceptions because the control wasn’t important in the first place, there wasn’t adequate up-front filtering.

Sharpen your focus on independent tests of controls

A testing plan defines management’s approaches, scopes and sample sizes required to support the assertions in the internal control report. The nature, timing and extent of independent tests of controls depends on many factors, including the effectiveness of entity-level monitoring, the criticality of the controls, the exposure to performance variability and the volume, complexity and velocity of the transactions flowing through the process. In addition, the existence of a process-based self-assessment program supplemented with self-tests, as discussed above, should have an impact. For example, an effective self-assessment program may justify management’s decision to alternate independent tests of controls or, in the case of low risk areas, eliminate the need for independent tests altogether.
When developing a testing plan, management must decide which controls to test, how to test those controls, how often to test and by whom. Test plans should optimize the use of self-assessment, entity-level monitoring and independent testing. For example, if unit managers and process owners conduct periodic and comprehensive self-assessments using web-enabled technology with positive results and there are strong entity-level monitoring controls and analytics, the timing and extent of controls testing may be reduced because management already has a body of evidence supporting a conclusion on the effectiveness of controls. In these situations, some view independent testing as an evaluation of the quality of the self-assessment process.

A well developed testing plan lays the groundwork for a cost effective approach. It sets forth, among other things: the responsibility of process owners to determine the operating effectiveness of internal controls for which they are responsible; the nature of the internal controls that will be tested at the entity level and at the process level; the testing standards and sampling approach for each area; the nature and extent of the tests of controls to be performed; and the nature of “failure conditions,” i.e., when does a control pass and when does it fail, and the actions to take when failure conditions occur (i.e., when a control fails to pass a test). The test plan identifies the personnel responsible for performing the planned tests of controls, and the frequency with which tests
are to be done (which often will mirror the operating frequency of the control, i.e., continuous, daily, weekly, monthly or quarterly). The plan articulates the process for reporting test exception rates, the parties to whom test results are reported, and the parties responsible for evaluating test results and reaching a conclusion as to operating effectiveness. The process for identifying gaps and undertaking remediation to close those gaps, including the individuals responsible, is also addressed.

The certifying officers or the Section 404 Compliance Committee should approve the test plan. Once the plan is finalized and approved, it is a good idea to review it with the external auditor to obtain any input he or she may have and to reduce the risk of surprises later during the attestation process.

Insist on effective execution of independent tests

The project team, internal audit, risk control specialists or other management personnel (whose responsibilities lie outside of the area tested) execute independent tests according to management’s approved plan. The PCAOB provides criteria for the auditor to evaluate the use of the work of others for purposes of reducing the work the auditor must perform. The PCAOB’s criteria include the competency of the individuals performing the work, and refer to “ the special status that a highly competent and objective internal auditor has in the [external] auditor’s work,” meaning the auditor will be able to rely to a greater extent on the work of a “highly competent and objective internal auditor” than on work performed by others within the company. Factors relating to competence include, among other things, education, certifications and performance evaluation. Factors relating to objectivity include organizational status, reporting lines, nature of audit committee access and internal policies with respect to assigning individuals to test areas to which they were recently assigned. Individuals are “objective” when a reasonable person can conclude they are able to make evaluations with impartiality and are free of bias. What’s the message? “Competent and objective” execution serves a dual purpose because the greater the extent to which the external auditor decides to rely on the work of others, the lower the audit fee.

Make testing easier with improved processes

Increase maturity of your processes

Internal controls over processes that are well documented and effectively designed and executed are easier to test. The more mature the process (meaning the extent to which it performs consistently, the degree of clarity to which it is defined and the level of effectiveness at which it is managed), the more likely it will be effectively monitored and the more effectively its controls will operate. As process maturity is improved over time, less controls testing is needed.

When controls embedded in mature processes are tested, the results are much more likely to be satisfactory than when controls embedded in less mature processes are tested.

What’s the point? As management analyzes the high cost of testing, remediating and retesting controls, it becomes evident that a reduction in failure rates through improved processes is a sure pathway to cost-effective testing. We see enough instances of high exception rates to suggest that financial reporting processes do not receive the same rigor and attention as day-to-day operating processes. While this may not have been an issue in the past, it is now. The cost of Section 404 compliance has placed this issue in the spotlight. High exception rates cost time and money to assess and correct.

Optimize your automated controls

Experience indicates the number of manual controls on which management is placing reliance is substantial and, in many cases, excessive. Companies relying heavily on manual controls should look for opportunities to simplify and auto mate those controls. Many companies with redundant and overlapping manual controls have not taken full advantage of potential automated controls. For companies with ERP systems, programmed controls and configurable settings is an area where efficiencies can be gained. When companies place reliance on automated controls, the testing is easier, e.g., the “test of one” scoping for systems-based controls is more cost-effective than the scoping around labor-intensive tests of manual controls. Automated application controls are not susceptible to the breakdown and human error that can plague manual controls, provided they are designed, maintained and secured effectively. To justify minimal scopes for testing automated controls, there must be strong controls over changes in and security over systems and data; the operating effectiveness of these general controls should be tested annually.

Leave a trail if you want full credit

The auditing standards do not contain a presumption that in the absence of documentation evidencing performance of a control, the control is not operating effectively. For example, a signature or other documentary evidence is not more important than the execution of the control itself, i.e., a signature on a voucher does not necessarily indicate a careful review of the voucher package actually occurred. Therefore, for purposes of the assessment process supporting management ’s assertion in the internal control report, the absence of documentation is not determinative that the control is not effective in operation. However, for purposes of the auditor ’s evaluation supporting his or her audit of ICFR, he or she must be able to validate controls performance through inspection and re-performance tests. Thus auditors require evidence that the control was properly executed.

To keep net audit costs down, process owners should retain documents and documentation evidencing the execution of critical controls to facilitate testing, e.g., evidence supporting review controls. While the duration of retaining such documentation need not be as long as the working papers and related documentation supporting management’s assertion in the internal control report, it should be a sufficient period of time to cover the certification period, the external auditor review period as well as additional legal and regulatory requirements, if any, mandating retention of specific documents. We expect “best retention practices” to evolve as companies and their auditors gain more experience with the Section 404 compliance process. In the meantime, it is wise to leave a trail for controls the external auditors are likely to test.

Make it real with project management discipline

Going forward without project management discipline is not a good idea. A project management office (or an equivalent function) should be in place to stay on top of the compliance effort. To place the Section 404 effort in perspective, organizations must conduct multiple tasks by multiple people relating to multiple controls within multiple processes across multiple units and locations in multiple geographies. The action steps required to identify, document, assess, test and remediate internal controls can become too overwhelming of a task for even the most talented and best-intentioned individuals.

Therefore, we recommend management establish project management discipline to hold appropriate personnel accountable and bring the process to successful completion, on time and on budget. There will also be control deficiencies requiring remediation that will need to be tracked to ensure they get accomplished timely. The choice is clear: Monitoring “ teeth” are needed, or else it’s just “hope and pray” the job gets done.

Make sure your supporting software is what you need

There are different versions of SOA repository tools available that facilitate project management. Tool functionality required for short-term compliance includes, but is not limited to: methodology framework; project management; work flow review and approval (including e-mail integration); documentation management (including template libraries, issue tracking and corrective action plans); and standard and ad hoc reporting. Functionality required for long-term compliance includes, but is not limited to: direct linkage to ERP controls; dynamic and graphical process modeling; control monitoring and enforcement through alerts and early warn ing; enterprise content management (including versioning and records archiving); and integration of business intelligence and analytical tools. The tools and technologies supporting SOA compliance are in a constant state of flux as providers upgrade them to meet ever-changing market needs. Companies are advised to stay abreast of developments in the SOA software market to ensure they are leveraging the capabilities they need to support cost-effective compliance.

Don’t institutionalize the status quo

In closing, the “project to process” transition is all about taking control, not institutionalizing the high-cost compliance process experienced by many first year adopters. Begin that transition by focusing on immediate priorities. Maximize the “ lessons learned,” from year one, recognize changes in the control environment and determine remediation status relating to deficiencies carried over from first year Section 404 compliance. Continue to clarify roles and responsibilities and focus on immediate change management issues. Ensure these factors are incorporated into your Year Two compliance plan. Review your updated plan with the external auditor.

Think broader than testing when planning a cost-effective approach to validating internal controls. Tests of controls are only one element of the body of evidence available to management. The stronger a company’s compliance environment (i.e., its self-assessment program and entity-level monitoring) and the greater the maturity of its processes (i.e., the extent to which they are defined and managed), the less independent testing is needed to support management’s assertion regarding the effectiveness of ICFR. Given many companies may be unable to fully realize a cost-effective approach this year, management may want to incorporate longer-term expectations around the compliance process and appropriate action items in next year’s budget cycle.

Which Approach Makes More Sense?

Institutionalize the Status Quo and Continue the Game of Reaction

  • Continue to function in a project, “fire drill ” mode
  • Apply a quantitative and coverage approach to scoping
  • Test all of the controls tested last year
  • Apply detailed testing almost exclusively to validate effectiveness
  • Do whatever the auditor tells us to do
  • Under-utilize programmed controls
  • Focus risk assessment on financial reporting risks
  • Apply manual “find and fix” controls with unacceptable defect rates driving costly rework

Improve the Value Proposition and Take Control of the Game

  • Design and implement a cost-effective compliance process
  • Apply a risk-based approach to scoping
  • Improve filtering to focus primarily on the key controls and rotate testing of controls over low risk areas
  • Also apply self-assessment and entity-level monitoring
  • Decide what to do; inform the auditor
  • Optimize programmed controls
  • Focus risk assessment more broadly on business risks
  • Improve the processes upstream to eliminate defects at the source

Key Questions to Ask

Key questions for board members:

  • Has management reviewed with the audit committee its plan to validate controls operating effectiveness in the current year? Does the plan consider such elements as self-assessment and entity-level monitoring so that management is not solely relying on independent controls testing?
  • Are you satisfied that management’s testing plan allows for sufficient time to perform remediation work to cure control deficiencies in time for retesting before year-end?
  • Is the audit committee satisfied that the overall plan for validating the performance of internal controls is cost effective and optimizes net audit costs without compromising audit effectiveness?
  • Has the audit committee inquired of the external auditor as to how he or she plans to improve the cost-effective ness of the Section 404 attestation process in the current year? Does the auditor’s plan include the integration of the audits of the financial statements and ICFR?

Key questions for management:

  • Have you planned for optimizing the efficiency and effectiveness of your Section 404 compliance process through appropriately integrating process owner self-assessment, entity-level monitoring and independent testing of controls? Is your plan supported by a PMO (or an equivalent funcion) that monitors its execution?
  • Have you thought about how you can improve the effectiveness and efficiency of your business processes to facilitate cost-effective monitoring and testing of controls?
  • Is your testing plan sufficiently focused on the most critical controls? Does the plan clearly spell out the “rules of engagement ” up-front so that evaluators will know what actions to take should a test indicate a control is not operating effectively?
  • Are you satisfied that control evaluators meet the external auditor’s criteria relating to competence and objectivity?
  • Are you thinking about incorporating appropriate action items into next year’s budget cycle to increase the cost effectiveness of the compliance process?

The Bulletin (Volume 2, Issue 4)

Click here to access all series

Ready to work with us?