How ML can transform the financial forecasting process

Seeing the Future More Clearly: How Machine Learning Can Transform the Financial Forecasting Process

The Challenge and Opportunity

Forecasting is a key opportunity for finance to add a significant amount of value and provide strategic insight to an organization. One of the areas in which chief financial officers (CFOs) aspire to help their organizations excel is in providing forward-looking, insightful information about future revenues, expenses and cash flow to help decision-makers chart a profitable course for the company.

Developing financial forecasts that are accurate (even if never perfect) is a difficult task for a CFO’s team, and a significant forecast error has far-reaching implications for financial performance management. It is not surprising, therefore, that many CFOs and senior finance executives express frustration with traditional forecasting processes within their organizations. These processes typically suffer from the following shortcomings:  

  • Manual processes driven by spreadsheets that require a significant amount of time and resources to both produce and update forecasts
  • Ineffective forecast models that roll forward current results multiplied by arbitrary growth factors, rather than using business drivers and data
  • Models that limit the ability to do insightful scenario analysis
  • Reliance on limited data sources
  • Lack of integration with sales and operations forecasting
  • Human and organizational bias

These challenges and inefficiencies result in an inordinate amount of time taken to develop and update forecasts, and they can lead to significant forecasting errors. More importantly, this impacts decision-makers, who have to make critical business decisions relying on information that is not sufficiently insightful, accurate or timely.

In a business environment marked by rapid change, economic uncertainty and technological disruption, maintaining the status quo with regard to forecasting is not a viable option. For finance executives seeking to improve their forecasting accuracy and processes, machine learning (ML) presents a unique opportunity to fundamentally transform financial forecasting. Machine learning technology, if implemented well, can, first and foremost, significantly improve the accuracy of forecasts, as we discuss further in this paper. In addition, these tools can be leveraged to automate forecasting models and perform computations on large data sets at high speeds. By automating the labor-intensive components of forecasting and improving predictions, analysts can focus on delivering higher value to decision-makers.

What Is Machine Learning?

Machine learning is a branch of artificial intelligence. The majority of machine learning applications today focus on making predictions, which is why the technology lends itself well to supporting the one area where Finance must make its best informed prediction: forecasting. Machine learning, a method of data analysis that automates analytical model building, is based on the idea that systems can “learn” from data, identify patterns and make predictions with minimal human intervention. Machine learning is iterative, in that models built using the technology independently adapt when they are exposed to new data.

Machine learning in our everyday lives is ubiquitous, even if not often recognised as such — think of how a smartphone groups pictures based on people’s faces, or how Amazon and Netflix provide shopping or movie recommendations based on past activity. As machine learning’s applications grow in number, not only does it become more cost-effective to use the technology, but the technology itself improves over time. The level of accuracy and efficiency with which machines make predictions will continue to help humans focus on the value-added activities that require insight and judgment.

Benefits of Machine Learning in Financial Forecasting

Utilising machine learning to automate the financial forecasting process presents several unique benefits for senior finance executives and their teams. The key benefits are summarised below.

1. Ability to Produce More Accurate Forecasts, Faster 

As mentioned earlier, machine learning-enabled forecasting can rid financial forecasting of the labour-intensive work of collecting and reconciling data. The tools can be configured to collect and reconcile very large data sets in an automated fashion. Moreover, machine learning tools can help to determine business drivers and greatly reduce forecast error. Machine learning algorithms are designed to learn from the data over time and predict which drivers have the greatest impact on financial performance. Over time, the model becomes more accurate and produces forecasts more quickly.

“The incredible volume of data that is managed within any organization on any given day can be overwhelming, making obsolete the traditional methods used to extract value from that data. Machine learning allows analysts to detect, identify, categorize and predict trends and outcomes, resulting in an organization that is able to effectively compete in a big data world. The potential for change that machine learning brings can fundamentally transform key business processes such as financial forecasting.”
Shaheen Dil

2. Ability to Use More Data 

With spreadsheet-driven forecasting processes, there are limits to how many data sources and how much data can be computed and consumed within forecasting models. Machine learning tools can greatly enhance the volume and types of data that can be used because the tools can hold more data and compute 
it faster than humans. For example, a consumer products company can easily pull in search engine or social media data to determine when consumers are searching for or posting about their products the most. This type of data can give more insight into the peaks and valleys of revenue for the forecaster.

3. Enabling Value-Adding Activities

Traditional forecasting processes typically require analysts to spend most of their time reconciling and compiling data rather than working on value-added analysis and interacting with the business. Using a machine learning solution to produce at least a baseline forecast can help analysts move away from these mundane tasks and focus on understanding operational drivers, key business events, and microeconomic and macroeconomic factors that may impact the business, bringing those insights into the forecasting process. Leveraging machine learning can ultimately help financial analysts partner more closely with the business and support decision-making.

Designing a Machine Learning Model for Financial Forecasts

To establish the basic parameters of a forecasting model, executives must develop initial hypotheses around what they believe are the drivers of their business. This is a key element of success, and worth spending time to develop. Drivers can include operational activities (client visits, number of ads in a flyer, projects, etc., depending on the business) or other relevant financial factors.

Once the finance leaders have formed hypotheses about the drivers, they can then test them using data. This highlights another key benefit of machine learning: the ability to process more data from a larger number of sources to test hypotheses than a human analyst could. The data sources can be internal (e.g., ERP, sales, warehouse) or external data (e.g., market share, share price, search engine, interest rates, social media).

Another key benefit of machine learning is the technology’s ability to access and combine both structured and unstructured data. In the case of forecasting, structured data could be data from the organisation’s ERP or warehousing system, whereas unstructured data could be comments or reviews on social media platforms of the company’s products or services. With that being said, the old adage “garbage in, garbage out” still applies, so it is important to verify the accuracy and quality of the data inputs. Poor-quality data can impact the models and produce less than ideal results.

With the hypotheses defined and tested using data, a machine learning programmer can then start to build algorithms and train the model to analyse patterns in the data and make predictions. The baseline forecast produced by this model will help analysts drive conversations with the business and layer information learned from the business onto the forecast, producing a more accurate and insightful financial picture. During this process of model building, it is very important that the models are tested and validated to ensure that the machine is not making mistakes in its learning process. Part of this validation exercise could be comparing the models against current forecasts or using older data and comparing the model results against actual results.

Finally, as the machine learns from the inputs it receives, it will start to create stronger correlations in patterns and refine its forecasts over time. As new data and drivers become known or available, they can be used to update the model, closing the iteration loop. (See graphic below.)

Note that the hypotheses about the drivers and the data need not be perfect, as the model can help determine whether there is a true correlation between the driver and financial results. The iterative process can continue until users are confident about both the accuracy of the prediction that the model generates and the model’s ability to adapt to the changing needs of the business and the economic environment. An additional advantage is the fact that changes can be made on the fly and impacts calculated just as quickly.

Getting Started With Machine Learning Adoption

Where should companies begin their journey of adopting a machine learning approach to forecasting? Below is a simplified road map describing the key steps. The scoping phase of the process is the most important phase. It is imperative that companies phase the implementation appropriately, but also focus on pieces that will likely make the greatest impact on the organisation. The last step, “Processes, Reporting and Analytics,” requires that certain processes around forecasting change to accommodate the adoption of the machine learning tool. In addition, reporting is one the key deliverables coming out of forecasting and presents an opportunity for greater accessibility and effectiveness leveraging visualisations and other presentations.

The steps above can be leveraged as the enterprise adds more forecast items, incorporates more data sources and refines the models as the business changes. 

Last but not least, launching a machine learning programme should be guided by key measures of success to ensure that the company is working toward achieving its goals (e.g., improved forecast accuracy, improved speed, more agile forecast updates, or all of the above). 

Machine learning will transform finance, making finance operations more effective and driving transformation that will allow employees to focus on value-adding activities such as enhancing their capabilities in customer experience and delivering better results to their internal and external customers.
Shawn Seasongood

Protiviti research shows that the role of CFOs and finance executives continues to evolve as they are increasingly asked to be strategic partners to the business. Providing insightful, timely and action-oriented forecasting information is essential to meeting these demands. Machine learning promises to be a game changer for any finance leader looking to take forecasting to the next level.
Marty Murray,

Machine learning and artificial intelligence is an exploding area of development and the hottest technology “trend,” according to Forbes1 and others. Any finance executive seeking to transform the forecasting process should consider leveraging machine learning as a key part of producing financial forecasts: predicting future results. Financial forecasting is perhaps the one area where Finance can help drive the most value within an organisation and have direct impacts on revenue, profitability and shareholder value. Improving the ability to produce more accurate forecasts more quickly can help Finance partner with the business to exploit opportunities to improve top-line revenue growth, course-correct overspending and improve cash flow, among many other things. The machine learning solution can aid the finance function and the business in seeing the future more clearly by helping to reduce forecasting error. While no forecast is as good as hindsight, the margin of error can be narrowed significantly, and any forecaster should not forego the opportunity.