Big Data Adoption in Manufacturing

Big Data Adoption in Manufacturing: Forging an Edge in a Challenging Environment
Big Data Adoption in Manufacturing

Some 200 years after the introduction of mechanical production, the emergence of big data analytics has ushered in Industry 4.0, the digitalization and integration of physical assets. For manufacturers that want to grow and remain relevant, there may not be a better time.

Organizations that survived the Great Recession of 2007-2009 have experienced a rocky recovery, and many are bracing for more of the same. By late 2015, only nine of 30 manufacturing sectors in the U.S. had returned to production levels that exceeded their pre-recession peaks, according to a Supply Chain Digest analysis of Federal Reserve data.1 What’s more, in its annual economic forecast in December 2016, industry trade group Manufacturers Alliance for Productivity and Innovation estimated that annual manufacturing output growth would come in at less than 1 percent in 2016 and 2017 and improve only slightly thereafter through 2020.2

The underwhelming prognostication mirrors the outlook from the C-suite. According to Executive Perspectives on Top Risks for 2017, the fifth annual risk survey conducted by Protiviti and North Carolina State University’s ERM Initiative, manufacturing and distribution executives flagged potential macroeconomic risks as having the most significant impact on their businesses.3

Organizations have long understood that innovation during challenging times is paramount to remaining competitive, and today they recognize that big data analytics represents the cutting edge. In fact, many manufacturing and distribution executives predict that the failure to implement a big data strategy posed a risk to their operating and financial performances, according to the Protiviti and North Carolina State University’s ERM Initiative 2017 risk report.4

Smart Manufacturing

Technically speaking, big data analytics is a process that integrates and analyzes massive data sets to reveal previously undiscovered correlations or trends. Management then uses those insights to help generate cost savings, higher production yields, greater profits and improved efficiencies in reorganized production and business processes. Ultimately, advances in big data analytics are expected to augment the inter-connectivity of equipment on the factory floor as part of a larger movement toward the Internet of Things and greater manufacturing intelligence.

In the manufacturing world, big data has the potential to combine and analyze real-time information in every step of the production process, from receipt of a customer’s order to customer fulfillment and satisfaction. Increased efficiencies in those operations, and every task in between, are guaranteed to fatten the bottom line or unlock resources for growth initiatives. Key big data analytics target areas include the following typical cost leaders:

  • Raw materials — Collecting information on the consistency of delivery, quality and performance of raw materials and integrating the data with variables such as output efficiency and customer satisfaction can help identify substandard suppliers or periods when delivery delays increase. Deficiencies in those areas complicate work orders and forecasting, particularly for companies that employ just-in-time manufacturing processes, and prevent manufacturers from fulfilling orders or reaching production goals. Organizations can use the insights to improve forecasting, shape contract negotiations with vendors and line up alternative suppliers to guard against supply disruptions, among other benefits.
  • Labor — Labor generally accounts for 30 to 40 percent of a manufacturer’s overall costs — especially for small and midsize enterprises. Integrating data from various job tasks can reveal where error rates most often occur in a production process or identify productive activities versus those that simply increase overhead. Management teams can use that information to tweak production plans, assembly lines and/or labor requirements. The information may also reveal that workers need additional or advanced training to perform operations, that certain processes result in too much raw material waste or too many work hours, or that a machine in the manufacturing line is falling short of performance expectations.
  • Distribution — Companies that apply big data analytics to transportation operations can identify potential bottlenecks, new market opportunities and periods of increased transportation expense, as well as other insights. Advances in the Internet of Things in conjunction with intelligent transportation systems that follow trucks and convey traffic and road condition information to the cloud, for example, give companies the ability to track their products going to market or raw materials arriving for a production run. Those data feeds can help manufacturers better manage costs, deadlines and efficiencies by rearranging routes to avoid delivery delays. Taking a longer view, the information could be used to guide decisions on whether to open and/or close storage facilities as part of a consolidation, change delivery methods, or make other logistical tweaks.

Marketing Fad or Bona Fide Technology?

Some skeptics argue that big data is nothing more than this decade’s marketing fad — electronic manufacturing systems have generated data for many years, after all. However, most organizations have failed to harness and utilize that data to the fullest. In many cases, companies can spend days or even weeks combining and analyzing disparate information to uncover a trend, whereas big data analytics fosters a speedier process.

By way of example, a fast-growing pet food maker, based in the United States with a global expansion in progress, needed visibility into its business health — financials, sales and production — as it sought to bring all manufacturing in-house. Up to that point, the midcap’s enterprise resource planning (ERP) platform was a mixture of hosted and on-site IT infrastructure that typically required a considerable commitment to consolidate data from multiple sources.

By bringing new systems on board, the company derived benefits, such as eliminating nearly US$200,000 in annual expenses related to the reporting process. It also was able to access real-time inventory, sales and financial information to make quick and effective decisions. Perhaps most significant, users of the system today are more aware of the importance of the data, which is helping the company become a more proactive planner and decision-maker.

The company’s future big data technology upgrades include the use of analytics to support additional distribution channels and international expansion efforts. It also intends to open more plants and pursue the integration of third-party data en route to replacing a reactive operating culture with a proactive one.

Challenges to Change

Like the pet food company, manufacturers that implement big data analytics can improve their operational and financial capabilities, transforming a culture that typically reacts to issues into one that is more equipped to foresee it.

But moving from the big data planning and investment phase to the implementation and execution phase requires some deft negotiation to avoid the roadblocks inherent at most companies. These include:

Unwieldy Data and Processes

Potential Roadblocks to Reaching the Implementation and Execution Phase

  • Unwieldy Data and Processes
  • Inferior and Disparate Systems
  • Location: Cloud or On-Premises?
  • Poor Planning and Implementation
  • Incompatible Technologies
  • Expertise Shortage
  • Inadequate Tools

Many organizations already possess systems that produce reams of data for various tasks through ERP tools, manufacturing execution systems (MES), sales and marketing campaigns, product life cycle management, and emerging technologies, such as the interconnectivity of devices as part of the IoT.

Yet to date, few businesses have the capability to assess and take advantage of data produced by such solutions, let alone integrate information to discover variables or trends that can be acted upon. Manufacturers may be analyzing just a fraction of the information as stand-alone, static data sets to determine what happened in the past instead of trying to discover what the information tells them about the future.

Inferior and Disparate Systems

Shortcomings inherent in current systems can pose challenges even when manufacturers are in the early phases of exploring a big data solution. In many cases, legacy technologies that companies are using have not been designed to facilitate open access to data.

What’s more, a company’s business information technology solution often comprises a mishmash of systems that IT departments and various vendors have patched together over several years. Some manufacturers find it difficult to convert data from those piecemeal systems into exploitable information.

Location: Cloud or On-Premises?

Determining whether to implement cloud-based or on-premises software solutions when considering a technology upgrade has the potential to stall decision-making and subsequent action. While there are pros and cons to each approach, stakeholders can get bogged down by complex considerations, such as costs, security, user access, possible regulatory requirements, and deployment and scalability.

Poor Planning and Implementation

Planning to add or beef up ERP and MES solutions — or build an interface between the two — can spell disaster if the systems are not paired thoughtfully or implemented in a thorough and robust manner. The systems may end up tracking irrelevant business units or processes, for example, while burying relevant information. In other cases, the systems may produce the information a manufacturer initially wanted, but the company learns too late that it needs to be combined with other data that cannot be generated.

Incompatible Technologies

When manufacturers use several different systems that silo data or have been highly customized, it complicates integration — even when the systems are providing relevant information. Myriad locations around the globe that typically generate decentralized data streams further muddy  integration efforts.

Expertise Shortage

A dearth of workers who have the capacity to analyze and derive meaning from the available information poses one more stumbling block. While companies want to employ smart people who understand all facets of the business and manufacturing process, making sense of massive volumes of data requires specialized talent that is in short supply.

Inadequate Tools

Companies need analytics tools that can help them report on and visualize data easily so they can take carefully considered action. While organizations need people who can work with the data, they also need tools that enable non-data scientists or analysts to interpret and apply data insights.

Example: Return on Investment

As daunting as the challenges appear, companies can overcome them if IT departments and management collaborate and bring joint ownership to the planning and implementation phases of a big data integration solution. A 60-year-old multinational chemical and textile business recently did just that, and early on in its big data transformation process, has already generated valuable time and money-saving efficiencies.

After an extended period of growth, the manufacturer found itself with 22 independent ERP systems and databases, which included the adoption of several add-ons by third-party vendors. The opaque and siloed infrastructure required the organization to undertake meticulous manual procedures to generate key reports, including monthly financial statements that, once completed, were virtually obsolete.

To remedy the unacceptable situation, the manufacturer sought to deploy new technology that would consolidate and integrate all mission-critical operational, financial and sales data. The organization also wanted to position itself for a flexible business intelligence environment to facilitate timely and accurate business decisions.

Within weeks of the technology upgrades, the manufacturer enjoyed numerous operational and financial reporting improvements, including the discovery of a product line deficiency that was costing the company thousands of dollars a day. It also achieved a significant increase in financial report timeliness, accuracy and performance. In addition, the firm saw a 50 percent reduction in the resources required to operate the new technology platform.

Companies looking for the kind of success that the chemical and textile manufacturer realized need to recognize that the path to a big data analytics solution will require substantial investment and patience. Even then, the full integration of information may still be weeks or months away.

Where to Leverage Big Data Analytics

Still, pursuing big data analytics promises big rewards. The following are additional use case scenarios that companies should consider when contemplating a big data  analytics implementation.

Bills of Materials

Manufacturers often have complex bills of materials, and each one encompasses thousands of components. Companies historically have lacked the ability to chart the performance and integrity of those inputs as they cycle through production — and how they influence the production process — before they are refined into the finished product.

By combining big data analytics strategies with in-memory database systems, companies will be better able to predict or detect real-time conflicts or problems, including how missing or defective components impact the factory floor’s productivity. Manufacturers can also analyze the information to identify why more labor may be required to run one production line over another or why the amount of material thought to be necessary in a production process  differs from  what  is actually consumed.

Proactive Maintenance

Traditionally, organizations have used MES sensor information to sound a warning when a machine’s operating conditions have deteriorated, leaving maintenance personnel with little time to address the issue and avoid a shutdown. By analyzing historical performance and downtime sensor data, however, companies can craft big data predictive models to anticipate higher production demand periods and identify potential pinch points inherent in a single machine or assembly line. Subsequently, this will allow manufacturers to better schedule maintenance to keep equipment operating at optimal levels with fewer costly outages.

Quality Control

Big data analytics can also provide manufacturers with new quality assurance methods that can cut costs related to in-house testing as well as to downstream customer support and warranty obligations. By analyzing historical information in every part of the production process, such organizations can use predictive analytics to focus on fewer but more specific and better-targeted quality tests.

The potential savings can be especially valuable to companies like microchip makers that historically have put each product through thousands of trials to guarantee performance. Intel applied big data predictive analytics to reduce the time needed to test its microprocessors, for example, which saved $3 million in the first year alone, according to a company performance review.5 It also expected to save 10 times that amount as it rolled out the process to its other products.

Customer Tastes

Social media founders likely didn’t have manufacturers in mind when they launched their technologies. But savvy companies that make products for consumers are beginning to recognize that social media data can identify changing tastes, habits and trends among a target audience. Using that information, organizations are anticipating market demand and creating new products in smaller windows of time.

By combining social media data with information from sales channels, manufacturers can also develop customer profiles with improved visibility into individual client needs, which could pave the way for more custom offerings for certain market segments.

Getting Started

Companies that harness big data and put it to work can look forward to impressive benefits. Yet launching a poorly planned project simply to follow an industry trend all but guarantees failure. Incorporating a big data strategy often proves daunting and requires careful planning. Here are some practical ideas that organizations should consider when mapping out a big data endeavor:

  • Identify a specific use case. Manufacturers should have clear and easily definable goals. As part of that planning process, companies need to determine whether the systems they have in place will achieve the desired results and/or what enhancements might be required.
  • Assign supervision. While technology fuels big data solutions, organizations need to remember that they are addressing a manufacturing problem, not an IT issue. Therefore, it is recommended that someone with an operations background oversee the project, both as a sponsor and on behalf of program management, to provide hands-on insight when necessary.
  • Verify information sources. Manufacturers need to specify the systems churning out the project data and ensure that they can relay the right amount of information in a timely manner. Also, companies may want to seek out subject-matter experts who can provide advice and/or review the project.
  • Ensure reliability of information. If the quality of data is suspect, companies should question whether the specific use case is really an appropriate starting point for an initial big data undertaking. An organization that discovers its information is useless well into a project may drop subsequent projects, which most likely will leave inefficiencies in place.
  • Confirm analytics to be measured. Just as companies need to identify a specific use case, they also need to know in advance exactly what it is that the big data application is supposed to be measuring so they can respond with the right solution. It’s also a good idea to have an action plan based on the results of the analytics.


An expected slow-growth period in manufacturing output over the next few years continues to place pressure on organizations to adopt technologies that can give them an edge over the competition, and many are turning toward big data analytics. The strategy is considered to be the next wave of the Industrial Revolution — or Industry 4.0 — and gives companies the ability to combine and analyze data generated throughout a plant to uncover trends and potential efficiencies.

Big data analytics has the potential to affect every step of a manufacturing process, and management teams that utilize the strategy can become more proactive planners and better decision-makers. In turn, those improvements will drive efficiencies in operations and lead to greater profits and cost savings, among other benefits. While many organizations must first overcome challenges, such as disparate legacy systems, companies that successfully implement big data systems will position themselves for continued growth and the ability to adapt over time to any economic environment.

1Supply Chain News: U.S. Manufacturing Recovery by Key Sectors,” Supply Chain Digest, February 29, 2016.
2U.S. Industrial Outlook: December 2016, Manufacturers Alliance for Productivity and Innovation, December 22, 2016.
3Executive Perspectives on Top Risks for 2017: Key Issues Being Discussed in the Boardroom and C-Suite, Protiviti and North Carolina State University’s ERM Initiative, December  2016.
5Accelerating Business Growth Through IT: 2012-2013 Intel IT Performance Report, Intel, January 2013


Ready to work with us?