SPREADSHEETS ARE OBSOLETE IN THE AGE OF BIG DATA—WHAT IS REPLACING THEM?
Excel spreadsheets are an essential part of any workforce regardless of the type of the business, so much so that Excel has become synonymous with spreadsheets. Most laptop computers and smartphones come with spreadsheets pre-loaded, making them one of the most widely accessible applications worldwide. Spreadsheets are used for data entry, basic calculations, data sorting and analysis, creating presentations, graphs and charts, budgeting and accounting, demand planning and scheduling, and much more.Many people feel spreadsheets have an excellent user-friendly interface with basic functions that are easy to learn. There is plenty of online and offline self-help information available for learning how to use them. Creating data tables in graphs and charts is quick and easy and more experienced users can exploit advanced functionalities like rule-based formulas, conditional formatting, pivot tables, macros, and others.
1. Why Are Excel Spreadsheets Becoming Obsolete?
Given all those benefits, why are they becoming obsolete? Excel spreadsheets were a great productivity application with excellent standalone business analytics with basic reporting. However, spreadsheets have several limitations which hamper their effective deployment in the demand planning process. Chief among is the fact that spreadsheets were not designed for online collaborative work. A single spreadsheet file can be edited by only one user at a time which becomes time consuming if it requires inputs by multiple users.
2. Spreadsheets Are Not Scalable
Spreadsheets are not scalable in today’s digital economy where data streams into data repositories by the second from mobile devices, online websites and devices embedded everywhere in IoT. The higher the volume of data, the slower the spreadsheet processes leading to more chances for the data to become corrupted.
3. Spreadsheets Are Not Easily Shared
They also create islands of information that cannot be easily accessed or shared across the organization. Merging multiple spreadsheet files into a consolidated information file is a difficult and time-consuming task. The more spreadsheets we have, and the bigger they are, the more prone they are to human errors, resulting in misreporting.
4. Spreadsheets Don’t Support Real Time Decision Making
Spreadsheets are incapable of supporting decision making in real-time due to the data being outdated and inaccurate. It is time consuming gathering the most up-to-date information from multiple users and summarizing the information. In many cases, spreadsheets are vulnerable to deliberate manipulations due to an inherent lack of inability to provide controls and quality governance.
Knowing When It’s Time To Move on From Excel
According to several research studies over the past 20 years, well over 70% of demand planners name Excel spreadsheets as their tool of choice for demand planning. Or maybe not! Given the technology advancements over the past 5-10 years this seems illogical. As a demand planner, you need to ask yourself the following:
- Is it getting harder and harder to find empty Excel spreadsheets cells, as you run out of columns and rows?
- Do your spreadsheet cell labels have more letters than the license plate on your car?
- Do you find yourself waking up in the middle of the night in cold sweats because you can’t scale to live streaming data and digital information using spreadsheets?
- Are you feeling confused during business meetings because your spreadsheet results are not the same as others, even though you used the same data sources?
- Feeling dizzy from trying to figure out why your spreadsheet keeps crashing, requiring a blueprint to find the bad cell calculations?
If you experience any of the symptoms described above, you may be suffering from demand planner spreadsheet overload syndrome.A healthy well-rounded diet of data mining, event stream processing, predictive and prescriptive analytics, visual analytics with interactive reports and graphics, all aided by artificial intelligence and machine learning, is the cure to your Excel dependency.
Why Change Now?
Digitization is making the supply chain faster, more intelligent, more connected and autonomous. Forward-thinking C-level decision makers are presenting opportunities for companies to embrace the digital ecosystem, becoming a strategic partner with their customers and consumers. This is aided by the convergence of three factors:
All of this is giving rise to an awareness and willingness to apply analytics to everything. Not just to strategic initiatives, but to day-to-day tasks. There are more and more business analysts embracing and using advanced analytics and machine learning, interpreting and applying the results, effectively becoming citizen data scientists.
- Powerful, more affordable computing power.
- Abundant data.
- The availability of analytics and algorithms—especially with cloud-based open source analytics.
Companies are now looking to invest in new analytics-driven forecasting and planning technology supported by artificial intelligence and machine learning that allows them to measure sales promotions and marketing events to mathematically calculate promotion lifts and determine if they generate revenue and profit—without complicated spreadsheets. Scenarios can be run in real time and the impact automatically reconciled up/down the business hierarchy using a web interface instead of spreadsheets.As data collection and analytics tools, applications and solutions have become more affordable and powerful, they’ve become easier for companies to justify. For many companies, data management technology has advanced so quickly that the challenge now is not about getting the budget, but how to make practical use of all the data collected from IoT devices. Spreadsheets are just not scalable enough to handle live streaming data.Furthermore, data storage costs have declined significantly over the past decade making it more affordable to store the transactional data collected at increasingly granular levels across markets, channels, brands, products and key account configurations. Easy access web-based applications can be used to access the data, without requiring downloads or pivot tables to support spreadsheets. Faster in-memory cloud processing is making it possible to run “what if” simulations in seconds that previously had to run overnight.Over the last decade, a wide variety of other, non-spreadsheet-based forecasting and planning tools, applications, and enterprise solutions have become available that can handle big data gathered from ever-increasing data sources.
New Demand Planning Technology Is Now Available
New disruptive solutions powered by advanced analytics simplify data management, streamline common planning processes, and supports sophisticated workflow creation and deployment. These new, intelligent planning solutions allow the user to:
- Improve sell-through rates by accessing and modeling downstream data (POS/syndicated scanner data) to better anticipate and predict consumer demand. It uses consumption-based forecasting to shape shipments (transactions), also known as sell-in demand based on point of sale or sell-out demand. This results in more accurate shipment plans.
- Improve planning process efficiency using artificial intelligence and machine learning capabilities to provide demand planners with a digital assistant to improve their forecast value add (FVA). The digital assistant guides demand planners up/down the business hierarchy to manage overrides by exception, thereby reducing errors and planning efforts.
- Scalable planning capabilities provide an environment for data scientists and business users to collaborate more effectively through the integration of web-based dashboards, reports and planning workbooks in an integrated navigation environment. All with the look and feel of spreadsheets, with unlimited rows and columns, supported with robust data integration capabilities combined with advanced analytics and machine learning.
- Ease of implementation, expansion and orchestration of analytic workflows. Cloud-ready, out-of-the-box modeling strategies with predefined models solve complex demand forecasting problems faster. Open API’s provide an extension of modeling capabilities with open-source tools like Python and R. This allows companies to put Python and R into production, as well as provide scalability.
The goal is to provide an environment that encourages adoption and the ability to introduce an automated low touch demand planning process that helps manage demand planning by exceptions. Artificial intelligence and machine learning are key to this kind of low touch forecasting. No pivot tables or spreadsheet cell calculations required! This article originally appeared in the Spring 2020 issue of the Journal of Business Forecasting. Click here to become an IBF member and get the journal delivered to your door quarterly, as well discounted access to IBF training events and conferences, members only workshops and tutorials, access to the entire IBF knowledge library, and more