In the dynamic machine learning (ML) landscape, precision is paramount. However, selecting the right forecasting model can be difficult and time-consuming, often resembling a maze of complex algorithms and intricate performance metrics. As FP&A teams strive for more accurate data models to enable informed decision-making and optimize strategies, Sensible ML introduces a groundbreaking concept – the Model Arena. How? Much like in a sports bracket, models compete head to head, vying to be the most accurate predictor for specific product-location combinations. In this third post of our AI for FP&A series, let’s delve into this analogy to uncover the inner workings of Sensible ML’s Model Arena.
Miss the first two posts in the series? Catch up now with post #1 (AI for FP&A Starts with Data Quality) and post #2 (The Secret Ingredients of Machine Learning for FP&A: Features).
Sensible ML orchestrates model competitions that mirror the intensity of a sports bracket. The objective? To pinpoint the most accurate models tailored to each individual product-location combination for which Sensible ML is creating a forecast. This tailoring sets the stage for a groundbreaking approach to ML optimization.
Much like a sports bracket, Sensible ML divides models into a structured competition format. Each model is a contender, entering the arena with the aim to outperform other models. The criteria for success? Producing the most accurate prediction for each target or product-location combination line-item.
While some Corporate Performance Management (CPM) competitors opt for the simplicity of applying a single model across all targets, Sensible ML handpicks individual models for each target, ensuring tailored precision that goes beyond a one-size-fits-all approach.
This is the real power of AI For FP&A. To actually do all the hard work so FP&A teams don’t have to.
Polaris, a global leader in powersports whose products have vastly different characteristics, uses Sensible ML to forecasts for specific products and locations with distinct models across the business.
Contrast this with a one-size-fits-all approach, where reliance on a single forecasting model would fail to account for significant differences between their boating and snowmobile product categories for example. Below are some of the differences that various models consider for Polaris:
A single ML model would simply fail to capture the above nuanced factors in the Polaris product categories, leading to inaccurate forecasts, missed opportunities and poor decision-making. And if that’s the case, that’s a missed opportunity for AI and ML.
With OneStream’s Sensible ML, the Model Arena is built right into the solution. That means, with a single click of a button, business planners can automatically get the most accurate model selected for each forecasted line-item. AI for FP&A at scale!
Model monitoring is also a challenge for traditional ML tools, as model performance begins to degrade once placed into production. Manual efforts also only further hamper model performance. For these reasons alone, effective and scalable ML solutions must automatically compare models and contain the following capabilities to create speed to value for Finance and Business analysts:
In other words, users can continuously monitor model health scores and performance over time and auto-retrain in Sensible ML’s Model Arena. All the model variations in the arena compete against each other in a sports-bracket structure to identify the best-performing models and determine which ones to deploy. Below, Figure 1 depicts Sensible ML’s Model Arena selecting the most accurate model to predict the sales of menu items for a fictitious organization.
Figure 1: Sensible ML Model Arena Call Target
In Figure 1, model A (CatBoost) won the sports bracket. Why? The model received the lowest error score compared to the rest of the models. Therefore, model A was used to create the forecast number for the call product in the Rochester location. When a different target or product location combination is selected, however, a different ML model wins (see Figure 2).
Figure 2: Sensible ML Model Arena Pints Target
As shown in Figure 2, when the Pints product for the Rochester location is selected, model B (XGBoost) beat the others to win the sports bracket. Model B was the most accurate model and was therefore used to create the forecast number for Pints in the Rochester location.
In sum, OneStream’s Sensible ML’s Model Arena automates a targeted, personalized approach for FP&A teams that surpasses the limitations of generalized, one-size-fits-all approach. Bottom line – this helps FP&A create value from AI, unlock the true potential of their data and create competitive edge through unparalleled forecasting accuracy and tailored insights.
The future of accurate, personalized AI for FP&A is here – be a part of it with Sensible ML.
To learn more about Sensible ML, stay tuned for additional posts from our Sensible ML blog series. You can also download our white paper here.Download the White Paper