Some of the models used to forecast everything from financial trends to animal populations in an ecosystem are incorrect, according to an Idaho State University statistician.
In a new paper published in PLOS One, Jesse Wheeler, assistant professor in the mathematics and statistics department at Idaho State University, and his co-author, Edward Ionides, professor of statistics at the University of Michigan, make the case that algorithms powering the autoregressive integrated moving average (ARIMA) models in two common software environments are producing not-so-accurate parameter estimates. In statistics, parameter estimates are estimates that use collected sample data to make an inference about a population.
“This is like having a calculator that claims to add two plus two correctly, but sometimes it returns an incorrect answer, like two plus two equals three,” explains Wheeler, an expert in statistics and computing. “We often rely on statistical software like we do a calculator, so, if the calculator tells you that it is giving you a specific parameter estimate, it better do so with very high confidence.”
ARIMA models are among the most commonly used for analyzing data collected over time. They are used to relate the current value of something–say, the price of eggs or the number of bears living in a section of forest–to past values of the same measurement. This allows researchers to account for patterns and trends in historical data, helping facilitate scientific discovery and to forecast future values.
“ARIMA models are typically the first time series model that students learn in a classroom,” Wheeler said. “They are taught not only in statistics courses, but also in courses from other disciplines because they are so useful. ARIMA models are also usually the baseline comparison when developing new statistical and machine learning algorithms.”
During the course of Wheeler and Ionides’ research into the software used for ARIMA models, they found and fixed a potential optimization issue in the maximum likelihood estimation algorithm–an algorithm that uses sample data to fit a statistical model–used by the software that would lead to leading to suboptimal parameter estimates. In turn, Wheeler says, substandard parameter estimates can affect forecasting accuracy and other statistical analyses that depend on accurate parameter values.
“Most practitioners don’t even realize the issue exists. We found that the software’s maximum likelihood estimates were not fully optimized, leading to unreliable parameter estimates,” said Wheeler. “The algorithms that are employed claim to maximize the model likelihood, but fail to do so in a surprisingly large number of cases–as high as 60% of the time–depending on the data and model.”
In addition to pointing out the errors, the researchers propose a new algorithm to address the issue and demonstrate that it works in R.
“ARIMA models are used every day by researchers and industry professionals for forecasting and scientific analysis across many fields—economics, health care, weather, and more,” Wheeler said.
“If the software estimating these models has flaws, it can potentially lead to unexpected results or misguided decisions. By identifying and correcting these issues in the maximum likelihood approach, this research helps ensure that practitioners and researchers can rely on the results, ultimately improving both decision-making and scientific understanding. Even incremental improvements in estimation accuracy can have a significant real-world impact.”
More information:
Jesse Wheeler et al, Revisiting inference for ARMA models: Improved fits and superior confidence intervals, PLOS One (2025). DOI: 10.1371/journal.pone.0333993
Provided by
Idaho State University
Citation:
Not-so-model behavior: Popular software tools may give faulty forecasts (2025, November 6)
