Forcast Metrics

Created
TagsMetrics

Mean Absolute Percentage Error (MAPE) and Symmetric Mean Absolute Percentage Error
(SMAPE).

For MAPE, one needs to pay attention if your target value is skewed (i.e., either too big or too small).

On the other hand, SMAPE is not symmetric as it treats under-forecast and over-forecast differently.

M=1nt=1n[AtFtAt]M = \frac{1}{n} \sum_{t=1}^n \left[\frac{A_{t}-F_{t}}{A_{t}}\right]

Advantages:
Expressed as a percentage, which is
scale-independent and can be used for comparing forecasts on different scales.

Disadvantage
MAPE takes undefined values when there are zero values for the actual, which can occur, for example, demand forecasting.
Additionally, it takes extreme values when the actual is very close to zero.

MAPE is asymmetric, and it puts a heavier penalty on negative errors (when forecasts are higher than actual) than positive errors.
This is caused by the fact that the percentage error cannot exceed 100% for forecasts that are too low. There are no upper limits for
the forecasts that are too high. As a result, MAPE will favor models that under-forecast rather than over-forecast

SMAPE=100%nt=1n[FtAt]([At]+[Ft])/2\text{SMAPE} = \frac{100\%}{n} \sum_{t=1}^n \frac{[F_{t}-A_{t}]}{([A_{t}] + [F_{t}])/2}

Advantages
Fixes the shortcoming of the original MAPE — it has both the lower (0%) and the upper (200%) bounds.

Disadvantage
Unstable when both the true value and the forecast are very close to zero. When it happens, we will deal with division by a number very
close to zero.
SMAPE can take negative values, so the interpretation of an “absolute percentage error” can be misleading.
The range of 0% to 200% is not that intuitive to interpret. Therefore, the division by the 2 in the denominator of the SMAPE
formula is often omitted.