时间序列预测最⼤预测误差_预测误差的措施可以通过实验了
解它们
预测最⼤预测误差
()
exun
Measurement is the first step that leads to control and eventually improvement.
测量是导致控制并最终改善的第⼀步。
H. James Harrington
詹姆斯·哈灵顿
In many business applications, the ability to plan ahead is paramount and in the majority of such scenarios, we u
forecasts to help us plan ahead. For eg., If I run a retail store, how many boxes of that shampoo should I order today? Look at the Forecast. Will I achieve my financial targets by the end of the year? Let’s for
ecast and make adjustments if necessary. If I run a bike rental firm, how many bikes do I need to keep at a metro station tomorrow at 4pm?
在许多业务应⽤程序中，预先计划的能⼒⾄关重要，在⼤多数此类情况下，我们使⽤预测来帮助我们预先计划。 例如，如果我经营⼀家零售店，今天应该订购⼏盒这种洗发⽔？ 查看预测。 我会在年底之前实现财务⽬标吗？ 让我们进⾏预测并在必要时进⾏调整。 如果我经营⼀家⾃⾏车租赁公司，明天下午4点我需要在地铁站养多少辆⾃⾏车？
If for all of the scenarios, we are taking actions bad on the forecast, we should also have an idea about how good tho forecasts are. In classical statistics or machine learning, we have a few general loss functions, like the squared error or the absolute error. But becau of the way Time Series Forecasting has evolved, there are a lot more ways to asss your performance.
如果对于所有这些情况，我们都基于预测采取了⾏动，那么我们也应该对这些预测的良好程度有所了解。 在经典统计或机器学习中，我们有⼀些⼀般的损失函数，例如平⽅误差或绝对误差。 但是，由于时间序列预测的发展⽅式，有很多评估绩效的⽅法。
In this blog post, let’s explore the different Forecast Error measures through experiments and understand the drawbacks and advantages of each of them.
在此博客⽂章中，让我们通过实验探索不同的“预测误差”度量，并了解它们各⾃的弊端和优势。
时间序列预测中的指标 (Metrics in Time Series Forecasting)
There are a few key points which makes the metrics in Time Series Forecasting stand out from the regular metrics in Machine Learning.
有⼏个关键点使时间序列预测中的指标与机器学习中的常规指标脱颖⽽出。
1. Temporal Relevance
1.时间相关性
As the name suggests, Time Series Forecasting have the temporal aspect built into it and there are metrics like Cumulative Forecast Error or Forecast Bias which takes this temporal aspect as well.
顾名思义，“时间序列预测”内置了时间⽅⾯，并且诸如“累积预测误差”或“预测偏差”之类的指标也采⽤了该时间⽅⾯。
2. Aggregate Metrics
2.汇总指标
In most business u-cas, we would not be forecasting a single time ries, rather a t of time ries, related or unrelated. And the higher management would not want to look at each of the time ries individually, but rather an aggregate measure which tells them directionally how well we are doing the forecasting job. Even for practitioners, this aggregate measure helps them to get an overall n of the progress they make in modelling.
在⼤多数业务⽤例中，我们不会预测单个时间序列，⽽是⼀组相关或不相关的时间序列。 ⽽且⾼层管理⼈员不想单独查看这些时间序列中的每个时间序列，⽽是希望通过汇总指标来定向地告诉他们我们在预测⼯作中的表现如何。 即使对于从业者，这种总体衡量标准也可以帮助他们全⾯了解建模⽅⾯的进展。
3. Over or Under Forecasting
3.⾼于或低于预测
Another key aspect in forecasting is the concept of over and under forecasting. We would not want the forecasting model to have structural bias which always over or under forecasts. And to combat the, we would want metrics which doesn’t favor either over-forecasting or under-forecasting.
预测的另⼀个关键⽅⾯是预测过度和预测不⾜的概念。 我们不希望预测模型具有总是⾼于或低于预测的结构性偏差。 为了解决这些问题，我们希望采⽤既不偏⾼预测⼜不偏低预测的指标。
4. Interpretability
4.可解释性taboo>英语单词速记法口诀
The final aspect is interpretability. Becau the metrics are also ud by non-analytics business functions, it needs to be interpretable.
最后⼀个⽅⾯是可解释性。 因为这些度量标准也由⾮分析业务功能使⽤，所以它必须是可解释的。
Becau of the different u cas, there are a lot of metrics that is ud in this space and here we try to unify it under some structure and also critically examine them.
由于这些⽤例不同，因此在此空间中使⽤了很多指标，在这⾥我们尝试将其统⼀为某种结构，并对其进⾏严格审查。
预测指标分类 (Taxonomy of Forecast Metrics)
We can classify the different forecast metrics. broadly,. into two buckets — Intrinsic and Extrinsic. Intrinsic measures are the measures which just take the generated forecast and ground truth to compute the metric. Extrinsic measures are measures which u an external reference forecast also in addition to the generated forecast and ground truth to compute the metric.
我们可以对不同的预测指标进⾏分类。 宽⼴地，。 分为两个部分-内部和外部。 本质度量是仅采⽤⽣成的预测和基础事实来计算度量的度量。 外在测度是除了⽣成的预测和地⾯事实以外还使⽤外部参考预测来计算度量的测度。
Let’s stick with the intrinsic measures for now(Extrinsic ones require a whole different take on the metrics). There are four major ways in which we calculate errors — Absolute Error, Squared Error, Percent Error and Symmetric Error. All the metrics that come under the are just different aggregations of the fundamental errors. So, without loss of generality, we can discuss about the broad ctions and they would apply to all the metrics under the heads as well.
现在让我们继续使⽤内在度量(外在度量需要对这些度量采取完全不同的处理)。 我们有四种主要的计算误差的⽅法-绝对误差，平⽅误差，百分⽐误差和对称误差。 这些指标下的所有指标只是这些基本错误的不同汇总。 因此，在不失⼀般性的前提下，我们可以讨论这些⼴泛的部分，它们也将适⽤于这些主题下的所有指标。一对一外教课
绝对误差 (Absolute Error)
This group of error measurement us the absolute value of the error as the foundation.
这组误差测量以误差的绝对值为基础。
平⽅误差 (Squared Error)
Instead of taking the absolute, we square the errors to make it positive, and this is the foundation for the metrics.
我们将误差平⽅成正数，⽽不是取绝对值，这是这些指标的基础。
误差百分⽐ (Percent Error)
好好学习天天向上用英语怎么说
In this group of error measurement, we scale the absolute error by the ground truth to convert it into a percentage term.
在这组误差测量中，我们根据基本事实对绝对误差进⾏缩放，以将其转换为百分⽐项。
对称误差 (Symmetric Error)雅思报名费
Symmetric Error was propod as an alternative to Percent Error, where we take the average of forecast and ground truth as the ba on which to scale the absolute error.shoo
提出了“对称误差”作为“百分⽐误差”的替代⽅法，在“百分⽐误差”中，我们将预测和地⾯真实情况的平均值作为缩放绝对误差的基础。
实验 (Experiments)
Instead of just saying that the are the drawbacks and advantages of such and such metrics, let’s design a few experiments and e for ourlves what tho advantages and disadvantages are.
我们不只是说这些是此类指标的弊端和优势，⽽是让我们设计⼀些实验并亲⾃了解⼀下这些优缺点是什么。
规模依赖 (Scale Dependency)
In this experiment, we try and figure out the impact of the scale of timeries in aggregated measures. For this experiment, we
在此实验中，我们尝试找出时间序列规模对汇总度量的影响。 对于本实验，我们
1. Generate 10000 synthetic time ries at different scales, but with same error.
shorty⽣成10000个不同⽐例的合成时间序列，但误差相同。
2. Split the ries into 10 histogram bins
将这些系列划分为10个直⽅图箱
3. Sample Size = 5000; Iterate over each bin
样本⼤⼩= 5000; 遍历每个垃圾箱
4. Sample 50% from current bin and res, equally distributed, from other bins.
从当前箱中取样50％，从其他箱中平均分配资源。
5. Calculate the aggregate measures on this t of time ries
计算这组时间序列的合计度量
6. Record against the bin lower edge
记录在纸槽下边缘
7. Plot the aggregate measures against the bin edges.
相对于垃圾箱边缘绘制总体度量。
对称性 (Symmetricity)
The error measure should be symmetric to the inputs, i.e. Forecast and Ground Truth. If we interchange the forecast and actuals, ideally the error metric should return the same value.
误差度量应与输⼊对称，即“预测”和“地⾯真相”。 如果我们将预测值与实际值互换，则理想情况下，误差指标应返回相同的值。
To test this, let’s make a grid of 0 to 10 for both actuals and forecast and calculate the error metrics on that grid.
为了测试这⼀点，让我们将实际值和预测值都设为0到10的⽹格，并计算该⽹格上的错误度量。
互补对 (Complementary Pairs)
In this experiment, we take complementary pairs of ground truths and forecasts which add up to a constant quantity and measure the performance at each point. Specifically, we u the same tup as we did the Symmetricity experiment, and calculate the points along the cross diagonal where ground truth + forecast always adds up to 10.
在此实验中，我们采⽤互补的基础事实和预测对，它们加起来为⼀个常数，并测量每个点的性能。 具体来说，我们使⽤与对称性实验相同的设置，并计算沿对⾓线的点，其中地⾯真实+预测总和为10。
损耗曲线 (Loss Curves)
Our metrics depend on two entities — forecast and ground truth. We can fix one and vary the other one using a symmetric range of errors((for eg. -10 to 10), then we expect the metric to behave the same way on both sides of that range. In our experiment, we cho to fix the Ground Truth becau in reality, that is the fixed quantity, and we are measure the forecast against ground truth.
我们的指标取决于两个实体-预测和真实情况。 我们可以使⽤对称的误差范围(例如-10到10)来修正⼀
个误差，并改变另⼀个误差，然后我们期望该指标在该误差范围的两侧表现相同。在我们的实验中，我们选择了修正地⾯真理，因为实际上这是固定数量，我们正在根据地⾯真理来衡量预测。
上下预测实验 (Over & Under Forecasting Experiment)
In this experiment, we generate 4 random time ries — ground truth, baline forecast, low forecast and high forecast. The are just random numbers generated within a range. Ground Truth and Baline Forecast are random numbers generated between 2 and 4. Low forecast is a random number generated between 0 and 3 and High Forecast is a random number generated between 3 and 6. In this tup, the Baline Forecast should act as a baline for us, Low Forecast is a forecast where we continuously under-forecast, and High Forecast is a forecast where we continuously over-forecast. And now let’s calculate the MAPE for the three forecasts and repeat the experiment for 1000 times.
在此实验中，我们⽣成4个随机时间序列-地⾯真实情况，基线预测，低预测和⾼预测。 这些只是⼀个范围内⽣成的随机数。 Ground Truth和Baline Forecast是在2到4之间⽣成的随机数。Low Forecast是在0到3之间⽣成的随机数，High Forecast是在3到6之间⽣成的随机数。在此设置中，Baline Forecast应该充当基线对我们来说，低预测是我们不断进⾏低预测的预测，⾼预测是我们不断进⾏⾼预测的预测。 现在，我们为这三个预测计算MAPE，并重复进⾏1000次实验。
异常影响 (Outlier Impact)
To check the impact on outliers, we t up the below experiment.
迪马特奥为了检查对异常值的影响，我们设置了以下实验。
We want to check the relative impact of outliers on two axes — number of outliers, scale of outliers.
So we define a grid —number of outliers [0%-40%] and scale of outliers [0 to 2]. Then we picked a synthetic time ries at random, and iteratively introduced outliers according to the parameters of the grid we defined earlier and recorded the error measures.
我们要检查离群值在两个轴上的相对影响-离群值数量，离群值规模。 因此，我们定义了⼀个⽹格-离群值[0％-40％]和离群值[0⾄2]。 然后，我们随机选择⼀个合成时间序列，并根据我们先前定义的⽹格参数迭代引⼊离群值，并记录误差度量。
结果与讨论 (Results and Discussion)
five的序数词绝对误差(Absolute Error)
对称性(Symmetricity)
That’s a nice symmetric heatmap. We e zero errors along the diagonal, and higher errors spanning away from it in a nice symmetric pattern.
这是⼀个很好的对称热图。 我们在对⾓线上看到零误差，⽽在对⾓线上有⼀个很好的对称图案，误差更⼤。
损耗曲线 (Loss Curves)
