Measures the quality of quantile (probabilistic) forecasts using the pinball loss, also known as the quantile loss. The loss is averaged over all observations and all predicted quantile levels. Smaller scores indicate better calibrated quantile forecasts.
Details
For a single quantile level \(\tau\) with forecast \(q_i\) and observation \(y_i\) the pinball loss is $$ L_\tau(y_i, q_i) = \begin{cases} \tau\,(y_i - q_i), & \text{if } y_i \ge q_i \\ (1 - \tau)\,(q_i - y_i), & \text{if } y_i < q_i \end{cases} $$ The reported score is twice the mean of \(L_\tau\) over all observations and all quantile levels \(\tau\), matching the convention used by fabletools so that the median (\(\tau = 0.5\)) pinball loss equals the mean absolute error.
Dictionary
This mlr3::Measure can be instantiated via the dictionary mlr3::mlr_measures or with the associated sugar function mlr3::msr():
Meta Information
Task type: “regr”
Range: \([0, \infty)\)
Minimize: TRUE
Average: macro
Required Prediction: “quantiles”
Required Packages: mlr3, mlr3forecast
References
Koenker, Roger, Bassett, Gilbert (1978). “Regression Quantiles.” Econometrica, 46(1), 33–50.
See also
Chapter in the mlr3book: https://mlr3book.mlr-org.com/chapters/chapter2/data_and_basic_modeling.html#sec-eval
Package mlr3measures for the scoring functions.
as.data.table(mlr_measures)for a table of available Measures in the running session (depending on the loaded packages).Extension packages for additional task types:
mlr3proba for probabilistic supervised regression and survival analysis.
mlr3cluster for unsupervised clustering.
Other Measure:
mlr_measures_fcst.acf1,
mlr_measures_fcst.coverage,
mlr_measures_fcst.mase,
mlr_measures_fcst.mda,
mlr_measures_fcst.mdpv,
mlr_measures_fcst.mdv,
mlr_measures_fcst.mpe,
mlr_measures_fcst.msis,
mlr_measures_fcst.rmsse,
mlr_measures_fcst.wape,
mlr_measures_fcst.winkler
Super classes
mlr3::Measure -> mlr3::MeasureRegr -> MeasurePinball