--- title: "Confidence and Prediction Intervals" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Confidence and Prediction Intervals} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include=FALSE} knitr::opts_chunk$set(collapse = TRUE, comment = "#>") ``` ## Overview ![Confidence and prediction intervals](../man/figures/intervals_comparison.svg) > **Adapter support:** Confidence and prediction intervals are available > in **Batch** mode, **Streaming** mode (computed per chunk and merged > across overlap boundaries via `merge_strategy`, like `y`/`derivative`), > and **Online** mode when `update_mode = "full"` is set (errors if > combined with the default `"incremental"` mode). Confidence and prediction coverage levels are independent; for example, a 90% confidence interval can be paired with a 99% prediction interval. | Type | Represents | Width | Use | | --- | --- | --- | --- | | **Confidence** | Uncertainty in mean curve | Narrow | Trend location | | **Prediction** | Uncertainty for new points | Wide | New data location | --- ## Confidence Intervals Estimate uncertainty in the smoothed curve itself. ```{r intervals_1} library(rfastlowess) set.seed(42) x <- seq(0, 2 * pi, length.out = 100) y <- sin(x) + rnorm(100, sd = 0.3) model <- Lowess(fraction = 0.5, intervals = intervals_opts(confidence = 0.95)) result <- fit(model, x, y) # Plot with bands plot(x, y, pch = 16, col = "gray", main = "LOWESS with 95% Confidence Intervals") lines(result$x, result$y, col = "blue", lwd = 2) lines(result$x, result$confidence_lower, col = "blue", lty = 2) lines(result$x, result$confidence_upper, col = "blue", lty = 2) cat("95% CI at midpoint: [", result$confidence_lower[50], ", ", result$confidence_upper[50], "]\n") ``` --- ## Prediction Intervals Estimate where new observations might fall. ```{r intervals_2} library(rfastlowess) set.seed(42) x <- seq(0, 2 * pi, length.out = 100) y <- sin(x) + rnorm(100, sd = 0.3) model <- Lowess(fraction = 0.5, intervals = intervals_opts(prediction = 0.95)) result <- fit(model, x, y) plot(x, y, pch = 16, col = "gray", main = "LOWESS with 95% Prediction Intervals") lines(result$x, result$y, col = "blue", lwd = 2) lines(result$x, result$prediction_lower, col = "red", lty = 2) lines(result$x, result$prediction_upper, col = "red", lty = 2) cat("Prediction bounds: [", result$prediction_lower[1], ", ", result$prediction_upper[1], "]\n") ``` --- ## Both Intervals Request both types simultaneously: ```{r intervals_3} library(rfastlowess) set.seed(42) x <- seq(0, 2 * pi, length.out = 100) y <- sin(x) + rnorm(100, sd = 0.3) model <- Lowess( fraction = 0.5, intervals = intervals_opts(confidence = 0.90, prediction = 0.99) ) result <- fit(model, x, y) plot(x, y, pch = 16, col = "gray", main = "Confidence vs Prediction Intervals") lines(result$x, result$y, col = "blue", lwd = 2) # Confidence interval (narrow, blue) lines(result$x, result$confidence_lower, col = "blue", lty = 2) lines(result$x, result$confidence_upper, col = "blue", lty = 2) # Prediction interval (wide, red) lines(result$x, result$prediction_lower, col = "red", lty = 3) lines(result$x, result$prediction_upper, col = "red", lty = 3) legend("topright", c("Data", "Smoothed", "95% CI", "95% PI"), pch = c(16, NA, NA, NA), lty = c(NA, 1, 2, 3), col = c("gray", "blue", "blue", "red")) cat("95% CI at midpoint: [", result$confidence_lower[50], ", ", result$confidence_upper[50], "]\n") ``` --- ## Confidence Levels Common levels and their z-values: | Level | z-value | Interpretation | | --- | --- | --- | | 0.90 | 1.645 | 90% of intervals contain true value | | 0.95 | 1.960 | 95% of intervals contain true value | | 0.99 | 2.576 | 99% of intervals contain true value | ```{r intervals_4} # 99% confidence interval model <- Lowess(fraction = 0.5, intervals = intervals_opts(confidence = 0.99)) result <- fit(model, x, y) cat("99% CI at midpoint: [", result$confidence_lower[50], ", ", result$confidence_upper[50], "]\n") ``` --- ## Bootstrap Intervals Set `bootstrap` to compute intervals from resampled fits instead of the analytic formula. Pass `seed` for reproducible resampling: ```{r intervals_bootstrap} model <- Lowess( fraction = 0.5, intervals = intervals_opts( confidence = 0.95, prediction = 0.95, bootstrap = 200 ), seed = 42 ) result <- fit(model, x, y) cat("Bootstrap 95% CI at midpoint: [", result$confidence_lower[50], ", ", result$confidence_upper[50], "]\n") ``` The same `intervals` and `seed` arguments are accepted by `StreamingLowess()`, `OnlineLowess()`, and `predict()`. --- ## Standard Errors Access standard errors directly (available when intervals are computed): ```{r intervals_5} model <- Lowess(fraction = 0.5, outputs = "se") result <- fit(model, x, y) cat("Standard errors (first 5):", head(result$standard_errors, 5), "\n") ``` --- ## Availability > **Supported In All Three Adapters:** Confidence and prediction intervals > are available in **Batch**, **Streaming**, and **Online** mode > (`update_mode = "full"` only). | Feature | Batch | Streaming | Online | | --- | --- | --- | --- | | Standard errors | ✓ | ✓ | ✓ (`update_mode = "full"` only) | | Confidence intervals | ✓ | ✓ | ✓ (`update_mode = "full"` only) | | Prediction intervals | ✓ | ✓ | ✓ (`update_mode = "full"` only) | ```{r session-info} sessionInfo() ```