LOWESS Concepts
Understanding how LOWESS works and when to use it.
What is LOWESS?
LOWESS (Locally Weighted Scatterplot Smoothing) is a nonparametric regression method that fits smooth curves through scatter plots without assuming a global functional form.
Unlike parametric methods (linear regression, polynomial fitting), LOWESS adapts locally to the data structure, making it ideal for:
- Exploratory data analysis — Discover patterns without assumptions
- Trend estimation — Extract signals from noisy time series
- Baseline correction — Remove systematic effects in spectroscopy
- Genomic smoothing — Smooth methylation, ChIP-seq, or expression data
How It Works
LOWESS smoothing concept
For each point in your data, LOWESS:
- Selects neighbors — Choose the nearest points
(controlled by
fraction) - Assigns weights — Closer points get higher weights (using a kernel function)
- Fits locally — Perform weighted least squares regression
- Extracts value — Use the fitted value as the smoothed estimate
- Iterates (optional) — Reweight points based on residuals to reduce outlier influence
The Fraction Parameter
The fraction (also called bandwidth or span) is the most
important parameter. It controls what proportion of data is used for
each local fit.
Effect of the fraction parameter
| Fraction | Effect | When to Use |
|---|---|---|
| 0.1–0.3 | Fine detail, follows data closely | Rapidly changing signals |
| 0.3–0.5 | Balanced smoothing | Most applications |
| 0.5–0.7 | Heavy smoothing | Noisy data, trend extraction |
| 0.7–1.0 | Very smooth | Strong noise, global trends |
Rule of Thumb: Start with
fraction=0.67(the default) and adjust based on visual inspection. Use cross-validation for automated selection.
Robustness Iterations
Standard LOWESS is sensitive to outliers. Robustness iterations downweight points with large residuals:
Effect of robustness iterations
| Iterations | Effect | When to Use |
|---|---|---|
| 0 | No robustness (fastest) | Clean data, speed-critical |
| 1–3 | Moderate robustness | Most applications |
| 4–6 | Strong robustness | Data with outliers |
| 7+ | Very strong | Heavy contamination |
Confidence vs Prediction Intervals
Confidence and prediction intervals
| Interval Type | What It Represents | Width |
|---|---|---|
| Confidence | Uncertainty in the mean curve | Narrow |
| Prediction | Uncertainty for new observations | Wide |
- Use confidence intervals to show where the true trend likely lies
- Use prediction intervals to show where new data points might fall
Execution Modes
Choose the right mode based on your use case:
| Mode | Use Case | Memory | Features |
|---|---|---|---|
| Batch | Complete datasets | Entire dataset | All features |
| Streaming | Large files (>100K pts) | One chunk | Residuals, robustness |
| Online | Real-time data | Fixed window | Incremental updates |
Quick Decision Guide
| Situation | Mode |
|---|---|
| Data fits in memory; needs intervals or CV | Batch |
| Data too large for memory or arrives in chunks | Streaming |
| Data arrives point-by-point in real time | Online |
Key Advantages
| Feature | LOWESS | Polynomial Regression | Moving Average |
|---|---|---|---|
| No parametric assumptions | ✓ | ✗ | ✓ |
| Adapts to local structure | ✓ | ✗ | Partial |
| Robust to outliers | ✓ | ✗ | ✗ |
| Uncertainty estimates | ✓ | ✓ | ✗ |
| Handles irregular sampling | ✓ | ✓ | ✗ |
Next Steps
- Quick Start — See it in action
?Lowess— All configuration options- Boundary Handling — Edge bias reduction strategies
- Robustness — Outlier downweighting methods
- Scaling Methods — MAD, MAR, Mean scale estimation
- Merge Strategies — Chunk reconciliation in Streaming mode
- Use Case Tutorials — Application-specific guides
sessionInfo()
#> R version 4.6.1 (2026-06-24)
#> Platform: x86_64-pc-linux-gnu
#> Running under: Ubuntu 26.04.1 LTS
#>
#> Matrix products: default
#> BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
#> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.32.so; LAPACK version 3.12.0
#>
#> locale:
#> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
#> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
#> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
#> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
#> [9] LC_ADDRESS=C LC_TELEPHONE=C
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
#>
#> time zone: Etc/UTC
#> tzcode source: system (glibc)
#>
#> attached base packages:
#> [1] stats graphics grDevices utils datasets methods base
#>
#> other attached packages:
#> [1] rfastlowess_5.0.0 rmarkdown_2.32
#>
#> loaded via a namespace (and not attached):
#> [1] digest_0.6.39 R6_2.6.1 fastmap_1.2.0 xfun_0.61
#> [5] maketools_1.3.2 cachem_1.1.0 knitr_1.52 htmltools_0.5.9
#> [9] buildtools_1.0.0 lifecycle_1.0.5 cli_3.6.6 sass_0.4.10
#> [13] jquerylib_0.1.4 compiler_4.6.1 sys_3.4.3 tools_4.6.1
#> [17] evaluate_1.0.5 bslib_0.12.0 yaml_2.3.12 otel_0.2.0
#> [21] jsonlite_2.0.0 rlang_1.3.0