---
title: "Use Case: Real-Time Processing"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Use Case: Real-Time Processing}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

<!-- markdownlint-disable MD024 -->

```{r setup, include=FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
```

Streaming and online LOWESS for live data.

## Overview

When data arrives continuously—from sensors, logs, or streaming pipelines—you
need incremental smoothing that doesn't require reprocessing the entire
dataset.

---

## Online Mode: Point-by-Point

For true real-time applications where each point must be processed
immediately.

`window_capacity = 25` limits the internal buffer to the 25 most recent
observations; each `add_point()` call costs O(window) rather than growing with
total history. `min_points = 5` suppresses output until the window holds enough
points for a stable fit — calls made before that threshold return `NULL`.
`update_mode = "incremental"` re-fits only the most recent point rather than the
full window, halving typical latency at a modest accuracy cost.

### Sensor Data Example

```{r use_case_real_time_1}
library(rfastlowess)

model <- OnlineLowess(
    fraction = 0.3,
    window_capacity = 25,
    min_points = 5,
    update_mode = "incremental"
)

# Simulate real-time data arrival
count <- 0
for (i in 0:99) {
    yi <- 20.0 + 5.0 * sin(i / 10.0) + sin(i * 1.7) * 0.5

    result <- add_point(model, i, yi)
    if (!is.null(result)) {
        if (count < 5) cat(sprintf("Time %d: smoothed = %.4f\n", i, result$y))
        count <- count + 1
    }
}
cat(sprintf("... (%d more)\n", count - 5))
```

---

## Streaming Mode: Chunk Processing

For large datasets that arrive in batches or files.

`chunk_size` controls how many data points are processed in one pass; matching
it to your file-read buffer or message-batch size avoids unnecessary copying.
`overlap` retains that many points from the previous chunk as context so the
neighbourhood at chunk boundaries is not artificially truncated.
`merge_strategy = "weighted_average"` blends the overlapping region smoothly;
use `"last"` if chunk boundaries are guaranteed to be well separated and no
blending is needed.

> **Always call finalize():** The streaming adapter buffers overlap data. Call
> `finalize()` after the last chunk to retrieve the buffered tail.

### Log File Processing

```{r use_case_real_time_2}
library(rfastlowess)

chunk1_x <- 0:49
chunk1_y <- sin(chunk1_x) + 0.1
chunk2_x <- 50:99
chunk2_y <- sin(chunk2_x) + 0.1

model <- StreamingLowess(
    fraction = 0.1,
    iterations = 2,
    chunk_size = 50,
    overlap = 10,
    merge_strategy = "weighted_average"
)

# Process chunks as they arrive
process_chunk(model, chunk1_x, chunk1_y)
process_chunk(model, chunk2_x, chunk2_y)

# CRITICAL: Get buffered overlap data
result <- finalize(model)
cat(sprintf("y[0]: %.6f\n", result$y[1]))
```

---

## Real-Time Dashboard Example

The dashboard pattern uses a plain LOWESS fit on a manually managed sliding
window rather than `OnlineLowess`. This is the simplest approach when your UI
framework already owns the data buffer and you only need the most recent
smoothed value per frame. The trade-off is a full O(window^2) refit on every
tick; for high-frequency streams prefer `OnlineLowess` with
`update_mode = "incremental"` to bound per-frame cost.

```{r use_case_real_time_3}
library(rfastlowess)

n <- 100
x <- seq(0, 2 * pi, length.out = n)
y <- sin(x) + 0.1

window_x <- numeric(0)
window_y <- numeric(0)
last_smoothed <- 0

for (i in seq_along(x)) {
    window_x <- c(window_x, x[i])
    window_y <- c(window_y, y[i])

    if (length(window_x) > 50) {
        window_x <- window_x[-1]
        window_y <- window_y[-1]
    }

    if (length(window_x) < 2) next
    model <- Lowess(fraction = 0.4)
    result <- fit(model, window_x, window_y)
    last_smoothed <- result$y[length(result$y)]
}
cat(sprintf("Smoothed (dashboard, latest tick): %.4f\n", last_smoothed))
```

---

## Choosing Parameters

### Online Mode

| Parameter | Guidance |
| --- | --- |
| `window_capacity` | Enough history for `fraction` to work |
| `min_points` | 2–5 typically; higher for stability |
| `update_mode` | `"incremental"` for speed, `"full"` for accuracy |

### Streaming Mode

| Parameter | Guidance |
| --- | --- |
| `chunk_size` | Balance memory vs. processing overhead |
| `overlap` | 10–20% of chunk_size for smooth transitions |
| `merge_strategy` | `"weighted_average"` for quality, `"average"` otherwise |

---

## Performance Considerations

| Mode | Memory | Latency | Use Case |
| --- | --- | --- | --- |
| **Online** | Fixed (window) | ~1ms/point | Sensors, dashboards |
| **Streaming** | ~chunk_size | ~100ms/chunk | Large files, ETL |
| **Batch** | Full dataset | N/A | Analysis, reports |

---

## See Also

- [Execution Modes](adapter-choice.html) — Detailed mode comparison
- [Merge Strategies](merge.html) — Chunk reconciliation in depth
- [Scaling Methods](scaling.html) — Robustness scale estimation
- [Time Series](use-case-time-series.html) — General time series analysis

```{r session-info}
sessionInfo()
```