Fetch HDB++ History — How Data Is Served and Plotted

This document explains how fetch_hdbpp_history works end-to-end, and how the data is rendered in our line charts and heatmaps.

Line Chart

• For each attribute, the API returns ≈ returned_points (≈ maxPoints for avg/lttb/none; ≈ maxPoints for minmax because it uses ceil(maxPoints/2) buckets × 2 points per bucket).
• With live mode on, each series is capped to ≤ maxPoints as new packets arrive.

Heatmap

• We render a global evenly spaced time grid of
  columns = min(maxPoints, 2000) across the whole window.
• For each visible attribute row, we draw exactly columns z-values.
• Total cells = (#visible attributes) × columns.

How maxPoints Is Used End-to-End

Server (fetch_hdbpp_history)

For each attribute in [start, end]:

• downsample="avg"

  • Buckets = maxPoints
  • ~1 point per bucket (timestamp at bucket center)
  • returned_points ≤ maxPoints

• downsample="minmax"

  • Buckets = ceil(maxPoints/2)
  • Up to 2 points (min & max) per bucket with real timestamps
  • returned_points ≤ 2 × buckets ≈ maxPoints (off by +1 if odd)
  • Empty buckets yield 0 points

• downsample="lttb"

  • Fetch all raw
  • LTTB reduces to ≤ maxPoints (typically exactly maxPoints)

• downsample="none" (or non-numeric fallback)

  • Raw ascending with LIMIT maxPoints

➡️ Server contract: ≈ maxPoints samples per attribute in the history payload.

Client (Our Widget)

• Line Plot

  • Plots exactly what the server returns per attribute.
  • For minmax, compute midpoint for line/marker; min/max shown as asymmetric error bars.
  • Count of plotted x/y pairs = returned_points.
  • With live mode, append new points and cap series at maxPoints.

• Heatmap

  • To avoid “only last ~1h shows” when unions are large, we build a uniform global timeline of
    columns = min(maxPoints, 2000) buckets over full window.
  • Each attribute row is forward-filled with its last known value at/before each grid time.
  • Per row → always columns z-values.
  • Example: maxPoints=1000, 24h selected →
    • each attribute row = 1000 cells
    • 10 visible attributes = 10,000 cells
  • Hard cap at 2000 columns for Plotly performance.

Concrete Example

• Range: last 24h
• Downsample: minmax
• maxPoints: 1000

Server: returns ~1000 points per attribute (2 × 500 buckets).
Line Plot: ~1000 points per attribute (with error bars).
Heatmap: 1000 columns across full 24h; each attribute row = 1000 cells;
total cells = (#visible attributes) × 1000.