Why gapless, continuous capture is the whole game for market data

The most common failure mode in market-data products isn't wrong numbers — it's missing ones. A feed that drops out during a volatile hour, or silently skips the minutes around a resolution, produces a dataset that looks complete and lies to your backtest exactly when it matters most.

Gaps hide where the signal is

Volatility clusters. The moments you most want in your data — a news shock, a settlement, a cross-venue dislocation — are the same moments a fragile capture pipeline is most likely to fall over, because they coincide with load spikes, reconnects, and venue rate limits. A gap during a calm overnight stretch costs you little. A gap during the event you're studying invalidates the study.

Why continuity is hard

• Venues push order-book updates as deltas; miss a sequence number and the book is corrupt until you resync from a fresh snapshot.
• Long-lived WebSocket connections wedge, silently stop delivering, or get throttled — you need gap detection and automatic resync, not just a socket.
• A deploy or a restart can lose the in-memory buffer that hasn't been flushed to storage yet.
• Coverage has to survive across many venues and thousands of instruments at once, forever.

How we treat continuity as the product

SupaGamma's capture services do sequence-gap detection with snapshot resync on reconnect, buffer to durable storage on a short cadence so a restart can't lose data, and run freshness watchdogs that alert (and auto-recover) when a stream goes stale. The archive is append-only and kept forever. Continuity is the thing you're actually paying for — the raw data itself is a commodity; a gapless, event-granular, multi-venue record of it is not.