01 / 08 Case study
Market Data Platform
A 14-package Rust infrastructure system separating ingestion, normalization, sequencing, distribution, historical persistence, and client state.
14
Rust packages in the active workspace
3
Desktop, web, and mobile clients
1
Canonical market-data protocol
02 / 08 The infrastructure problem
Every client should see the same market without knowing a provider's raw format
The platform turns provider-specific messages into one protocol, establishes canonical ordering, and distributes the same event into live delivery, historical persistence, and client state. A deterministic replay environment proves behavior without depending on an external network.
Sequencing
One order before fan-out
The core hub validates each normalized envelope, assigns a monotonic hub sequence, and sends the event through the same consumer boundaries for replay and live ingest.
Distribution
Reliable traffic and hot L1 traffic have different obligations
Snapshots, control, and recovery remain on reliable transport. Hot L1 updates may use a datagram path without changing the canonical payload contract.
Recovery
Replay and checkpointing are designed in, not added after failure
Deterministic fixtures pass through hub, aggregator, and client core. The client keeps hot state in memory and emits versioned checkpoint snapshots outside the critical path.
03 / 08 Platform stack
Protocol, hub, edge delivery, persistence, and client state as separate Rust responsibilities
The workspace contains a shared wire protocol, replay harness, core hub, aggregator, QuestDB writer, Postgres-backed app services, headless client core, shared app shell, and Dioxus applications for desktop, web, and mobile.
04 / 08 Data path
One normalized stream from provider edge to every client and store
Each boundary has one job. That allows a transport, provider, or client to change without rewriting the market model.
- 01
Provider ingest
An adapter removes provider-specific shape and publishes a normalized L1 envelope.
- 02
Core hub
Validates, sequences, and creates the canonical fan-out record.
- 03
Redistribution
Direct, Aeron, or Core NATS carries the internal stream behind one sink contract.
- 04
Edge + history
The aggregator serves live sessions while the market writer persists L1 history to QuestDB.
- 05
Client core
Decodes edge frames and owns subscriptions, stale state, reconnect, and checkpoint snapshots.
05 / 08 Decisions
The decisions that shape the system
Implemented
Replay and live publish share semantics
Both paths use the same hub validation, sequencing, and fan-out boundary so test and live behavior cannot quietly diverge.
Implemented
Provider formats stop before the hub
The hub, aggregator, and clients consume the shared protocol while Yahoo and generic JSON or key-value adapters remain at ingress.
Implemented + optional
Transport remains replaceable
Direct and Core NATS paths have concrete implementations; native Aeron is feature-gated and requires a separate media-driver smoke environment.
Outside hot path
Checkpoint writes do not block live updates
Client core marks and drains snapshots while concrete file or platform storage remains behind a persistence contract.
06 / 08 Executable evidence
Paths that can be verified without marketing assumptions
- Deterministic replay flow from core-hub through aggregator into trading-client-core
- Offline provider adapters for key-value, JSON, and Yahoo-shaped L1 messages through one ingest runtime boundary
- Direct, Aeron, and Core NATS redistribution contracts with separate smoke and integration paths
- QuestDB writer with reconciliation and replay verification for L1 historical persistence
- Versioned client-edge decoding and an atomic file checkpoint adapter outside the critical live path
08 / 08 The next move
Does your product depend on many sources but need one reliable internal model?
I can help define provider boundaries, normalization, live delivery, and persistence before integrations begin dictating the product.
Discuss the platform->Explore integration services->