01 / 08 Case study

Market Data Platform

A 14-package Rust infrastructure system separating ingestion, normalization, sequencing, distribution, historical persistence, and client state.

14

Rust packages in the active workspace

3

Desktop, web, and mobile clients

1

Canonical market-data protocol

02 / 08 The infrastructure problem

Every client should see the same market without knowing a provider's raw format

The platform turns provider-specific messages into one protocol, establishes canonical ordering, and distributes the same event into live delivery, historical persistence, and client state. A deterministic replay environment proves behavior without depending on an external network.

Sequencing

One order before fan-out

The core hub validates each normalized envelope, assigns a monotonic hub sequence, and sends the event through the same consumer boundaries for replay and live ingest.

Distribution

Reliable traffic and hot L1 traffic have different obligations

Snapshots, control, and recovery remain on reliable transport. Hot L1 updates may use a datagram path without changing the canonical payload contract.

Recovery

Replay and checkpointing are designed in, not added after failure

Deterministic fixtures pass through hub, aggregator, and client core. The client keeps hot state in memory and emits versioned checkpoint snapshots outside the critical path.

03 / 08 Platform stack

Protocol, hub, edge delivery, persistence, and client state as separate Rust responsibilities

The workspace contains a shared wire protocol, replay harness, core hub, aggregator, QuestDB writer, Postgres-backed app services, headless client core, shared app shell, and Dioxus applications for desktop, web, and mobile.

  • Rust
  • Dioxus
  • QuestDB
  • PostgreSQL
  • Aeron
  • Core NATS
  • WebSocket
  • WebTransport
  • Replay

04 / 08 Data path

One normalized stream from provider edge to every client and store

Each boundary has one job. That allows a transport, provider, or client to change without rewriting the market model.

  1. 01

    Provider ingest

    An adapter removes provider-specific shape and publishes a normalized L1 envelope.

  2. 02

    Core hub

    Validates, sequences, and creates the canonical fan-out record.

  3. 03

    Redistribution

    Direct, Aeron, or Core NATS carries the internal stream behind one sink contract.

  4. 04

    Edge + history

    The aggregator serves live sessions while the market writer persists L1 history to QuestDB.

  5. 05

    Client core

    Decodes edge frames and owns subscriptions, stale state, reconnect, and checkpoint snapshots.

05 / 08 Decisions

The decisions that shape the system

  • Implemented

    Replay and live publish share semantics

    Both paths use the same hub validation, sequencing, and fan-out boundary so test and live behavior cannot quietly diverge.

  • Implemented

    Provider formats stop before the hub

    The hub, aggregator, and clients consume the shared protocol while Yahoo and generic JSON or key-value adapters remain at ingress.

  • Implemented + optional

    Transport remains replaceable

    Direct and Core NATS paths have concrete implementations; native Aeron is feature-gated and requires a separate media-driver smoke environment.

  • Outside hot path

    Checkpoint writes do not block live updates

    Client core marks and drains snapshots while concrete file or platform storage remains behind a persistence contract.

06 / 08 Executable evidence

Paths that can be verified without marketing assumptions

  • Deterministic replay flow from core-hub through aggregator into trading-client-core
  • Offline provider adapters for key-value, JSON, and Yahoo-shaped L1 messages through one ingest runtime boundary
  • Direct, Aeron, and Core NATS redistribution contracts with separate smoke and integration paths
  • QuestDB writer with reconciliation and replay verification for L1 historical persistence
  • Versioned client-edge decoding and an atomic file checkpoint adapter outside the critical live path

08 / 08 The next move

Does your product depend on many sources but need one reliable internal model?

I can help define provider boundaries, normalization, live delivery, and persistence before integrations begin dictating the product.

Discuss the platform->Explore integration services