Skip to main content

Build a Low-Latency Market Feature Monitor Dashboard

Set up a real-time quantitative trading dashboard with QuantFlow, DolphinDB, and Grafana — in about an hour.

Building a real-time quantitative trading dashboard is traditionally a multi-week engineering effort — data pipelines, computation engines, streaming infrastructure, and visualization all need to be wired together. With QuantFlow, it takes about an hour.

1. Why DolphinDB for Streaming?​

DolphinDB is the streaming engine of choice because of one reason: speed with complicated computation that requires chained steps. Most streaming engines handle simple aggregations well but fall apart when you need rolling windows, lags, cross-sectional operations, and conditional logic chained across multiple steps — DolphinDB's ReactiveStateEngine handles this natively.

A streaming engine alone, however, only solves half the problem. You also need:

  • Market state reconstruction (bars, order books) from raw exchange data
  • A way to define features declaratively and compile them to streaming operators
  • A visualization layer that queries live data without adding latency

QuantFlow bridges all three. By combining DolphinDB with QuantFlow's MarketState engine and FeatureDAG compiler, and leveraging Grafana's visualization capability, you can set up a real-time market monitor dashboard with minimal effort.

2. The Streaming Architecture​

FeatureDAG parses your YAML definitions and generates a DAG representing the full feature computation graph — rolling windows, lags, arithmetic expressions, conditional logic, order book array extractions. The same DAG compiles to Polars expressions for batch research and DolphinDB reactive engine scripts for live trading.

Each computation step becomes a metric expression inside a ReactiveStateEngine. The compiler consolidates multiple features sharing the same input into a single engine, inlines intermediate expressions, and merges compatible features together. Instead of one engine per feature, a handful of consolidated engines produce multiple output columns in one pass.

Engines communicate through shared stream tables — an upstream engine writes rows; a downstream engine subscribes and reacts. Everything stays in-memory within the same DolphinDB process. No serialization between steps. No disk. No context switches.

Trades + LOB → MarketState → Stream Tables → Feature Engines → Grafana
   (raw)        (bars)       (in-memory)    (consolidated)   (WebSocket)

3. Setting Up the Dashboard​

Step 1 — Pull the Grafana Image​

docker pull dolphindb/dolphindb-grafana:9.1.0
docker run -d --name ddb_gra -p 5000:3000 dolphindb/dolphindb-grafana:9.1.0

This bundles Grafana 9.1.0 with the DolphinDB plugin pre-installed. No separate plugin installation needed.

Step 2 — Add the Data Source​

Log in at http://localhost:5000 (default credentials: admin/admin). Go to Configuration → Data Sources → Add, search for "dolphindb." The connection URL uses WebSocket format:

ws://host.docker.internal:8848

Use host.docker.internal if DolphinDB runs on the host machine. If DolphinDB is also containerized, use the container name instead.

Step 3 — Build the Dashboard​

Create panels on a Grafana dashboard, write DolphinDB queries to read from the stream tables generated by QuantFlow. Each panel queries a live stream table — the data updates in real-time as the streaming pipeline processes new ticks.

Step 4 — Start the QuantFlow Streaming Pipeline​

Once the pipeline is running, the dashboard comes alive. Grafana talks WebSocket directly to the DolphinDB server — every dashboard query executes inside the same DolphinDB process that's computing the features. No REST API, no middleware, no serialization overhead between computation and visualization.

4. What Makes This Fast​

The latency comes from eliminating every non-essential hop:

Traditional StackQuantFlow + DolphinDB
Data lands in Kafka/DBData streams directly into DolphinDB
Feature service queries DB per tickFeatures computed in-process, in-memory
REST API serves dashboardWebSocket connects Grafana to same process
Serialization between every layerArrow/zero-copy within single process

The result: sub-millisecond feature computation with live visualization that refreshes as fast as your data arrives.

5. Beyond the Dashboard​

This same architecture powers QuantFlow's entire streaming capability. The YAML definitions you write for research (batch/Polars) compile to the exact same streaming operators in DolphinDB — define once, run anywhere. The dashboard is just the most visible surface of a pipeline that can feed live trading signals, risk monitors, and alerting systems simultaneously.

Next Steps​