Multimodal Macro Trade Generation Desk

This system ingests central bank statements (text), yield curve charts (vision) and port congestion scenes (satellite proxy), projects them through a from-scratch Financial-CLIP encoder into a temporal Dynamic Knowledge Graph, and lets a VLA action head emit long-short portfolio weights as JSON, with the causal chain that argued for them.

01

System architecture

Three modality streams are encoded separately, fused against the graph's retrieved subgraph, and decoded into one action. Every arrow below is a real tensor path in the repository.

FOMC statements hawk / dove language, financial news Yield curve charts 2s10s panels, CPI vs GDP, policy path (PNG) Port satellite scenes container density at Long Beach, Shanghai Text tower transformer encoder shared 128-d space Financial-CLIP ViT InfoNCE on (chart, structural-break caption) Port CNN container density regressor (trade proxy) DYNAMIC KNOWLEDGE GRAPH entities · typed relations first_seen / last_seen recency-decayed weights Fed → hikes → rates → compresses tech multiples RAG: top-k nodes + n-hop VLA ACTION HEAD frozen vision · statement tokens · graph context · shipping + hawk/dove JSON ACTION {"QQQ": -0.05, "TLT": +0.10, …} retrieved subgraph phase 2: LLM triplet extraction over history
02

Live inference

This section runs the actual model in your browser. The Financial-CLIP text tower, the vision tower, the port CNN and the VLA action head are exported to ONNX and executed with onnxruntime-web, and the knowledge-graph retrieval is the same as-of logic as the Python package. Nothing here is replayed from a file, and nothing leaves your machine. Start from any held-out month, then drive it yourself: rewrite the statement, upload your own chart images to push through the vision tower, upload a port photo for the density CNN to read, or move the as of date to time-travel the graph so retrieval only sees relations observed by then.

loading engine…

action json

loading…
hawk / dove tone
dovishhawkish
—
container density
idlecongested
—

portfolio tilt

graph rationale ▌ argued chains ▌ retrieved evidence

    03

    The knowledge graph, live

    Every node and edge below was extracted from the corpus by the information extraction engine, with observation counts and first/last seen dates. Nodes are placed by causal depth: a node sits one column right of its deepest upstream cause, so reading left to right is reading the transmission chain, from the central bank through policy and market factors out to the tradable assets. Hover a node to isolate its relations; click one to filter the panel.

    institution indicator market factor theme asset

    strongest relations

    04

    Tilts and backtest

    Validation months only, beyond a purged chronological split. Top: the model's monthly weights per asset. Bottom: compounding those tilts one month forward against an equal weight long benchmark.

    predicted weights by month (long green · short red)

    growth of 1.0, monthly rebalance

    VLA tilts equal weight long
    05

    Three training phases

    Phase 1 · Financial-CLIP

    A compact ViT and a text transformer trained from scratch with symmetric InfoNCE on (chart, caption) pairs. Duplicate captions from overlapping windows are masked out of the softmax so they never act as false negatives. The port CNN trains alongside as the container density regressor.

    Phase 2 · Graph construction

    An information extraction engine walks every statement and news sentence, emitting (entity, relation, entity) triplets into the temporal graph. The default engine is a deterministic lexicon matcher scored against the corpus ground truth; an instruction-tuned LLM (Llama 3 class) is a drop-in alternative behind --llm.

    Phase 3 · VLA alignment

    Perception freezes. The action head fuses the chart embedding, statement tokens, the retrieved graph context vector and scalar signals, and regresses forward-looking risk-scaled target weights. Gross exposure is fixed rather than capped, so the head cannot hedge toward zero to dodge a noisy label.

    06

    Swapping in real data

    This page shows a synthetic validation run. The repository ships a calibrated regime-switching simulator in which text, charts, satellite scenes and returns are causally consistent, so the whole pipeline runs with no API keys and no network access. Every component consumes the same interfaces, so the free sources from the blueprint drop straight in via scripts/download_data.py: