Skip to content

feat(swarm): pre-fetch market data and ground worker prompts in real prices#93

Merged
warren618 merged 3 commits into
HKUDS:mainfrom
Teerapat-Vatpitak:feat/swarm-prefetch-grounding
May 12, 2026
Merged

feat(swarm): pre-fetch market data and ground worker prompts in real prices#93
warren618 merged 3 commits into
HKUDS:mainfrom
Teerapat-Vatpitak:feat/swarm-prefetch-grounding

Conversation

@Teerapat-Vatpitak

@Teerapat-Vatpitak Teerapat-Vatpitak commented May 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Pre-fetches recent OHLCV data for any suffixed stock / crypto symbols mentioned in user_vars and renders it as a "Ground Truth" markdown block spliced into every worker's system prompt.
  • Eliminates a structural failure mode where workers quote training-cutoff prices (e.g. NVDA reported as $145.89 in a 2026-05 run when the actual close that day was $215.20) and base downstream recommendations on them.
  • One new module + one new optional SwarmRun field + 12-test regression suite.

Update 2026-05-10: Rebased onto current main (post-#88, #90, #91, #92). Conflict with #92 in models.py and runtime.py was mechanical — both PRs add new SwarmRun fields and new blocks in start_run at adjacent lines; resolution preserved both sides. All 18 unit tests pass together (12 grounding + 6 run-metadata from #92); end-to-end fetch against NVDA.US returned a 2026-05-08 close of $215.20 — matches the original motivating bug exactly.

Why

Swarm workers are LLMs. Without explicit grounding, they cheerfully quote prices from their model's training cutoff — which is wrong by definition for any asset that has traded since the cutoff. A run targeting NVDA.US in 2026-05 produced a research report anchored on $145.89 while the actual yfinance close that day was $215.20; subsequent recommendations ("Hold/Accumulate, target $148-160") were wrong by 30%+ before any analyst even saw them. issue #42-class "weird outputs" land here too when a worker silently substitutes a remembered number for a missing data call.

Why prompt rules alone don't fix it

worker.py:158-171 already prints generic "use tools when needed" guidance. LLMs nevertheless prefer the cheap path of recalling a number over the expensive path of issuing a tool call — especially under the 20-tool-call hard limit imposed two paragraphs later. Adding a stronger "never quote prices" rule helps but does not eliminate the failure mode.

What this PR does instead

Make the data unavoidable. SwarmRuntime.start_run now:

  1. Scans every value in user_vars for suffixed symbols (NVDA.US / 700.HK / 600519.SH / BTC-USDT / 000001.SZ / 430090.BJ).
  2. Pre-fetches the last 30 days of OHLCV via the existing loader registry (resolve_loader handles per-market routing). Failures are non-fatal — the run still proceeds, just without grounding.
  3. Pins the result on a new SwarmRun.grounding_data field and persists it via the existing run.json path.

SwarmRuntime._execute_run renders the data into a markdown "Ground Truth" block once per run and threads it through _run_worker_with_retriesrun_workerbuild_worker_prompt, which splices it ahead of Execution Rules. The block contains the recent window range, last close, the last 5 daily closes, and an explicit instruction to prefer these prices over training-data prices and to call get_market_data for any range outside the window.

Files

File Change
agent/src/swarm/grounding.py (new) Symbol extraction, OHLCV fetch, markdown rendering.
agent/src/swarm/models.py New optional grounding_data: dict[str, list[dict]] | None = None on SwarmRun.
agent/src/swarm/runtime.py Extract + fetch in start_run; render once + thread through worker calls in _execute_run.
agent/src/swarm/worker.py New grounding_block: str = "" parameter on build_worker_prompt and run_worker; injected ahead of Execution Rules when non-empty.
agent/tests/test_swarm_grounding.py (new) 12 tests: extraction across all four loader-supported suffix shapes, ordering / non-string / word-boundary edges; fetch with a stubbed loader (normalised bars, empty-frame skip, empty-input no-op); markdown rendering (range / last close / explicit instruction); end-to-end prompt plumbing (block lands before Execution Rules; absent block omits the section).

Backward compatibility

  • SwarmRun.grounding_data defaults to None; existing run.json files parse unchanged (Pydantic deserialises the missing key to the default).
  • build_worker_prompt and run_worker each gain grounding_block="" so callers outside the swarm runtime keep working.
  • Empty grounding (no symbols detected, or every fetch failed) skips the entire section — no empty heading rendered.

Out of scope

  • Bare tickers without a suffix (just NVDA) are intentionally not matched. The false-positive cost on common English words is too high and every shipped preset already supplies suffixed symbols.
  • The block is a one-shot snapshot at run start. Long-running swarms may see stale data after many minutes — still strictly better than training-data prices from a year ago. A refresh hook can land later.
  • Layering note: this introduces a swarmbacktest.loaders.registry import. The alternative (going through the MCP tool registry) was strictly heavier and the current direction matches existing usage from swarm tools, but I'm happy to refactor toward the registry path if that fits the repo's intended layering better.

Test plan

  • pytest agent/tests/test_swarm_grounding.py -v — 12/12 pass.
  • pytest --ignore=agent/tests/e2e_backtest --tb=line -q — same baseline pass count + 12 new tests, no regressions.
  • Manual sanity: an empty user_vars produces no grounding block; a user_vars with NVDA.US produces a block whose last close matches the real yfinance close for that date.

Checklist

  • No changes to protected areas (agent/src/agent/, agent/src/session/, agent/src/providers/).
  • No hardcoded values; the symbol patterns mirror what the loaders already accept.
  • Code follows CONTRIBUTING.md (Conventional Commit prefix, Google-style docstrings, regression test added).
  • Backward-compatible (legacy run.json parses without modification; existing public APIs gain only kwargs with defaults).

…prices

Symptom
-------
Workers in fundamental_research_team and similar presets routinely quote
prices from their model's training cutoff. A run targeting NVDA.US in
2026-05 produced a research report anchored on $145.89 while the actual
yfinance close that day was $215.20. Subsequent recommendations
("Hold/Accumulate, target $148-160") were therefore wrong by 30%+ before
any analyst even saw them. issue #42-class "weird outputs" land here too
when a worker silently substitutes a remembered number for a missing
data call.

Why prompt rules alone don't fix it
-----------------------------------
worker.py:158-171 already prints generic "use tools when needed"
guidance. LLMs nevertheless prefer the cheap path of recalling a number
over the expensive path of issuing a tool call — especially under the
20-tool-call hard limit imposed two paragraphs later. Adding a stronger
"never quote prices" rule helps but does not eliminate the failure mode.

Fix
---
Make the data unavoidable. SwarmRuntime.start_run now:

  1. Scans every value in user_vars for suffixed symbols
     (NVDA.US / 700.HK / 600519.SH / BTC-USDT / 000001.SZ / 430090.BJ).
  2. Pre-fetches the last 30 days of OHLCV via the existing loader
     registry (resolve_loader handles per-market routing). Failures
     are non-fatal — the run still proceeds, just without grounding.
  3. Pins the result on a new SwarmRun.grounding_data field and
     persists it via the existing run.json path.

SwarmRuntime._execute_run renders the data into a markdown "Ground
Truth" block once per run and threads it through
_run_worker_with_retries -> run_worker -> build_worker_prompt, which
splices it ahead of Execution Rules. The block contains the recent
window range, last close, the last 5 daily closes, and an explicit
instruction to prefer these prices over training-data prices and to
call get_market_data for any range outside the window.

Backward compatibility
----------------------
* SwarmRun.grounding_data defaults to None; existing run.json files
  parse unchanged (Pydantic deserialises the missing key to the
  default). The legacy regression case is covered by
  test_swarm_run_metadata.py once that lands; this PR's tests cover the
  new module and prompt plumbing in isolation.
* build_worker_prompt and run_worker each gain a grounding_block= kw
  with default "" so callers outside the swarm runtime keep working.
* Empty grounding (no symbols detected, or every fetch failed) skips
  the entire section — no empty heading rendered.

What's not in scope
-------------------
* Bare tickers without a suffix (just "NVDA") are intentionally not
  matched. The false-positive cost on common English words is too high
  and every shipped preset already supplies suffixed symbols.
* The block is a one-shot snapshot at run start. Long-running swarms
  may see stale data after many minutes — still strictly better than
  training-data prices from a year ago. A refresh hook can land later.

Tests
-----
agent/tests/test_swarm_grounding.py covers symbol extraction (all four
loader-supported suffix shapes, ordering, non-string values, word
boundaries), fetch behaviour with a stubbed loader (normalised bars,
empty-frame skip, empty-input no-op), markdown rendering (range / last
close / explicit instruction text), and end-to-end prompt plumbing
(grounding block lands before Execution Rules; absent block omits the
section). All 12 pass; full pytest sweep against main shows the same
pre-existing 3 failures (test_backtest_runner_security x2,
test_loop_helpers normalize) as main and 875 passed (= baseline 863 +
the new 12 grounding tests).
Empirical end-to-end verification on a real run revealed the dispatch
in fetch_grounding_data was wrong:

    loader_cls = resolve_loader(code)   # resolve_loader takes a market key
    loader     = loader_cls()           # resolve_loader already returns
                                        # a ready instance, not a class

So a real call with code="NVDA.US" produced
"No available data source for market 'NVDA.US'. Tried: []." — the
fallback chain table is keyed by market type ("us_equity",
"hk_equity", "crypto", ...), not by raw symbol shape.

The unit tests passed only because they monkey-patched resolve_loader
to a `lambda code: lambda: _StubLoader(...)` that happened to satisfy
both the wrong call site and the wrong return-type expectation. A
useful reminder that mock-shaped tests can hide real interface bugs.

Fix
---
- Import the same `_detect_market` runner.py uses for source routing
  and feed its output to `resolve_loader`. The private import is a
  reasonable pragmatic choice — there is no public equivalent today,
  and replicating the regex table in two places would invite drift.
- Drop the `loader_cls()` instantiation step; `resolve_loader` already
  returns a ready loader instance.

Test
----
Updated `test_fetch_returns_normalized_bars` to assert the call is
routed through `"us_equity"` (captured via a stub `resolve_loader`).
This pins the contract: a regression that passes the raw code as the
market key, or rewrites the helper so dispatch goes elsewhere, will
fire here instead of waiting for someone to actually run a swarm.

Verified with the local pipeline:
    NVDA.US latest close = $215.20 on 2026-05-08  (7 bars, yfinance)
i.e. the same number this PR was supposed to make available to the
worker prompt, instead of the empty block the original commit
silently produced.

`pytest agent/tests/test_swarm_grounding.py -v` -> 12/12 pass.
Full sweep at the same baseline (877 passed, 3 pre-existing Windows-
specific failures untouched, separately addressed in #88).
@Teerapat-Vatpitak Teerapat-Vatpitak force-pushed the feat/swarm-prefetch-grounding branch from 0ea5e24 to 8446738 Compare May 10, 2026 13:21

@warren618 warren618 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer follow-up pushed to the PR branch. Verified locally: swarm grounding tests, run metadata/preset tests, py_compile, and diff check pass. Thanks @Teerapat-Vatpitak for the feature.

@warren618 warren618 merged commit b4675d8 into HKUDS:main May 12, 2026
1 check passed
@Teerapat-Vatpitak Teerapat-Vatpitak deleted the feat/swarm-prefetch-grounding branch May 13, 2026 04:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants