feat(swarm): pre-fetch market data and ground worker prompts in real prices#93
Merged
warren618 merged 3 commits intoMay 12, 2026
Merged
Conversation
…prices
Symptom
-------
Workers in fundamental_research_team and similar presets routinely quote
prices from their model's training cutoff. A run targeting NVDA.US in
2026-05 produced a research report anchored on $145.89 while the actual
yfinance close that day was $215.20. Subsequent recommendations
("Hold/Accumulate, target $148-160") were therefore wrong by 30%+ before
any analyst even saw them. issue #42-class "weird outputs" land here too
when a worker silently substitutes a remembered number for a missing
data call.
Why prompt rules alone don't fix it
-----------------------------------
worker.py:158-171 already prints generic "use tools when needed"
guidance. LLMs nevertheless prefer the cheap path of recalling a number
over the expensive path of issuing a tool call — especially under the
20-tool-call hard limit imposed two paragraphs later. Adding a stronger
"never quote prices" rule helps but does not eliminate the failure mode.
Fix
---
Make the data unavoidable. SwarmRuntime.start_run now:
1. Scans every value in user_vars for suffixed symbols
(NVDA.US / 700.HK / 600519.SH / BTC-USDT / 000001.SZ / 430090.BJ).
2. Pre-fetches the last 30 days of OHLCV via the existing loader
registry (resolve_loader handles per-market routing). Failures
are non-fatal — the run still proceeds, just without grounding.
3. Pins the result on a new SwarmRun.grounding_data field and
persists it via the existing run.json path.
SwarmRuntime._execute_run renders the data into a markdown "Ground
Truth" block once per run and threads it through
_run_worker_with_retries -> run_worker -> build_worker_prompt, which
splices it ahead of Execution Rules. The block contains the recent
window range, last close, the last 5 daily closes, and an explicit
instruction to prefer these prices over training-data prices and to
call get_market_data for any range outside the window.
Backward compatibility
----------------------
* SwarmRun.grounding_data defaults to None; existing run.json files
parse unchanged (Pydantic deserialises the missing key to the
default). The legacy regression case is covered by
test_swarm_run_metadata.py once that lands; this PR's tests cover the
new module and prompt plumbing in isolation.
* build_worker_prompt and run_worker each gain a grounding_block= kw
with default "" so callers outside the swarm runtime keep working.
* Empty grounding (no symbols detected, or every fetch failed) skips
the entire section — no empty heading rendered.
What's not in scope
-------------------
* Bare tickers without a suffix (just "NVDA") are intentionally not
matched. The false-positive cost on common English words is too high
and every shipped preset already supplies suffixed symbols.
* The block is a one-shot snapshot at run start. Long-running swarms
may see stale data after many minutes — still strictly better than
training-data prices from a year ago. A refresh hook can land later.
Tests
-----
agent/tests/test_swarm_grounding.py covers symbol extraction (all four
loader-supported suffix shapes, ordering, non-string values, word
boundaries), fetch behaviour with a stubbed loader (normalised bars,
empty-frame skip, empty-input no-op), markdown rendering (range / last
close / explicit instruction text), and end-to-end prompt plumbing
(grounding block lands before Execution Rules; absent block omits the
section). All 12 pass; full pytest sweep against main shows the same
pre-existing 3 failures (test_backtest_runner_security x2,
test_loop_helpers normalize) as main and 875 passed (= baseline 863 +
the new 12 grounding tests).
Empirical end-to-end verification on a real run revealed the dispatch
in fetch_grounding_data was wrong:
loader_cls = resolve_loader(code) # resolve_loader takes a market key
loader = loader_cls() # resolve_loader already returns
# a ready instance, not a class
So a real call with code="NVDA.US" produced
"No available data source for market 'NVDA.US'. Tried: []." — the
fallback chain table is keyed by market type ("us_equity",
"hk_equity", "crypto", ...), not by raw symbol shape.
The unit tests passed only because they monkey-patched resolve_loader
to a `lambda code: lambda: _StubLoader(...)` that happened to satisfy
both the wrong call site and the wrong return-type expectation. A
useful reminder that mock-shaped tests can hide real interface bugs.
Fix
---
- Import the same `_detect_market` runner.py uses for source routing
and feed its output to `resolve_loader`. The private import is a
reasonable pragmatic choice — there is no public equivalent today,
and replicating the regex table in two places would invite drift.
- Drop the `loader_cls()` instantiation step; `resolve_loader` already
returns a ready loader instance.
Test
----
Updated `test_fetch_returns_normalized_bars` to assert the call is
routed through `"us_equity"` (captured via a stub `resolve_loader`).
This pins the contract: a regression that passes the raw code as the
market key, or rewrites the helper so dispatch goes elsewhere, will
fire here instead of waiting for someone to actually run a swarm.
Verified with the local pipeline:
NVDA.US latest close = $215.20 on 2026-05-08 (7 bars, yfinance)
i.e. the same number this PR was supposed to make available to the
worker prompt, instead of the empty block the original commit
silently produced.
`pytest agent/tests/test_swarm_grounding.py -v` -> 12/12 pass.
Full sweep at the same baseline (877 passed, 3 pre-existing Windows-
specific failures untouched, separately addressed in #88).
0ea5e24 to
8446738
Compare
warren618
approved these changes
May 12, 2026
warren618
left a comment
Collaborator
There was a problem hiding this comment.
Maintainer follow-up pushed to the PR branch. Verified locally: swarm grounding tests, run metadata/preset tests, py_compile, and diff check pass. Thanks @Teerapat-Vatpitak for the feature.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
user_varsand renders it as a "Ground Truth" markdown block spliced into every worker's system prompt.SwarmRunfield + 12-test regression suite.Why
Swarm workers are LLMs. Without explicit grounding, they cheerfully quote prices from their model's training cutoff — which is wrong by definition for any asset that has traded since the cutoff. A run targeting
NVDA.USin 2026-05 produced a research report anchored on $145.89 while the actual yfinance close that day was $215.20; subsequent recommendations ("Hold/Accumulate, target $148-160") were wrong by 30%+ before any analyst even saw them. issue #42-class "weird outputs" land here too when a worker silently substitutes a remembered number for a missing data call.Why prompt rules alone don't fix it
worker.py:158-171already prints generic "use tools when needed" guidance. LLMs nevertheless prefer the cheap path of recalling a number over the expensive path of issuing a tool call — especially under the 20-tool-call hard limit imposed two paragraphs later. Adding a stronger "never quote prices" rule helps but does not eliminate the failure mode.What this PR does instead
Make the data unavoidable.
SwarmRuntime.start_runnow:user_varsfor suffixed symbols (NVDA.US/700.HK/600519.SH/BTC-USDT/000001.SZ/430090.BJ).resolve_loaderhandles per-market routing). Failures are non-fatal — the run still proceeds, just without grounding.SwarmRun.grounding_datafield and persists it via the existingrun.jsonpath.SwarmRuntime._execute_runrenders the data into a markdown "Ground Truth" block once per run and threads it through_run_worker_with_retries→run_worker→build_worker_prompt, which splices it ahead ofExecution Rules. The block contains the recent window range, last close, the last 5 daily closes, and an explicit instruction to prefer these prices over training-data prices and to callget_market_datafor any range outside the window.Files
agent/src/swarm/grounding.py(new)agent/src/swarm/models.pygrounding_data: dict[str, list[dict]] | None = NoneonSwarmRun.agent/src/swarm/runtime.pystart_run; render once + thread through worker calls in_execute_run.agent/src/swarm/worker.pygrounding_block: str = ""parameter onbuild_worker_promptandrun_worker; injected ahead of Execution Rules when non-empty.agent/tests/test_swarm_grounding.py(new)Backward compatibility
SwarmRun.grounding_datadefaults toNone; existingrun.jsonfiles parse unchanged (Pydantic deserialises the missing key to the default).build_worker_promptandrun_workereach gaingrounding_block=""so callers outside the swarm runtime keep working.Out of scope
NVDA) are intentionally not matched. The false-positive cost on common English words is too high and every shipped preset already supplies suffixed symbols.swarm→backtest.loaders.registryimport. The alternative (going through the MCP tool registry) was strictly heavier and the current direction matches existing usage from swarm tools, but I'm happy to refactor toward the registry path if that fits the repo's intended layering better.Test plan
pytest agent/tests/test_swarm_grounding.py -v— 12/12 pass.pytest --ignore=agent/tests/e2e_backtest --tb=line -q— same baseline pass count + 12 new tests, no regressions.user_varsproduces no grounding block; auser_varswithNVDA.USproduces a block whose last close matches the real yfinance close for that date.Checklist
agent/src/agent/,agent/src/session/,agent/src/providers/).CONTRIBUTING.md(Conventional Commit prefix, Google-style docstrings, regression test added).run.jsonparses without modification; existing public APIs gain only kwargs with defaults).