Skip to content

fix(backtest): create artifacts dir before writing validation.json#429

Merged
warren618 merged 1 commit into
HKUDS:mainfrom
isaveall:fix/validation-json-missing-artifacts-dir
Jul 8, 2026
Merged

fix(backtest): create artifacts dir before writing validation.json#429
warren618 merged 1 commit into
HKUDS:mainfrom
isaveall:fix/validation-json-missing-artifacts-dir

Conversation

@isaveall

@isaveall isaveall commented Jul 7, 2026

Copy link
Copy Markdown

What

When config["validation"] is set, BaseEngine.run_backtest() writes run_dir/artifacts/validation.json (step 7) before _write_artifacts() creates run_dir/artifacts (step 8). If that directory doesn't already exist, the write raises FileNotFoundError and aborts the backtest.

This surfaced when running an investment_committee swarm — the portfolio_manager sub-agent's run_dir has no pre-created artifacts/, so its validation-enabled backtest crashed (the swarm was fault-tolerant and still finished, but the PM's backtest artifact was lost).

Fix

Ensure the parent directory exists immediately before the write:

v_path = run_dir / "artifacts" / "validation.json"
v_path.parent.mkdir(parents=True, exist_ok=True)
v_path.write_text(...)

One line + a short explanatory comment. The bug is not platform-specific.

Test

Adds TestValidationArtifactDir to tests/test_engine_robustness.py: runs a backtest with validation enabled against a run_dir that has no pre-created artifacts/, and asserts validation.json is written.

  • With the fix: passes.
  • Reverting only the fix: fails with the exact FileNotFoundError.
  • Full tests/test_engine_robustness.py: 32 passed.

Fixes #427

When config["validation"] is set, run_backtest() writes
run_dir/artifacts/validation.json in step 7, but run_dir/artifacts is
only created by _write_artifacts() in step 8. If the directory does not
already exist (e.g. a swarm agent workspace), the write raises
FileNotFoundError and aborts the backtest.

Ensure the parent directory exists before the write. Adds a regression
test that runs a backtest with validation enabled against a run_dir
with no pre-created artifacts/ and asserts validation.json is written.

Fixes HKUDS#427

Signed-off-by: isaveall <isaveall0@gmail.com>
@warren618

Copy link
Copy Markdown
Collaborator

Belated note for the record: merged as-is (7311d6b) — clean 4-line fix, verified locally with a regression test before landing. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants