fix(backtest): create artifacts dir before writing validation.json#429
Merged
warren618 merged 1 commit intoJul 8, 2026
Merged
Conversation
When config["validation"] is set, run_backtest() writes run_dir/artifacts/validation.json in step 7, but run_dir/artifacts is only created by _write_artifacts() in step 8. If the directory does not already exist (e.g. a swarm agent workspace), the write raises FileNotFoundError and aborts the backtest. Ensure the parent directory exists before the write. Adds a regression test that runs a backtest with validation enabled against a run_dir with no pre-created artifacts/ and asserts validation.json is written. Fixes HKUDS#427 Signed-off-by: isaveall <isaveall0@gmail.com>
This was referenced Jul 8, 2026
Collaborator
|
Belated note for the record: merged as-is (7311d6b) — clean 4-line fix, verified locally with a regression test before landing. Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
When
config["validation"]is set,BaseEngine.run_backtest()writesrun_dir/artifacts/validation.json(step 7) before_write_artifacts()createsrun_dir/artifacts(step 8). If that directory doesn't already exist, the write raisesFileNotFoundErrorand aborts the backtest.This surfaced when running an
investment_committeeswarm — theportfolio_managersub-agent'srun_dirhas no pre-createdartifacts/, so its validation-enabled backtest crashed (the swarm was fault-tolerant and still finished, but the PM's backtest artifact was lost).Fix
Ensure the parent directory exists immediately before the write:
One line + a short explanatory comment. The bug is not platform-specific.
Test
Adds
TestValidationArtifactDirtotests/test_engine_robustness.py: runs a backtest with validation enabled against arun_dirthat has no pre-createdartifacts/, and assertsvalidation.jsonis written.FileNotFoundError.tests/test_engine_robustness.py: 32 passed.Fixes #427