ClaudeandClaude Opus 5.5
74169e1bfc
The local baseline still claimed "lahiri_assumed_by_skill" after the engine default moved to Raman, so the PyJHora comparison reported 59% instead of ~95% (every longitude 1.446 deg off). The baseline now passes --ayanamsa and --node-mode to the engine and records what the engine reports; the compare takes --ayanamsa and stops when the baseline was built differently. The nine public Rodden AA backtest figures join the fixed samples. Raman run (19 samples): ascendant, planets, nakshatras, D2/D4/D9/D10 and Ashtakavarga all 100%; Shadbala and three old-chart dashas differ (separate). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DbBXoyeZrYn6ryNcSe5Y2Y
Jyotish Benchmark Suite
This directory contains the public benchmark material recovered and sanitized in v6.1.9.
Scope
- Samples: 10 fictional/public smoke cases in
data/benchmark_samples.json. - Scripts: reproducible comparison scripts under
scripts/. - Reports: markdown summary reports under
reports/.
Raw JSON/CSV outputs are intentionally not committed. Re-run the scripts locally to regenerate them under benchmarks/jyotish/outputs/.
Privacy rule
All committed samples are marked fictional_or_public_test. Do not add real user birth data, private chart output, personal life events, PDF extraction text, or private full-reading JSON to this directory.
Running
From the repository root:
python3 benchmarks/jyotish/scripts/run_skill_baseline.py
python3 benchmarks/jyotish/scripts/run_swiss_direct_compare.py
python3 benchmarks/jyotish/scripts/run_transit_true_compare.py
python3 benchmarks/jyotish/scripts/run_shadbala_invariants.py
Some scripts require optional local dependencies such as PyJHora or pyswisseph. If PyJHora is installed outside the default environment, set PYJHORA_SITE or PYJHORA_PATH as needed.
Historical benchmark rounds
The recovered reports document the benchmark sequence used to harden the engine:
- Local full-reading baseline
- Swiss direct planetary comparison
- Swiss extended comparison
- PyJHora comparison
- Mean/True node arbitration
- Arudha/A10 comparison
- Ashtakavarga comparison and book-example arbitration
- Chara Dasha comparison
- True transit comparison
- Shadbala internal invariants
- Explanation regression notes