WindCast Agent forecasts the hourly output of two wind turbines in the Shelek wind corridor (Almaty region, Kazakhstan; coordinates from the brief) and runs the forecasting cycle autonomously: fetch archived forecast fields (Open-Meteo Previous Runs API, 5 NWP sources) → build 108 features → run a LightGBM×5 ensemble with an isotonic power curve kept as a fallback → produce an hourly 24–48 h forecast with a P10–P90 band → validate and analyse the result → compare with the previous issue and repeat the cycle for every requested date. The organisers' test period is 1–28 February 2026, rolling issues from 31 January. An LLM with typed tools orchestrates the cycle (OpenAI gpt-6-luna by default); the same state graph runs deterministically in --no-llm mode, so verification needs neither our API keys nor network access.

Boundary of the current implementation: the CLI replays the dates of a requested range; there is no weather-update watcher and no scheduler. Previous Runs stitches several model runs, and publication delays can push part of the values past the end of day D: availability of the whole slice at the historical issue hour is not proven (weather-input audit).

📌 Project status

⚡ For judges: verify in 5 minutes

Everything is in the repository: the organisers' data, the cache of every weather request, trained models, the rebuilt submission and its offline reports. After installing dependencies, no network and no API keys are needed — --no-llm yields the same forecast numbers.

Prerequisites: Git access to this private repository and Python 3.12 on Linux/macOS, or Docker with Compose v2. Python 3.12.6 and 3.12.14 were tested; newer minor versions were not. On macOS, LightGBM needs brew install libomp; on Debian/Ubuntu it needs sudo apt-get update && sudo apt-get install -y libgomp1. Docker includes this library. Installation needs internet access; the verification below then runs from the committed cache.

git clone https://github.com/BAITC-Hacks/hack-f0ab74e4-knewit-3.git
cd hack-f0ab74e4-knewit-3                              # skip clone/cd if already at the repo root
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt

# 1) Submission integrity: 1344 turbine-hours, [0,1], 48 h per issue (27 Feb — 24 h), freshest issue per hour
python -m scripts.verify_submission                   # expected: OK: 1344 turbine-hours

# 2) Offline replay, evaluation, manifest and numeric comparison (new or empty output directory)
WINDCAST_RUN_DIR="runs/judge-$(python -c 'import uuid; print(uuid.uuid4().hex[:8])')"
python -m scripts.reproduce --output-dir "$WINDCAST_RUN_DIR"

# 3) Tests and the operator dashboard
python -m pytest tests/ -q                            # expected: 145 passed
python -m streamlit run app.py -- --forecast-dir "$WINDCAST_RUN_DIR"  # http://localhost:8501

Run these commands in the same terminal. Each replay gets a new directory, so repeating the check preserves previous results. The dashboard stays running until Ctrl+C; opening it is optional.

You will see 28 completed issues and eight OK checks, including 1344 submission rows and 5904 evaluation rows. The dashboard shows the agent's execution graph with per-node timing, the hourly forecast with its P10–P90 band, the spread of the five NWP sources, the agent's report and a structured log. Step-by-step route: docs/JUDGE_GUIDE.md (RU).

Automated review and common setup issues

• Headless / AI reviewer: run scripts.verify_submission, scripts.reproduce and pytest above. A browser, .env, LLM key, training and live weather download are not required. Check exit codes; the replay writes a machine-readable reproduction_report.json inside $WINDCAST_RUN_DIR.
• Expected evidence: 145 tests with zero skips; eight replay checks pass; 1344 submission rows and 5904 evaluation rows match at 1e-9. Reproduction proves saved-result consistency; accuracy is reported separately in models_artifacts/evaluation/evaluation_report.json.
• Repository unavailable: the organiser must grant your GitHub account access. An API key for an LLM does not grant repository access. Run all commands from the cloned repository root.
• libomp / libgomp error: install the system library listed above, or use Docker below. If dependency installation fails on Python 3.13+, recreate the environment with Python 3.12.
• Output directory is not empty: choose a new directory; no deletion or --overwrite is needed. Keep the supplied data/, models_artifacts/ and forecasts/ intact for verification.
• Port 8501 is busy: add --server.port 8502 before the -- in the Streamlit command.
• Red GitHub Actions checks: as of 23 September 2026, the organisers' organisation billing lock prevents runners from starting. This is not a test result. Local macOS/Linux evidence and repeatable commands are in REPRODUCIBILITY.

python -m src.cli run-agent --start 2026-02-10 --end 2026-02-10 --no-llm --output-dir runs/demo
cat runs/demo/report_2026-02-10.md
python -c "import json;t=json.load(open('runs/demo/trace_2026-02-10.json'));print([(s['node'],s['status'],round(s['ms'])) for s in t['trace']])"

✅ Brief compliance: traceability matrix

🏅 Against the scoring criteria

📊 Results

The finished submission: February 2026

forecasts/submission.csv — 1344 rows (turbine, datetime, lead_day, power_pred), timestamps in Asia/Almaty as in the dataset. Each hour takes the freshest available issue, hence lead_day = 1 throughout; lead-2 forecasts of the same hours live in the daily CSVs and are used to compare power changes. Mean forecast load for the month: T1 0.465, T2 0.462 of rated capacity.

Retrospective ensemble evaluation

The evaluation ensemble was trained on targets through 30 Nov 2025 and tested on 1 Dec 2025 – 31 Jan 2026 by exactly the submission path (get_issued_forecast → build_features → predict). The table uses clean_targets: known targets with ≥4 of 6 ten-minute readings and no heuristically detected downtime (rules). MAE of normalised power: 0.158 = a mean error of 15.8 % of rated capacity.

5904 rows saved, no issue missing; 48 lead-2 rows for 1 Dec excluded because their issue would precede the training cutoff. P10–P90 coverage is 64.3 % against a nominal 80 %: the intervals are still too narrow — an open task. Metrics over all observed hours (downtime included) are in evaluation_report.json.

Important caveat. December–January had earlier influenced source selection and tuning, so this is a retrospective evaluation, not an untouched test. The team has no February 2026 actuals — we do not invent accuracy on the organisers' test.

🧱 Architecture

One issue cycle — graph implementation

flowchart LR
    W[("Open-Meteo<br/>Previous Runs<br/>5 NWP sources")] --> A
    subgraph DayGraph["DayGraph · src/agent/graph.py · one graph for LLM and --no-llm"]
        direction LR
        A["fetch_weather<br/>select the archived<br/>weather slice"] --> B["prepare_features<br/>prepare data<br/>up to 48 h × 108 features"]
        B --> C["run_model<br/>run the model<br/>LightGBM×5 + baseline + P10/P90"]
        C --> D{"validate_forecast<br/>bounds, completeness,<br/>NaN; flatline warning"}
        D -->|ok| E["compare_with_previous<br/>power change<br/>vs issue D−1"]
        D -->|first failure| R["recover_baseline<br/>recompute<br/>with the power curve"]
        R --> D
        D -->|second failure| X["stop: day not published,<br/>error trace saved"]
        E --> F["write_outputs<br/>hourly forecast CSV<br/>+ operator report"]
    end
    F --> O[("forecasts/<br/>forecast_t1,t2_D.csv<br/>report_D.md · trace_D.json")]
    O -.->|next CLI date| A

The LLM (OpenAI gpt-6-luna by default; Claude or NVIDIA NIM via environment variables) receives next_tool after every call, analyses the results and writes the report; an inadmissible transition does not change state. An API outage resumes the same graph from the current node without re-fetching weather or re-running the model — behaviour pinned by tests. Recovery uses the already-computed physical curve on the same archived weather issue: re-reading an unchanged cache is never passed off as a source update.

flowchart TB
    subgraph src["Sources"]
        S1["SCADA 10-min<br/>data/raw/turbine_1,2.csv<br/>11 Mar 2023 – 31 Jan 2026"]
        S2["NWP forecast archive<br/>data/weather_cache/ · 76 JSON<br/>Mar 2024 – Feb 2026"]
    end
    S1 --> H["Hourly aggregation<br/>≥4/6 intervals · downtime filter<br/>src/features/dataset.py"]
    S2 --> I["Slices by issue date<br/>get_issued_forecast · lead 1/2<br/>src/weather/openmeteo.py"]
    I --> F["108 features per slice<br/>build_features<br/>src/features/build.py"]
    H --> T
    F --> T["training_matrix<br/>training and inference share slices<br/>targets ≤ 31 Jan 2026"]
    T --> M["LightGBM×5 seeds · objective L1<br/>isotonic curve · quantiles 0.1/0.9<br/>src/models/train.py"]
    M --> P["models_artifacts/turbine_1,2.pkl<br/>train_report.json · manifest.json"]
    T --> EV["Evaluation protocol<br/>fit ≤ 30 Sep · tune Oct–Nov 2025 · evaluate Dec 2025 – Jan 2026<br/>src/backtest/evaluate.py"]
    EV --> ER["models_artifacts/evaluation/<br/>evaluation_predictions.csv · evaluation_report.json"]

Modules and contracts

Details: docs/ARCHITECTURE.md (RU). Dashboard: streamlit run app.py — the agent's execution graph with per-node status and timing, the forecast with its P10–P90 band, NWP source spread, report and decision log. Two modes: Test period (February 2026) reads forecasts/ or --forecast-dir; Retrospective evaluation reads the saved models_artifacts/evaluation/ predictions — the final model is never recomputed (docs/DASHBOARD.md). Unfinished runs are hidden.

🚀 Running

python3.12 -m venv .venv && source .venv/bin/activate
python -m pip install -r requirements.txt

WINDCAST_RUN_DIR="runs/judge-$(python -c 'import uuid; print(uuid.uuid4().hex[:8])')"
python -m scripts.reproduce --output-dir "$WINDCAST_RUN_DIR"
python -m pytest tests/ -q
python -m streamlit run app.py -- --forecast-dir "$WINDCAST_RUN_DIR"

docker compose up --build -d         # dashboard on http://localhost:8501; returns to the terminal
docker compose run --rm --build reproduce  # no network, strict tolerance 1e-9
docker compose down                 # stop the dashboard after review

test -e .env || cp .env.example .env                 # OPENAI_API_KEY (or ANTHROPIC_API_KEY / NVIDIA NIM), see comments in the file
python -m src.cli list-models        # which models your key can access
python -m src.cli run-agent --start 2026-02-10 --end 2026-02-10 --output-dir runs/llm-demo
streamlit run app.py -- --forecast-dir runs/llm-demo

python -m src.cli train                                    # retrain → REPLACES models_artifacts/
python -m src.cli check-tz                                 # hour-shift diagnostic: peak of forecast/measured wind correlation
python -m src.backtest.evaluate --output-dir runs/eval     # retrospective evaluation by protocol
python -m scripts.compare_runs --a forecasts --b runs/x    # diff of two runs
python -m src.backtest.experiments                         # spatial gradients (ADR-007)
python -m src.backtest.prof_ideas                          # two-stage calibration (ADR-008)

📁 Repository structure

.
├── README.md · README.ru.md · README.kk.md   ← this file (EN default), Russian, Kazakh
├── requirements.txt              ← package versions pinned; verified on Python 3.12.6
├── Dockerfile · docker-compose.yml · .env.example
├── app.py                        ← operator dashboard (Streamlit + Plotly)
├── src/
│   ├── config.py                 ← coordinates, periods, 5 NWP sources, weather variables
│   ├── cli.py                    ← train · run-agent · check-tz · list-models
│   ├── weather/openmeteo.py      ← Previous Runs API, cache, slices by issue date
│   ├── features/dataset.py       ← SCADA → hourly, downtime filter
│   ├── features/build.py         ← 108 features, issue stack, training_matrix
│   ├── models/train.py · predict.py
│   ├── agent/graph.py · tools.py · llm.py · loop.py
│   └── backtest/evaluate.py · experiments.py · prof_ideas.py
├── data/
│   ├── raw/turbine_1.csv · turbine_2.csv     ← organisers' dataset (10-min, Mar 2023 – Jan 2026)
│   └── weather_cache/*.json                  ← 76 saved Open-Meteo responses → offline runs
├── models_artifacts/
│   ├── turbine_1.pkl · turbine_2.pkl         ← submission models, trained through 31 Jan 2026
│   ├── train_report.json · manifest.json     ← tuning metrics, versions, SHA256
│   └── evaluation/                           ← evaluation models (through 30 Nov 2025), prediction CSV, metrics JSON
├── forecasts/                                ← CANONICAL OUTPUT (updated only by the integrator)
│   ├── submission.csv                        ← 1344 rows, February 2026
│   ├── forecast_t{1,2}_YYYY-MM-DD.csv        ← 28 × 2 CSV: 48 hours; the last issue — 24
│   ├── report_YYYY-MM-DD.md                  ← operator report (current bundle: --no-llm template)
│   └── trace_YYYY-MM-DD.json                 ← graph trace: nodes, statuses, timing, next_tool
├── scripts/verify_submission.py · compare_runs.py · plot_holdout.py
├── tests/                                    ← 145 tests
└── docs/                                     ← brief, architecture, data, decisions, evaluation, judge route (RU)

Documentation (RU): CASE.md the official brief and rubric · JUDGE_GUIDE.md step-by-step judge route · DASHBOARD.md dashboard modes and data sources · ARCHITECTURE.md graph, training, module contracts · DATA.md dataset profile and cleaning rules · RESEARCH.md weather sources and diagnostics · DECISIONS.md 11 ADRs incl. negative results · EVALUATION.md evaluation protocol · FEATURE_AVAILABILITY.md archive availability · EXPERIMENTS.md experiment commands · INTEGRATION.md verified checks · NEXT_STEPS.md · TASKS.md · GIT_WORKFLOW.md.

Tests (tests/, 145 passing): the agent graph behaves identically in every mode, an LLM outage resumes from the saved node, early writes are rejected, failed validation recovers once or refuses to publish; training features equal inference features for the same issue and lags never cross an issue boundary; future targets never enter training; fit/tune/evaluate masks are disjoint and metrics recompute from CSV; missing wind stays NaN; --output-dir isolation; the independent verifier; the dashboard renders both modes and hides unfinished runs. Before publishing: python -m pytest tests/ -q · python -m scripts.verify_submission · git diff --check.

🔬 Experiments and negative results

The numbers below are preserved in historical ADRs from before the feature-slice fix. They describe the earlier selection; the repository holds no comparable CSVs for recomputing these experiments.

In the earlier experiment a calibration regressor gave a wind MAE of 1.801 vs 1.779 m/s for the NWP mean (ADR-008). That is the result of one configuration; the optimality of the mean does not follow from it.

🚧 Limitations (honest)

• February 2026 accuracy is unknown — the actuals are with the organisers. All our metrics are retrospective.
• December–January had earlier influenced source selection and tuning, so the evaluation is not an untouched test.
• P10–P90 intervals under-cover (64.3 % vs nominal 80 %) — do not call them a calibrated 80 % interval.
• Previous Runs stitches forecasts from different model runs and does not store the publication time of each cell. Given publication delays, part of the selected values may appear only after day D; availability of the whole slice at a fixed issue hour is not proven (FEATURE_AVAILABILITY).
• Downtime and curtailment are unpredictable from weather: the model forecasts available wind generation, not dispatcher decisions; metrics over all observed hours differ slightly (see all_observed in the evaluation report).
• GitHub Actions cannot run until the organisation billing lock is resolved. Local Linux Docker replay and tests passed; see verification.
• A scheduler, NWP-update watching, input-distribution drift control and automatic retraining are not implemented; compare_with_previous measures only the change of the power forecast.

🔭 Development potential and originality

1. A neural NWP as an additional source. Check ECMWF AIFS: coverage of the required dates, variables and publication time, then compare under the replay protocol.
2. Interval calibration: a conformal correction of P10–P90 from tuning-period residuals; target coverage 80 %.
3. Analog ensemble (AnEn): retrieval of similar historical forecast situations — empirical uncertainty and references to comparable hours with measured generation inside the agent's report.
4. Live SCADA features, if the operator provides them with a verifiable availability time; separate drift control of the weather inputs and of quality, then automatic retraining; balancing-market bidding from the P10–P90 band. Availability of archived EPS members and their contribution to interval quality needs its own study.

What the current version combines: MOS correction and the wind-to-power dependence in one model on forecast weather; the oracle diagnostic that steered effort into the weather input; a state graph with recovery and traces instead of a free LLM loop; an independent submission verifier; issue-date replay with saved provenance of every row.

📚 Data and third-party components

• Organisers' dataset: data/raw/turbine_{1,2}.csv — 10-minute series 11 Mar 2023 – 31 Jan 2026 (142 360 and 149 499 rows; 6.6 % and 1.9 % gaps, T1 has a 41-day hole in May–June 2024). Processing rules: docs/DATA.md.
• Geolocation from the brief's links: T1 43.645150, 78.535604, T2 43.643198, 78.538828 (~300 m apart → one weather point, separate models).
• Weather: Open-Meteo Previous Runs API; the responses used by the saved run live in data/weather_cache/.
• Libraries: pandas, NumPy, scikit-learn, LightGBM, httpx, Streamlit, Plotly, matplotlib, pytest, an OpenAI-compatible HTTP client, anthropic SDK.
• LLM in the product: default model gpt-6-luna (OpenAI Chat Completions, tool calling); Claude and NVIDIA NIM adapters. The current canonical run is offline; the exact model ID of the earlier live run was not recorded.
• AI development assistants: OpenAI Codex CLI and Claude Code, per the hackathon rules; architecture, ML decisions and verification are the team's.
• Literature: Glahn & Lowry (1972) MOS; Hong et al. (2016) GEFCom2014; Giebel & Kariniotakis (2017); Draxl et al. (2015) — docs/RESEARCH.md.

👥 Team

KnewIT 3 — 3 developers. Work split: docs/TASKS.md; git process and the run-owner rule: docs/GIT_WORKFLOW.md. Repository: https://github.com/BAITC-Hacks/hack-f0ab74e4-knewit-3.