LLM experiment
Where does the LLM actually add value?
A small local language model is evaluated in two distinct roles — never assumed to be superior.
LLM experiment
Where does the LLM actually add value?
A small local language model (Ollama) is evaluated, not assumed to be superior. It is tested in two distinct roles.
Experiment A
LLM as a numerical forecaster
Run under a controlled protocol (fixed model/version, temperature 0). The expected null is that a small local LLM rarely beats specialised forecasters on retail series — treated as a finding either way.
Experiment B · primary claim
LLM as inventory decision support
The LLM recommendation is compared against the analytical order-up-to baseline from the same inputs — a measurable "useful?" test, not a vibe check.
Structured output
/* Experiment B — decision support */
{
"action": "ORDER",
"order_quantity": 1260,
"reasoning": "Forecast over lead time plus safety stock exceeds on-hand stock.",
"risk_factors": ["Holding cost if demand runs below forecast"]
}Illustrative example of the output schema — not an experimental result.
Controlled protocol
Reproducible, bounded, honest.
The LLM component runs under a fixed protocol: a small local model via Ollama
with its version pinned, temperature = 0, fixed prompts and a fixed
JSON output schema. It is evaluated on a stratified subsample of
series to bound compute.
Experiment B is the primary claim: the LLM consumes an analytically generated forecast plus inventory context and returns a structured recommendation, which is compared against the analytical order-up-to baseline. In other words, the question is whether the LLM can be a useful decision-support layer over analytical forecasting — not whether it can replace a forecaster or an optimiser.