Dispatch 004
Week ↗

Raising the context window

Monday started with a quiet one. I bumped the default context window for the LLM in bolt-diy-fork to 32768. The change touched two files, eleven more lines than it looked like they'd need. `DEFAULTNUMCTX` we…

Commits
1
Systems
3
Read
1min
2024-11-18 · SIGNAL1 commitsbolt-diy-fork1CC-A-LLM1MagiCode1
Commit signal

Rendered from this day’s 1 commit — no stock art.

Monday started with a quiet one. I bumped the default context window for the LLM in bolt-diy-fork to 32768. The change touched two files, eleven more lines than it looked like they'd need.

DEFAULT_NUM_CTX went into the model config module and got mirrored in .env.example so anyone spinning things up locally doesn't trip over a hard-coded ceiling. It's a tuning knob that had been silently throttling longer conversations, and the fix was just making it explicit with a variable instead of buried numbers.

The thirty-two-k threshold is a common cap for models in this space. It's the kind of thing that sits there unnoticed until someone is like why does this conversation forget what we said three turns ago. Then suddenly there's a PR.

It's the kind of thing that sits there unnoticed until someone is like why does this conversation forget what we said three turns ago.

Also today: it was the kind of commit you make before coffee and double-check twice. Twelve additions, one deletion. Sometimes the day is just a small lift done right.

No fanfare, no breaking changes. Context window raised, .env.example kept in sync. Next time someone complains the model short-filters their second paragraph, they'll have more room to breathe.