// SOFTWARE
Hardcoding prompts: a new form of technical debt
6 min readnijitech
When model prompts get scattered through source code, every text change turns into a release. This is exactly what happened to SQL queries a decade ago.
Full post
In an application that gains an AI feature, prompts usually go to the most practical place: inside the function making the call. In the first week that is the right call — it is fast and it works. By month six the same decision has produced the slowest-changing part of the product.
A familiar story. A decade ago SQL queries were embedded in application code too, and every query change required a deployment. The result is the same: a text change becomes an engineering task.
Four costs of a hardcoded prompt
What actually happens
- A one-word correction requires cutting a release
- The same instruction gets copied into several files; one is updated, the other forgotten
- You cannot see which prompt changed when — the cause of a behaviour change cannot be found
- The person who should be writing the text (the domain expert) cannot reach the file
The fourth is the most insidious. A prompt is really product copy: its tone, scope and limits need domain knowledge. Buried in code, the right to write that copy has been taken away from the person who should be writing it.
A prompt is configuration
The fix is not complicated: prompts live somewhere separate from the code, under version control, and the application calls them by an identifier. From then on, changing a prompt is a configuration task rather than a deployment.
Versioning makes a change reversible
When a prompt changes, behaviour changes, and sometimes it gets worse. Without versions there is nowhere to roll back to; all you have is the observation that yesterday was better.
This is where the link to the evaluation set appears. With a versioned prompt and a runnable eval set together, you can measure which category regressed on every change. Missing either one leaves the measurement incomplete.
What happens when the model changes
Models age, and when a cheaper or stronger version ships you want to move. If prompts are scattered through the code, the migration becomes a search-and-replace operation and every prompt has to be tried individually against the new model.
With prompts in one place, the same migration is a configuration change and one eval run. The difference is between hours and days.
Where to start
The first three steps
- Gather every prompt in the codebase into one place — just move them, nothing else
- Give each prompt an identifier and a version number
- Write the next change as a new version; do not delete the old one
None of the three requires an architectural change. The payoff arrives on the first day you need to roll back.
From the glossary: Model orchestration · Evaluation set (eval)