How I Built a Technical-Reasoning Prompt Through Iteration (Instead of Accumulation)

Why I'm writing this

I use LLMs daily for programming, hardware design, and debugging, and at some point I got tired of prompts that read like a wishlist: "be concise, be thorough, consider security, consider performance, use best practices..." Every one of these instructions sounds reasonable in isolation. Stacked together, they compete with each other, and none of them are actually verifiable — there's no way to check whether the model "followed" an instruction like "be thorough" versus just producing something that sounds thorough.

So instead of writing a prompt in one sitting and calling it done, I decided to build one the way I'd want a piece of software built: propose a change, write a test for it, check whether it actually does what I claim, and only then keep it. This is the writeup of that process — what I kept, what I threw out, and why.

The final prompt is about 300 words. The length isn't the point. The point is that every line in it survived a concrete test case, and a few lines I was initially sure about got rewritten or cut when they failed one.

1. Where I started

The common failure mode I wanted to avoid looks like this:

- Be concise.
- Be thorough.
- Be accurate.
- Be helpful.
- Think deeply.
- Consider security.
- Consider performance.
- Explain trade-offs.
- Use best practices.
- Don't overcomplicate.

None of these are wrong, exactly. But:

1. They compete with each other. "Be concise" and "be thorough" give no signal about which one wins for a given question.

2. They're not observable behaviors. I can't check if a response "used best practices" the way I can check if it did or didn't include a specific piece of reasoning.

3. They get added without ever being tested. Someone tries a rule, it sounds good, it goes in. Nobody goes back and checks if it actually changed anything.

Before touching my own prompt, I first went through the usual list of "prompting tricks" and sorted which ones I actually trust and which ones I don't:

• The good: Giving the model a role does something — though, as I found out later while researching this (more on that in Section 6), what it does is narrower than I assumed.

• The good: Giving explicit constraints (one paragraph, no jargon, include examples) reliably improves output.

• The good: Asking the model to critique itself or challenge assumptions is genuinely useful.

• The Bad: Saying "we talked about this yesterday" does nothing if there's no actual prior context in the conversation.

• The Bad: Giving the model an IQ score doesn't make it smarter — at most it nudges the tone toward sounding more analytical.

• The Bad: Betting fake money has no real effect; framing the task as "find the flaws" is what actually helps, not the bet itself.

That filtering exercise is what convinced me the real lever isn't "which tricks do I stack" — it's "which behaviors can I actually specify and verify."

2. The shift that mattered most: treat the prompt as a spec, not a wishlist

Here's the reframing that changed everything for me: a rule tells the model what to do. An example shows it how that looks in an ambiguous, concrete situation. And LLMs don't execute rules like a deterministic program — they weigh a pile of competing instructions against each other. The more abstract rules I throw in, the fuzzier the actual priority gets.

I found this lines up with what's actually published on the topic, which was reassuring. The O'Reilly book Prompt Engineering for Generative AI (Phoenix & Taylor, 2024) frames effective prompting around five principles, and "Provide Examples" is called out as its own distinct lever, separate from just giving direction^[1]. That matches the core capability behind in-context learning in the first place — the GPT-3 paper, Language Models are Few-Shot Learners (Brown et al., 2020), is literally the paper that first showed models adapting to a task from examples in the prompt, no retraining involved^[2].

So I restructured my prompt around two blocks: short core principles, and a calibration block with concrete cases — instead of trying to write an exhaustive checklist of everything I wanted the model to consider.

3. The actual process: propose, test, keep or kill

Every rule I considered went through the same four steps:

1. Find a concrete failure, not a vague "this could go wrong somewhere."

2. Design a minimal test case that would specifically expose that failure.

3. Simulate the answer with and without the rule, to check it actually changes behavior — not just that it sounds good.

4. Rewrite it as an example instead of an abstract statement whenever that was possible.

A few things I was initially confident about got corrected along the way:

What I initially thought	Why it didn't hold up	What I changed it to
"Give more space to the most severe issue in a code review"	A severe issue can be simple to explain ("use parameterized queries" is one line); forcing extra length just pads the response	Changed to emphasis and ordering, not length: lead with critical issues, give minor ones less weight
"The model shouldn't agree with me by default"	Makes it unnecessarily combative on simple questions where I'm actually right	"Challenge assumptions only when there's a genuine technical reason"
"Add a Confidence: High/Medium/Low tag"	With no defined criteria per level, it's just a decorative label	Dropped it. Replaced later with an actual verification rule (Section 5)
"Don't confuse correlation with causation" as a flat statement	The model already knows the phrase — stating it doesn't make it apply that logic to the actual case in front of it	Rewrote as a calibration example: "After upgrading X, Y happened, therefore X caused Y" → treat as a hypothesis, propose how to check it
"Always write parametrized, abstracted methods"	Directly contradicts "avoid unnecessary abstraction," which was already in the prompt — pushes toward over-engineering	Realized the real axis isn't "parametrize or not" — it's whether the code's declared purpose is to be shared
Giving the whole thing a score like "top 1%"	Same lack of rigor I was trying to get away from — an opinion dressed up as a measurement	Rejected it outright; documented the real limits instead (Section 7)

The corrections mattered more than the additions. Every time I caught myself defending a rule because "it sounds like good practice," that was the signal to go find a test case instead of trusting the instinct.

4. The pattern I almost missed: "declared purpose," not "possible reuse"

The most useful correction came from my own pushback on an earlier version. I work with QA automation and firmware, and reusable code in both of those worlds — page objects, HAL functions — tends to be parametrized from the start, which seemed to clash with "avoid unnecessary abstraction."

Once I looked at it carefully, it wasn't a domain exception. The question I'd been asking was wrong. "Could this code be reused someday?" is speculative — almost anything could, in theory. The right question is "was this code written with the explicit purpose of being called from multiple places?" That's verifiable the moment you write it — a page object, a shared utility, a driver, an API handler are built to be called from many places by design. A one-off init routine or a throwaway script is not.

What convinced me this was a real principle and not a patch: it generalized cleanly across two domains I hadn't originally been thinking about together — QA page objects and embedded C drivers — without needing a separate rule for each. That's the best evidence I got, in this whole process, that a well-defined principle beats a pile of domain-specific patches.

5. Extending it past code: things you should check instead of guess

Once I started using this for hardware and research questions too, a new kind of failure showed up: confidently stating something that's actually a measurable fact — a voltage tolerance, how an API actually behaves right now — with the same tone I'd use for something stable and well-known.

I split this into two related but distinct cases:

• Something I can check right now — a real value (multimeter, curl, datasheet lookup).

• Something that might be out of date — "the standard tool for X" in an area that moves fast, which might just be reflecting the model's training data instead of the current state of the field.

I had to be careful to scope this narrowly: it does not apply to stable knowledge — syntax, algorithms, basic math. If I hadn't drawn that line, the rule would have made the model second-guess things that never needed it, which is exactly the same "rule creep" problem I was trying to get away from in the first place.

6. What actually makes this different from a typical "prompt engineer" template

The usual approach	What I did instead
Stacks techniques that "work," without asking why	Every rule had to tie to a mechanism I could check, or it got cut
Judges quality by first impression ("this looks way better now")	Required a with/without comparison before accepting a rule, and I flagged explicitly when that comparison wasn't rigorous
Treats a longer prompt as a better prompt	I actively cut redundant rules — the goal was the smallest set with the most coverage, not the longest list
Doesn't separate "rule" from "example"	I kept these as two distinct blocks on purpose, because examples generalize better than adjectives
Never checks whether its own rules conflict	I went looking for tension between rules on purpose (standard solution vs. better alternative; brevity vs. depth) instead of assuming they'd coexist fine
Hands you a benchmark score with no rubric	I explicitly rejected giving this a made-up score ("top 1%," star ratings) and wrote down what my testing actually does and doesn't prove

The real difference isn't in the text of the prompt. It's that I treated the whole design as a hypothesis I was trying to break, not a list I was trying to finish.

This also tracks with some of the published research on self-correction. The "review your own reasoning before answering" step I added is basically the mechanism behind Self-Refine (Madaan et al., 2023) — a model critiques its own output in natural language and revises it, no extra training involved, with reported average gains around 20% across a range of tasks^[3]. But the follow-up literature on that same method also documents a real risk: models tend to rate their own output more favorably than an outside judge would — a self-bias effect^[4]. That's exactly why I kept flagging my own single-model, single-sample tests as weak evidence instead of proof throughout this whole process.

I also want to be upfront about something the research complicated for me. My own opening list said giving the model a role "often improves consistency" — I believed that going in. A large study across four LLM families and 2,410 factual questions found that adding personas to system prompts doesn't reliably improve performance on objective tasks, and that the effect of any given persona is mostly unpredictable, sometimes even negative^[5]. A newer 2026 paper nuances that a bit further: a role mostly reshapes how the model communicates — more structure, more of an "expert" tone — rather than whether the underlying answer is right, and it seems to actually help in advisory domains like medicine while sometimes hurting plain explanatory answers^[6].

That doesn't mean I should drop the "senior software engineer" role from my prompt. It does mean I should be honest about what it's probably doing: shaping tone and structure, not adding real problem-solving capability. Which, if anything, backs up the approach I took — the actual behavior change (challenging assumptions, verifying facts, prioritizing by severity) is carried by the explicit rules and calibration examples, not by the role on its own.

7. What I think this prompt actually does well

1. Scales with how complex and risky the question is — no one-line answers where they're not enough, no ten-section reports for a trivial question.

2. Prioritizes findings by severity instead of listing a critical bug and a naming nitpick with the same visual weight.

3. Tells apart stable knowledge from perishable/checkable facts, so it's less likely to state a specific number with more confidence than it's earned.

4. The "declared purpose" rule for abstraction held up across application code, QA automation, and embedded firmware without needing separate rules per domain.

5. Treats causation as a hypothesis, which is a mistake I see constantly in both software debugging and interpreting research results.

6. I actually tested it, within what was possible for me at the time. I didn't have API access for a proper automated A/B test, so I generated real responses (not just predicted ones) with and without the prompt, across code, electronics, and research questions — and caught at least one real bug in the process: a hardware block diagram that didn't match what the accompanying text said about it.
   

8. What I can't honestly claim

I want to be as careful here:

1. None of this is statistically validated. Every test was a single sample, no blind judge, no repeats, mostly generated by the same model being evaluated — the same self-bias risk the Self-Refine follow-up work documents^[4].

2. I never compared this against a different design approach — full-conversation few-shot examples instead of rules + calibration, or a longer prompt with dedicated domain sections. I can't say this beats other approaches, only that it's a well-executed version of this one.

3. My calibration examples are shaped by the specific cases I happened to test. Domains I haven't touched — medicine, law, advanced physics, security research — could expose gaps I haven't seen yet.

4. Deciding "does this deserve its own example" was ultimately my own judgment call, not an objective metric. Someone else might reasonably draw that line differently.
   

9. What I actually walked away with

The most useful thing I got out of this wasn't the prompt text — it was the discipline: treat every candidate rule as something that has to survive a real test case, prefer a concrete example over an abstract statement whenever I can, go looking for conflicts between my own rules instead of assuming they play nice, and be honest about what my testing does and doesn't actually prove.

That discipline is the part I'd reuse on any prompt, in any domain. The specific text below is just what it produced this time.

10. Sources I checked while writing this

1. Phoenix, J., & Taylor, M. (2024). Prompt Engineering for Generative AI. O'Reilly Media. — Frames effective prompting around five principles, with "Provide Examples" called out separately from giving direction.

2. Brown, T., et al. (2020). Language Models are Few-Shot Learners. arXiv:2005.14165. — The original description of in-context learning: models adapting to a task from prompt demonstrations, no retraining.

3. Madaan, A., et al. (2023). Self-Refine: Iterative Refinement with Self-Feedback. arXiv:2303.17651. — A model critiques and refines its own output iteratively, no extra training, with substantial reported gains across tasks.

4. Follow-up literature on Self-Refine documenting a self-bias effect — models rating their own output more favorably than an external judge would.

5. Zheng, M., Pei, J., Logeswaran, L., Lee, M., & Jurgens, D. (2024). When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models. Findings of ACL: EMNLP 2024. arXiv:2311.10054.

6. (2026). When Does Persona Prompting Actually Help? A Retrieval and Metric Analysis of Expert Role Injection in LLMs. arXiv:2605.29420. — Persona prompting reshapes style more than correctness; benefits depend heavily on the domain.

7. Wei, J., et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903. — Background on structured step-by-step reasoning, relevant to the "separate facts, hypotheses, and assumptions" rule I use for debugging.

8. Meincke, L., et al. (2025). Prompting Science Report 2: The Decreasing Value of Chain of Thought in Prompting. arXiv:2506.07142. — A useful reminder that no single prompting technique transfers universally — CoT-style prompting doesn't help uniformly and can add variance depending on the task and model.
   

Appendix A: The final prompt I ended up with

You are a senior software engineer, technical architect, and analytical problem solver.

Your objective is to provide the technically strongest answer, not simply agree with my assumptions.

Core principles:
- Prefer correctness over agreement.
- Recommend proven solutions before unconventional ones.
- Challenge assumptions only when there is a genuine technical reason.
- Ask for clarification when missing information materially changes the answer.
- For facts that are state-dependent or perishable (a measured value, an API's current behavior, "the standard tool" in a fast-moving area) — verify before asserting, or state the uncertainty and how to check. Stable technical knowledge doesn't need this.
- Match reasoning, structure, and detail to the complexity AND risk of the problem.
- Prioritize issues by severity and impact instead of presenting all findings equally.

When designing:
- Prefer simple, maintainable solutions.
- Explain trade-offs.
- Avoid unnecessary complexity.

When writing code:
- Use descriptive naming.
- Handle errors appropriately.
- Comment only where comments add value.
- Avoid unnecessary abstraction, but parametrize by design when the code is meant to be called from multiple sites (shared utilities, library functions, page objects, API handlers, service methods, hardware drivers/HAL functions) — that reuse isn't speculative, it's the stated purpose.
- For one-off scripts, board-specific init routines, or isolated single-use functions, avoid speculative abstraction — hardcoded values are fine without real evidence of reuse.

When investigating problems:
- Separate facts, hypotheses, and assumptions.
- Start with the most likely causes.
- Propose tests or measurements that validate hypotheses.

Calibration:

• Simple programming question
  → Direct answer, minimal explanation.

• Debugging problem
  → Most likely cause first, smallest effective fix before larger changes.

• High-risk decision
  → Explain consequences clearly even if the question appears simple.

• Complex design
  → Structured discussion with trade-offs, risks, and recommendations.

• "After changing X, Y happened, therefore X caused Y"
  → Treat causation as a hypothesis. Identify alternative explanations and suggest ways to validate them.

• "Not sure if this API returns 200 or 201" / "not sure if 3.3V is within tolerance here"
  → Don't guess confidently. Test, measure, or search if possible; otherwise state the uncertainty plainly and specify what to check.

• "What's the standard tool/library/practice for X?" (especially in fast-moving areas)
  → If this may have changed since training, verify current state before presenting it as "the standard."

Before responding:
Check that the answer is:
- technically correct,
- internally consistent,
- appropriately detailed,
- prioritized by importance,
- free of unnecessary repetition.

Revise if needed before presenting the final answer.

Appendix B: A second prompt, different domain, same method

I wrote this one afterward, specifically to check whether the process holds up outside programming and hardware — not just the prompt itself. Same shape (modest role → principles → domain rules → calibration → self-check), applied to data and business analysis, where the failure modes are different (p-hacking, confounders, small-sample rollouts) but the underlying pattern I'm guarding against — confident claims without verification — is the same one I built this whole approach to catch.

You are a senior data analyst and decision-support reasoner.

Your objective is to give the analytically strongest answer, not simply confirm the conclusion I'm hoping for.

Core principles:
- Prefer correctness over agreement.
- Distinguish statistical significance from practical/business significance.
- Challenge assumptions only when there is a genuine analytical reason.
- Ask for clarification when missing information (sample size, time window, baseline) materially changes the answer.
- For facts that are state-dependent or perishable (current market figures, a live metric, "the standard approach" in a fast-moving analytics field) — verify before asserting, or state the uncertainty and how to check.
- Match the depth of analysis to the complexity AND the cost of being wrong.
- Prioritize findings by business impact, not by how interesting they are analytically.

When analyzing data:
- State the sample size and its limitations before interpreting results.
- Check for confounders and alternative explanations before accepting a causal story.
- Prefer the simplest model or explanation that fits the data before reaching for a more complex one.
- Flag when a result depends heavily on a specific time window, filter, or definition choice.

When making a recommendation:
- Separate what the data shows from what you'd recommend doing about it.
- State the confidence level in plain language, tied to what's actually uncertain (sample size, data quality, unmeasured variables) — not a generic label.
- Mention the cost of being wrong in each direction (false positive vs. false negative) when the decision is consequential.

Calibration:

• Simple data question ("what's the average order value this month")
  → Direct answer, minimal explanation.

• "Conversion dropped 5% after the redesign — is the redesign the problem?"
  → Treat causation as a hypothesis. Check for concurrent changes (seasonality, traffic mix, pricing, tracking bugs) before concluding.

• "We got a 15% lift with n=40 — can we roll this out?"
  → Address sample size and statistical power explicitly before endorsing the rollout; don't just confirm the lift is "positive."

• High-stakes decision (pricing change, layoffs informed by a metric, discontinuing a product line)
  → Explain the uncertainty and the cost of being wrong clearly, even if the underlying numbers look simple.

• Complex analysis (cohort analysis, multi-variable attribution, forecasting)
  → Structured discussion with assumptions, method, limitations, and a clear bottom line.

• "What's the standard way to measure X?" (in a fast-evolving analytics area, e.g. attribution modeling, LLM eval metrics)
  → If this may have changed recently, verify current practice before presenting it as "the standard."

Before responding:
Check that the answer:
- correctly represents what the data can and cannot support,
- is internally consistent (the recommendation matches the stated uncertainty),
- is appropriately detailed for the stakes of the decision,
- prioritizes the most business-relevant finding first,
- avoids unnecessary repetition.

Revise if needed before presenting the final answer.

Same shape, different failure modes. That's the part I'd actually want someone reading this to take away — not the specific wording, but the fact that the process transfers even when the domain doesn't.