LLM Parameters

You can write an amazing prompt… and still get a weird response.

Sometimes it’s not your prompt. It’s the parameters.

LLM parameters are the settings that control how a model responds: whether it plays it safe or gets creative, whether it rambles or stays tight, and how much it’s allowed to “think out loud” (or at least produce longer outputs).

If prompts are your steering wheel, parameters are the engine tuning.

The Big 5 Parameters You’ll Use Most

1) Temperature (creativity vs consistency)

Temperature controls randomness. Lower temperature = more predictable. Higher temperature = more varied.

• Use low (0–0.3) for: compliance language, specs, code fixes, factual summaries
• Use medium (0.4–0.7) for: brainstorming, marketing variants, soft creativity
• Use high (0.8–1.2+) for: wild ideation, story tone experiments (and occasional nonsense)

2) Top-p (nucleus sampling)

top_p is another way to manage randomness. Instead of “turning up creativity,” you limit the model to the most likely tokens whose probabilities add up to p.

In practice:

• If you’re using top_p, keep temperature moderate
• Many teams choose either temperature or top_p as the main “creativity dial” to simplify tuning

3) Max tokens (how long it can talk)

max_tokens is the model’s word budget.

Set it too low and outputs get cut off. Set it too high and you might get a novel when you asked for a paragraph.

Pro move: match max_tokens to the deliverable.

• Email: maybe 150–300 tokens
• PRD sections: 600–1200 tokens
• Code patch + tests: depends, but budget for the diff and explanations

4) Presence & frequency penalties (stop repetition)

These penalties reduce repetitive output:

• Frequency penalty: discourages repeating the same phrases
• Presence penalty: encourages introducing new topics instead of looping

If your model keeps rephrasing the same sentence 6 ways, a small penalty can clean it up.

5) Stop sequences (clean endings)

Stop sequences tell the model where to stop generating.

Examples:

• Stop after “### END”
• Stop at the beginning of an unwanted section
• Prevent the model from continuing into a “bonus” you didn’t ask for

Example 1: Sales Email (Make It On-Brand, Not Random)

Let’s say a sales rep wants a tight outreach email. You want consistency across reps, so you keep creativity controlled.

Write a 90-word outbound email to a VP of Marketing.
Goal: invite them to a 4-week AI upskilling program for their team.
Tone: friendly, confident, not cheesy.
Include: one clear CTA and a subject line.

Suggested parameters:

• temperature: 0.3 (consistent tone)
• max_tokens: 250 (prevents rambling)
• stop: "\n\n" after the email (optional)

Example 2: Engineering Debug (Accuracy > Creativity)

Now an engineer is asking the model for a patch and tests. You want the model to be boring—in a good way.

You are a senior backend engineer.
Given the stack trace and code below:
1) Identify the root cause
2) Provide a minimal patch
3) Add two pytest tests that reproduce the bug and verify the fix
Output only: (a) bullet root cause, (b) code blocks.

Suggested parameters:

• temperature: 0.0–0.2 (reduces hallucinated fixes)
• top_p: 1.0 (keep it simple)
• max_tokens: 1200+ (enough room for tests)

How to Choose Settings Without Overthinking

Here’s a quick mental model:

• Need repeatable, correct output? Lower randomness.
• Need more options? Increase randomness a bit.
• Getting cut off? Increase max_tokens.
• Seeing loops? Add small penalties.
• Want clean formatting? Use stop sequences.

And yes—different roles will tune differently. Legal wants low temperature. Marketing wants a touch higher. Engineers want deterministic patches. Support teams want consistent empathy.

Takeaway

LLM parameters are the control knobs that shape your AI’s behavior. Prompts tell the model what to do; parameters influence how reliably and creatively it does it.

If you’re building serious workflows (or you just want fewer “why did it say that?” moments), learn these dials: temperature, top_p, max_tokens, penalties, and stop sequences. They’ll make your prompts feel 10x more dependable.