Mastering LLM Prompts: Lessons Learnt on the Job!!

Introduction

Large Language Models (LLMs) like OpenAI’s GPT-4 have revolutionized AI-driven applications. However, getting the best out of them requires careful prompt engineering. Through trial and error, we’ve identified several best practices that can drastically improve the reliability and accuracy of LLM responses.

Whether you’re building chatbots, automation tools, or AI-powered data analysis, these insights will help you optimize LLM performance, reduce errors, and enhance output quality.

We won’t spend much time with the following extremely important point as it is the first point mentioned in almost every LLM mastering tips!

1. Context is King: Define a Persona for Better Responses

2. Highlight Important Instructions Clearly (IMPORTANT)

Even with structured prompts, LLMs sometimes overlook key instructions. Use bold formatting (**important**) or triple asterisks (***) to emphasize critical points.

✅ Example: “Summarize the following article. ***The summary must be exactly 3 bullet points. Do NOT exceed 50 words total.***”

Why this works: LLMs tend to follow emphasized instructions better than plain text instructions.

3.Reinforce Important Instructions More Than Once

Even if you highlight critical points, the model might still ignore them. Reinforce the instruction in different parts of the prompt to ensure compliance.

✅ Example:”Summarize the following article in exactly 3 bullet points. Each point must not exceed 50 words.”

[Insert article text]

Reminder: ***Ensure that the summary contains exactly 3 bullet points and does not exceed 50 words.***

Why this works: Repetition helps prevent LLMs from missing key requirements.

4.Use JSON Schema for Structured Outputs

If you’re expecting structured responses (e.g., JSON output), define a JSON schema in the system message to guide the model.

✅ Example:

{
  "json_schema": {
    "name": "LLMPromptBestPractices",
    "description": "Schema defining key best practices for crafting effective LLM prompts.",
    "schema": {
      "type": "object",
      "properties": {
        "title": {
          "type": "string",
          "description": "The title of the blog post."
        },
        "summary": {
          "type": "string",
          "description": "A brief summary of the blog content."
        }
      },
      "required": ["title", "summary"]
    }
  }

Why this works: JSON schemas provide a clear structure for LLMs, reducing formatting errors.

5. Validate JSON Output to Avoid Empty Fields

Even with a well-defined schema, LLMs sometimes return incomplete or empty fields. Always validate the response before using it.

if not response.get(“title”):

print(“Error: Missing title in JSON output.”)

6. Validate ID Outputs to Prevent Incorrect Values

When instructing an LLM to return an ID from a predefined list of inputs based on certain instructions, there’s a risk that it could generate incorrect or mismatched IDs. In such cases, validation is crucial to ensure the ID is correct and aligns with the expected data.

✅ Example: If the LLM is instructed to return an order number based on specific input data (such as customer ID, product type, etc.), always validate the returned ID by checking if it exists in the database or within the predefined list before taking any action.

7. Understanding LLM Token Limits: Context Length vs. Output Tokens

LLMs have two types of token limits:

1. Context Length (max input + output tokens) → Exceeding this leads to an error.
2. Output Tokens (max tokens per response) → Exceeding this results in partial responses.

✅ Solution:

• Increase max_tokens if responses get cut off.
• Implement logic to detect and retry when output is incomplete.

8. Handling Formatting Issues in HTML Output

Even when explicitly asked to format responses in HTML, LLMs sometimes return incorrectly formatted text (e.g., ***bold text*** instead of <b>bold text</b>). To avoid such issues, it’s important to provide specific and detailed instructions about your formatting requirements.

✅ Example:

Format the response in **HTML** using proper tags:

– Bold: <b></b>

– Italics: <i></i>

9. Avoid Overloading the LLM with Too Many Tasks

If a prompt asks for too many different tasks, the quality of the response decreases.

✅ Solution: Break tasks into separate prompts.

🚫 Bad Prompt:

Summarize the article, extract key quotes, list references, and generate a title.

✅ Better Approach:

1️. First, generate a title.

2️. Next, summarize the article.

3️. Then, extract key quotes.

10.Write Steps in Proper Order

When providing instructions that involve multiple steps, list them in the correct sequence. If there are nested steps, use bullet points for clarity.

✅ Example:

Follow these steps:
1. Retrieve user data from the database.
2. Process the data and extract relevant details:
– Check for missing values.
– Normalize data fields.
3. Generate a summary report.

Why this works: A structured approach improves comprehension and execution accuracy.

11. Prompt LLMs to Validate Their Own Outputs

LLMs can self-check their output if prompted correctly.

✅ Example:

“Generate a JSON response and validate that all required fields are included. If a field is missing, return an error message instead.”

Why this works: Encourages the LLM to verify its own output before returning a response.

12.Provide Detailed Instructions if LLM Ignores Key Points

Sometimes, if instructions are too broad, the LLM may not follow them correctly. To improve compliance, provide more granular details.

✅ Example:

“Extract patient diagnosis and history from the report.”

✅ Better Approach:

“Extract the patient’s diagnosis from the ‘Diagnosis’ section and the medical history from the ‘Patient History’ section. Do not include any other information.”

13. For Deterministic Results, Lower the Temperature

LLMs use a temperature setting to control randomness. Lower values make the output more consistent.

✅ Example:

– `temperature = 0.0 – 0.2` → Best for factual tasks (e.g., financial report analysis)

– `temperature = 0.7 – 1.0` → Best for creative tasks (e.g., storytelling)

Conclusion: Turning Prompts into Power

Mastering LLM prompts is both an art and a science. By applying these best practices, you can maximize the effectiveness of LLMs, reduce errors, and ensure more reliable outputs. Whether you’re a developer, data scientist, or AI enthusiast, these techniques will help you unlock the full potential of AI-driven automation.

Happy prompting! 🚀