From Prompts to Context: Why Clear Instructions Alone Are Not Enough

A Frustrating Scenario

Imagine this situation: you’re writing a perfect prompt about “the latest best practices for Python MySQL connections.” You carefully design a persona (“You are a Python database expert with 10 years of experience”), clear instructions (“Only provide 2024 best practices, no deprecated methods”), and specific format requirements (“List main approaches, pros/cons, code examples, security considerations”).

This prompt is flawless. Yet, GPT confidently gives you code using 2018-deprecated methods with security vulnerabilities. Why?

Your prompt is perfect, but the knowledge is wrong. The model doesn’t know 2024’s best practices because its training data cutoff doesn’t include this new information.

Where’s the Problem?

Traditional Prompt Engineering only controls “how you say it” but NOT “what the model knows.” The model cannot access:

1. Documents published after its training cutoff
2. Your private company documents
3. Real-time data
4. Specialized domain knowledge it wasn’t trained on

No matter how brilliant your prompt, you can’t fix a knowledge gap. It’s like showing a doctor who never studied modern medicine the latest medical journals—he might not understand them, or explain things using outdated knowledge.

The Emergence of a Solution: RAG

In 2020, Lewis et al. proposed Retrieval-Augmented Generation (RAG) at NeurIPS. The idea is simple: instead of relying solely on the model’s internal knowledge, retrieve relevant documents from an external knowledge base and inject them into the model’s context window.

The model now “reads” the latest documents before answering.

For example, when a user asks “latest best practices for Python MySQL connections,” the system:

1. Retrieves relevant content from latest technical blogs, official docs, Stack Overflow
2. Adds this content to the context
3. Lets the model answer based on this up-to-date information

Now the model has knowledge it wasn’t trained on.

From RAG to Context Engineering

RAG was just the beginning. In September 2025, Anthropic formally proposed “Context Engineering” on their engineering blog. Andrej Karpathy endorsed it on June 25, 2025: “Context engineering is the delicate art and science of filling the context window with just the right information for the model to take the next step.”

This is broader than just RAG—it includes:

• Knowledge layer (system prompts, tool definitions)
• Memory layer (conversation history, long-term memory)
• Retrieval layer (RAG, vector search, knowledge graphs)
• Generation layer (output constraints, CoT guidance)

Context Engineering is a complete discipline, not just document retrieval, but designing the entire information environment.

The Core Paradigm Shift

The fundamental shift: from “optimizing what you say” to “optimizing what the model knows.”

• Prompt Engineering = optimizing the instruction (static)
• Context Engineering = optimizing the information environment (dynamic)

This is like shifting from optimizing recipes (how to cook) to optimizing the entire kitchen environment (ingredients, tools, processes).

Preview

Next post: we’ll dive deep into Context Engineering’s four pillars, RAG implementation, and how to build your first knowledge-base Q&A bot.

This is part 3 of the “AI Engineering Paradigms” series. The series structure:

1. The Four Stages of AI Engineering Paradigms
2. Practical Recommendations for Different Stages
3. From Prompts to Context: Why Clear Instructions Alone Are Not Enough ← Current Article
4. The Four Pillars of Context Engineering
5. The Essence and Pitfalls of Harness Engineering
6. Loop Engineering: Letting AI Evolve Itself