Context Engineering: The #1 Skill for Building AI Agents

Ever since Andrej Karpathy mentioned “context engineering” on Twitter, the term has taken off. The excitement grew even more when Cognition called it the number-one skill for engineers building AI agents. But hype isn’t clarity, so the goal here is simple: explain what context engineering actually is in an intuitive way that even a high school student can understand.

Before that, let’s revisit something familiar: prompt engineering.

What Prompt Engineering Really Means

Imagine you want to summarize an article. If you type:

“Summarize the following article.”

…you’ll get a generic summary. A more intentional prompt might say:

·       Summarize in three bullet points.

·       Use simple language suited for high school students.

·       Highlight the key arguments.

You define structure, tone, and audience, and the model follows.

 This is classic prompt engineering: carefully crafting a single instruction to control one model output.

Why Agents Change the Game

Move from chatbots to agents and everything shifts. Agents don’t just answer; they act.

Example:

 “Book a flight between city A and city B.”

An agent with access to a tool like the Expedia API will:

1.       Search for flights

2.       Compare results

3.       Pick the cheapest one

4.       Book the ticket

An agent is an LLM acting as a “brain,” supported by tools, memory, and external knowledge.

Crucially, agents operate in turns:

·       Turn 1: Make a plan

·       Turn 2: Call a tool

·       Turn 3: Analyze results

·       Turn 4+: Continue until finished

Every turn is a new LLM call, and each call carries context.

 These intermediate prompts aren’t written by you, they’re generated by the LLM itself.

This is where real challenges begin.

The Pitfalls of Growing Context

With every turn, more text enters the context window. Over time:

·       Costs increase

·       One weak intermediate prompt can break the workflow

·       You risk hitting the model’s context limit

·       Even before the limit, long contexts create issues:

o   Context poisoning

o   Context distraction

o   Context confusion

o   Context clash

The principle is simple:

 More context isn’t better. Better context is better.

Why Context Engineering Matters

Karpathy’s core point is that building robust LLM applications, especially agents, requires more than writing a clever prompt.

People think of prompts as short descriptions typed manually. But in agents, prompts evolve dynamically. You need a system that automatically fills the context window with the right information at the right time, based on:

·       Tool outputs

·       Retrieved knowledge

·       Persistent state

·       Instructions

Cognition frames this as the natural evolution of prompt engineering. Instead of designing one prompt, you design an entire workflow where many prompts are produced and consumed across turns.

A useful analogy:

·       The LLM is the CPU

·       The context window is the RAM

If the RAM is cluttered or irrelevant, the CPU makes poor decisions.

 Context engineering ensures the RAM contains only what matters.

The context comes from three sources:

1.       Tools

2.       Knowledge (databases, PDFs, RAG, etc.)

3.       Instructions

Four Core Strategies for Context Engineering

1. Context Compression

When the context becomes too large, you compress it:

·       Remove unneeded details

·       Keep only key decisions and signals

·       Ensure the agent still understands prior steps

Compression preserves meaning while staying within token limits.

2. Writing Context Outside the LLM

Not everything needs to live inside the context window. You can store information externally:

·       A scratchpad saved via a tool call

·       A state object in systems like LangGraph

·       Long-term memory (like ChatGPT remembering you across sessions)

External memory reduces pressure on the LLM’s limited tokens.

3. Selecting the Right Context

Storing isn’t enough, you also need to retrieve selectively:

·       Pull relevant history

·       Fetch knowledge via RAG

·       Load key scratchpad content

·       Pass along only the tool outputs that matter

The model should see only what is relevant for the current step.

4. Isolating Context

Large tasks can be split across multiple agents:

·       Agent 1 handles A

·       Agent 2 handles B

Each agent receives a smaller, more relevant context window.

This improves performance and reduces cost, though some tasks cannot be parallelized if one depends on another.

The Art and Science of Context Engineering

Karpathy describes context engineering as both an art and a science.

It’s a science because it relies on:

·       Clear task descriptions

·       Clean few-shot examples

·       Well-structured tool outputs

·       Disciplined state and history management

·       Token budgeting

It’s an art because LLMs are sensitive to noise, repetition, ordering, and phrasing.

Too little context causes failure.

 Too much context also causes failure.

Finding balance is the essence of the skill.

Conclusion

Context engineering is the discipline of managing what enters the LLM’s context window, so the model performs optimally at every step. It’s the natural evolution of prompt engineering, especially for building autonomous agents.

If you learn to select, compress, isolate, and externalize context intelligently, you can build reliable, efficient, and scalable AI systems.

Key Takeaways

·       Agents generate many intermediate prompts—managing their context is crucial.

·       Good context engineering balances relevance, length, and clarity.

·       Context is a curated mix of tool outputs, retrieved knowledge, and instructions—not just raw history.