+------------------------------------------------------------------------------+ | | | Dhananjaya D R @/logs @/software @/resume @/contact | | | +------------------------------------------------------------------------------+ With AI, prompting isn't enough. Context engineering is the real edge. ________________________________________________________________________________ The conversation with LLMs is shifting from "prompt engineering" to broader "Context Engineering" — "the art of providing all the context for the task to be plausibly solvable by the LLM." With the rise of Agents, it becomes more important what information we load into the "limited working memory." We are seeing that the main thing that determines whether an Agent succeeds or fails is the quality of the context you give it. Most agent failures are not model failures anymore; they are context failures. To understand context engineering, we must first expand our definition of "context." It isn't just the single prompt you send to an LLM. Think of it as everything the model sees before it generates a response. Types of Context ________________________________________________________________________________ 1. Instructions / System Prompt: An initial set of instructions that define the behavior of the model during a conversation. Example: "You are a Python programming tutor. Always explain code step-by-step. Include error handling in examples. Prefer clean, readable code over complex one-liners. When showing functions, always include docstrings." 2. User Prompt: Immediate task or question from the user. Example: "How do I read a CSV file in Python and find the average of a column?" 3. State / History (short-term Memory): The current conversation, including user and model responses that have led to this moment. Example: You: "I'm working with a pandas DataFrame" AI: shows pandas solution You: "What about sorting this data?" AI: remembers you're using pandas and shows .sort_values() method 4. Long-Term Memory: Persistent knowledge base, gathered across many prior conversations, containing learned user preferences, summaries of past projects, or facts it has been told to remember for future use. Example: The AI remembers you prefer JavaScript over Python, you're building a React app, you like TypeScript, and you always want error handling included — even in new conversations weeks later. 5. Retrieved Information (RAG): External, up-to-date knowledge, relevant information from documents, databases, or APIs to answer specific questions. Example: You ask "What's new in Python 3.12?" — the AI searches current Python documentation and release notes to give you the latest features, rather than relying on old knowledge. 6. Available Tools: Definitions of all the functions or built-in tools it can call (e.g., check_inventory, send_email). Example: run_code() — to execute your Python script and show output check_syntax() — to validate your code for errors format_code() — to auto-format your code properly search_github() — to find relevant code examples create_file() — to generate and save code files 7. Structured Output: Definitions of the format of the model's response, e.g., a JSON object. +------------------------------------------------------------------------------+ | JSON Example | +------------------------------------------------------------------------------+ | 1 { | | 2 "solution": { | | 3 "code": "import pandas as pd\ndf = pd.read_csv('data.csv')", | | 4 "explanation": "This reads CSV and calculates column average", | | 5 "dependencies": ["pandas"], | | 6 "error_handling": "Add try/except for file not found", | | 7 "next_steps": ["Check for null values", "Validate data types"] | | 8 } | | 9 } | +------------------------------------------------------------------------------+ Real coding scenario: You're building a web scraper. The AI uses its instructions (be helpful with Python), your prompt (scrape product prices), conversation history (remembers you're using BeautifulSoup), your preferences (you like clean code), current documentation (latest BeautifulSoup syntax), tools (can test the code), and gives you organized output with code, explanations, and error handling. Managing Your Context is the Key to Successful Responses ________________________________________________________________________________ We all have a habit of shoving everything into the prompt and asking for a solution. But in reality, longer contexts do not generate better responses. Overloading your context can cause your agents and applications to fail in surprising ways. Contexts can become poisoned, distracting, confusing, or conflicting. Common Context Problems ________________________________________________________________________________ 1. Context Poisoning: When a hallucination makes it into the context. Example: AI: "The `splice()` method in JavaScript removes elements and returns the modified array" You: "Show me an example using splice" AI: builds example assuming splice() returns modified array (wrong — it actually returns removed elements) The AI "poisoned" the conversation with wrong info, then doubled down on the mistake. 2. Context Distraction: When the context overwhelms the training. Example: You paste 500 lines of legacy Java code with weird patterns, then ask: "How do I create a simple Hello World in Java?" The AI might mimic the messy style from your pasted code because it's overwhelmed by the context. 3. Context Confusion: When superfluous context influences the response. Example: You: "I'm debugging my React app. My cat just knocked over my coffee. How do I fix this useState hook error?" AI: somehow incorporates the coffee spill into its debugging advice or gets distracted by the cat mention The cat and coffee are irrelevant to the coding problem but confuse the AI's response. 4. Context Clash: When parts of the context disagree. Example: Earlier: "I'm using Python 2.7 for this legacy project" Later: "Show me how to use f-strings" AI gets confused because f-strings don't exist in Python 2.7, but both pieces of context seem important Or: You: "I prefer functional programming" Also you: "Show me the best OOP design patterns" AI struggles between your stated preference and current request Mitigating and Avoiding Context Failures ________________________________________________________________________________ 1. RAG: Selectively adding relevant information Example: You ask: "How do I handle API errors in my React app?" Good RAG: Fetches React error handling docs, HTTP status codes, try-catch patterns Bad RAG: Fetches entire React documentation, Node.js guides, database error handling The AI gets just the relevant pieces, not everything about React. 2. Context Quarantine: Isolating contexts in dedicated threads Example: Thread 1: Working on Python data analysis project Thread 2: Building React frontend Thread 3: DevOps deployment questions Instead of mixing all three in one conversation where the AI might suggest pandas solutions for your React problems. 3. Context Pruning: Removing irrelevant information +------------------------------------------------------------------------------+ | Before Pruning | +------------------------------------------------------------------------------+ | You: "I'm building a web app. My dog is barking. How do I center a div?" | | AI: [gives CSS solution] | | You: "That worked! Now I need to add authentication. It's raining." | +------------------------------------------------------------------------------+ +------------------------------------------------------------------------------+ | After Pruning | +------------------------------------------------------------------------------+ | You: "How do I center a div?" | | AI: [gives CSS solution] | | You: "Now I need to add authentication." | +------------------------------------------------------------------------------+ The weather and pets are removed since they don't help with coding. 4. Context Summarization: Condensing accumulated context Example: Long conversation about building a REST API becomes: +------------------------------------------------------------------------------+ | Context Summary | +------------------------------------------------------------------------------+ | Summary: User is building a Node.js REST API with Express, | | MongoDB database, JWT authentication, and needs error handling. | | Prefers async/await over promises. Working in TypeScript. | +------------------------------------------------------------------------------+ This preserves important context without the full conversation history. 5. Context Offloading: Storing information outside the conversation What it is: Moving data to external storage that tools can access when needed. Coding Example: Instead of keeping your entire codebase in the conversation: Store code files in a tool: save_to_workspace("app.js", code) Reference when needed: get_from_workspace("app.js") AI can retrieve specific files only when relevant This keeps the AI focused and prevents context problems from derailing your coding session. _.. .' `", ; \ .---._; ^, ; .-' ;{ : .-. ._; .--"" \*' o/ o/ / , / : _`"; ; \; `. `"+' | } / _.'T"--"\ : / .'.--""-,_ \ ; \ / /_ `,\ ; : / / `-.,_ \`. : |; { .' `- ; `, \ : \ `; { `-,__..-' \ `}+=, : \ ; `. `, `-,\" ! |\ `; \}?\|} .-' | \ ; .'}/ i.' \ `, ``''-' / \ /J|/{/ `' "My precioussss context... we must engineer it carefully, yesss..." ________________________________________________________________________________