Contextual AI

AI jargon cycles faster than AI itself.

A few months back we heard "vibe coding", now contextual AI is having its moment.

Early AI adopters leaned heavily on prompt engineering—tweaking inputs like formulas to coax the right output from language models.

But as enterprise adoption matures, the smartest organizations are discovering a more durable approach: context engineering.

🎙️ AI Confidential Podcast - Confidential Computing Summit 2025

☕️ AI Tangle - OpenAI's General-Purpose "ChatGPT Agent" Enters The Fray

🔮 AI Lesson - Five Prompting Methods Every AI User Should Know

🎯 The AI Marketing Advantage - How AI Is Changing How Brands Approach Creativity

💡 AI CIO - There Is No Moat

 📚 AIOS - This is an evolving project. I started with a 14-day free Al email course to get smart on Al. But the next evolution will be a ChatGPT Super-user Course and a course on How to Build Al Agents.

The right hires make the difference.

Scale your AI capabilities with vetted engineers, product managers, and builders—delivered with enterprise rigor.

Why Context Engineering Beats Prompt Engineering for Real AI Work

Your data team built an AI assistant to auto-summarize client calls. It works great in demos. But in production, results are hit or miss—because one prompt doesn’t fit every rep, client, or context. This is where most enterprise AI initiatives stall—not on model performance, but on missing context. Real success demands a shift from clever prompting to structured context engineering.

Prompt engineering is about crafting better inputs. Context engineering is about designing the system around the model so it performs consistently at scale. This means giving it context about the tasks you want it to accomplish.

As Andrej Karpathy notes in this tweet, “Every industrial-strength LLM app is really about filling the context window with just the right information.”

Karpathy, one of the most influential figures in the AI space, previously led Tesla’s AI efforts and served as a founding member of OpenAI. He set the tone for modern prompt-centric programming with the release of Vibe Coding, a landmark talk that reframed how developers think about programming with LLMs.

While large language models are filled with general information, they often lack the specific context needed to solve the problem you're working on. Without this situational grounding—details about the task, audience, prior steps, constraints, or business logic—the AI may produce outputs that are technically fluent but practically irrelevant.

Every LLM operates with a limited context window—the fixed number of tokens (words, characters, metadata) it can process at once. GPT-4o, for instance, supports up to 128K tokens. Everything the model knows about your request—prompt, examples, retrieved docs, instructions—must fit into that window.

Fill it poorly, and results degrade. Fill it well, and you get robust, on-target outputs.

To translate words to tokens, a rough rule of thumb is that 1 token ≈ 0.75 words. For example, a 1,000-word document typically uses around 1,300–1,400 tokens. Tools like OpenAI’s tokenizer help estimate and manage token counts in production.

Most people think “prompt engineering” means writing clever one-liners for ChatGPT. In business that’s table stakes. The real challenge—and opportunity—is context engineering: the precision craft of feeding just the right information into the model’s context window for optimal output.

Karpathy put it clearly: "People associate prompts with short task descriptions... But every serious LLM app is really about context engineering."

Enterprise AI isn’t about one prompt, one answer. It’s about systems that deliver reliable, role-specific performance across workflows. That means providing input—and building a software layer across the organization—that can:

This is not a wrapper. It’s a coordinated orchestration layer, and context engineering is its core.

It’s not just formatting data. It’s answering:

“What exactly does this LLM need to know, and in what structure, to do this job well—every time?”

When you hand work to a large language model, the brief matters as much as the data. Here’s how to make an LLM output sharp, on-brand, and cost-effective.

Tell the model who it is and why it's here—"You are a benefits analyst; draft a summary for HR." Anchoring the role nudges the model toward the domain knowledge and tone your readers expect. Technically, the role won’t give you a more accurate response, but it will guide tone and structure to match user expectations.

Connect the model to a retrieval layer so it cites your company wiki instead of guessing. RAG solves a fundamental problem: large language models are trained on massive datasets, but they can't access your specific company documents, recent updates, or proprietary information in real-time. They're like brilliant analysts who've read everything ever published—except your actual files.

Here's how it works: Your documents get chunked and stored in a searchable format. When you ask a question, the system first searches this knowledge base for the most relevant passages—like a smart search engine finding the best excerpts. Then the LLM receives both your original question AND the retrieved passages, crafting an answer that combines its general knowledge with your specific information.

Without RAG:

With RAG:

Result: fewer hallucinations, tighter sourcing, no manual copy-paste.

Show, don't tell. A couple of well-chosen Q&A pairs steer the model better than a page of instructions. Drop them right after the system prompt so the AI can imitate your structure and detail level.

Models now read tables, screenshots, even product photos. Feed the budget spreadsheet alongside the plain-language brief—the AI cross-references both to produce a narrative that matches the numbers.

Keep the shopping cart (current state) separate from long-term preferences (memory) and tag every conversation with a thread ID. Users can return days later and pick up exactly where they left off.

LLMs read only so many tokens at once. Summaries, deduplication, and similarity scoring trim your prompt to the essentials before it hits the model. You pay for fewer tokens and reduce distraction.

The sweet spot is balance. Starve the model of context and it hallucinates. Flood it with everything and costs climb while precision falls. These six layers give you that balance—nothing less, nothing more.

I built the Artificially Intelligent Operating System (AIOS) to ensure that anyone—regardless of background—can gain the skills to capitalize in the age of AI. Whether you’re a business leader, operator, builder, or career professional, this platform is designed to help you apply AI, not just understand it.

You don’t need a PhD in machine learning—you need a practical framework for integrating AI into the work you already do.

Get started by enrolling now and you’ll be first in line for a new series of AIOS courses launching later this year.

You may be an individual contributor, and this framework works for you as well. But this is really how I’d look at designing your “Enterprise GPT”, the systems that are used throughout your organization.

Break business processes into atomic, repeatable units (e.g., “Summarize client call,” “Generate compliance checklist”).

For each task, define:

For enterprise systems use middleware to fetch, filter, and format context dynamically:

Decide:

If you’re still training teams to only write better prompts, you’re solving the wrong problem. Train your systems to assemble better context. That’s how AI moves from gimmick to infrastructure.

Memory in ChatGPT helps index past interactions, giving the LLM contextual awareness.

The difference between a mediocre AI response and an exceptional one often comes down to one thing: context. While many users focus on crafting the perfect question, the real magic happens when you provide the rich background information that helps AI understand not just what you want, but the world in which that request exists.

This week, we're exploring how to layer context into your prompts to get dramatically better results. The examples below show five different scenarios where strategic context transforms a basic request into a powerful, targeted prompt that delivers exactly what you need.

Think of context as the difference between asking a stranger for directions versus asking a local friend. The stranger might give you a route, but the friend knows about the construction on Main Street, the faster shortcut through the park, and that parking is impossible downtown on Fridays. Context is what transforms AI from a stranger into that knowledgeable friend.

In these examples, I've explicitly labeled contextual information to demonstrate the technique, but remember: you don't need to announce "Context:" for this to work. The key is naturally weaving in the background details, constraints, audience information, and environmental factors that shape your ideal response.

Here’s an example for ChatGPT or Claude, they likely understand how to write a blog post but they don’t know all the details that you want to include, and the point of the blog post. By sharing them you can get a much better output with fewer rewrites and edits.

You might have sales numbers that are pretty standard but your company has a certain format, and certain concerns (e.g. maximizing sales in a certain geography, or profits versus growth). Giving details about how you want to analyze the data will get a better result.

You may have a product launch and you enter that in ChatGPT but without the right details you may not get a response that meets your criteria. You often will have more up-to-date or propriety information available then will be available to the model. In this example you can see examples, of things that are important to you and your organization. The market research in this example comes from the web but you may want to add details you know from analyst reports or other sources that aren’t generally available.

I appreciate your support.