Three Different Answers to One Question: Why AI Agents Fail on Context, Not Intelligence

<p></p> <h2>Key Findings</h2> <ul> <li>An executive asks, “What was net revenue last quarter?” The dashboard gives one number, an analyst gives another, and an AI agent produces a third. The failure may not be the model’s reasoning. It may be that no shared, approved definition of <em>net revenue</em> was available to all three.</li> <li><strong>55% of surveyed executives</strong> cite insufficient business context as the primary cause of low-quality AI-agent insights, with fragmented or inaccessible data second at <strong>45%</strong>, according to the <a href="https://www.technologyreview.com/2026/11/02/1147660/raising-the-quality-of-insights-in-agentic-analytics/">MIT Technology Review Insights report</a> published November 2, 2026, in partnership with Snowflake.</li> <li>On average, only <strong>48% of insights</strong> generated by analytics agents lead to actions that improve the business, up from <strong>37%</strong> one year earlier. At a small cohort of “insight leaders,” more than <strong>70%</strong> of agent insights lead to business-improving actions.</li> <li><strong>52% of organizations</strong> receive more agent insights than they can properly evaluate, and only <strong>24%</strong> can evaluate all agent insights. Unvalidated output is the default state of agentic analytics, which makes governed definitions and inspectable calculations a control issue rather than a data-hygiene issue.</li> <li>On November 3, 2026, Snowflake announced <strong>Business Ontology</strong> and the generally available <strong>Cortex AI Gateway</strong> at its Expedition 2026 conference, addressing the two halves of the problem: what agents know (governed business definitions) and what agents may do (unified access, security, and spending controls).</li> </ul> <h2>What You’ll Learn</h2> <ul> <li>Why different systems can return different answers to the same business question, and what the MIT Technology Review Insights survey data says about how common that failure is.</li> <li>How to structure a three-part context stack for enterprise agents, and why a “definition census” is the cheapest first step.</li> <li>Which access and cost controls to establish before scaling agent count, based on Snowflake’s November 3, 2026 announcement.</li> <li>Why CrowdStrike’s same-day AI kill switch and SAP’s sovereign deployment of Cohere North complete the executive checklist for trusted agents.</li> </ul> <h2>Who Should Read This</h2> <p>CIOs, CDOs, CFOs, analytics leaders, and business owners accountable for trustworthy reporting, AI investment, or data access. If teams debate the meaning of a core metric, or cannot explain why an agent’s answer differs from the dashboard, the problem is operational, not merely technical.</p> <h2>The Context Gap Is a Business-Control Problem</h2> <p>An agent can query the correct tables and still produce a misleading answer if it does not know which definition the business approves. “Active customer,” “qualified lead,” or “net revenue” can vary by department, reporting period, or use case. If those differences are not governed, a model may choose a plausible interpretation without knowing it conflicts with executive reporting.</p> <p>The survey data points to an insight-quality challenge, not a simple lack of AI activity. On average, <strong>48% of agent-generated insights</strong> lead to actions that improve the business, a marked advance from <strong>37%</strong> one year ago, but still leaving more than half of agent output failing to help the business. The gap between descriptive and prescriptive quality matters most: respondents generally rate agents’ descriptive insights (what happened) well, while reviews of prescriptive insights (what to do next) are decidedly mixed. Recommendations require knowledge of business priorities, constraints, and decision authority, much of which is rarely encoded in a data table.</p> <p>The benchmark is instructive. A small cohort the report terms <strong>“insight leaders”</strong> achieves more than <strong>70%</strong> of agent insights leading to business-improving actions, against the <strong>48%</strong> average. The leaders’ edge is widest on prescriptive insights, and fewer of them report that their agents struggle with access to context. The differentiator is not a bigger model. It is governed context: <strong>60%</strong> of surveyed companies are prioritizing AI and data governance initiatives and <strong>45%</strong> are adding a semantic layer to their data architecture to improve agent insight quality.</p> <p>On November 3, 2026, at its Expedition 2026 conference, Snowflake announced <strong>Business Ontology</strong>, a new capability in Horizon Context within Horizon Catalog. It gives core business definitions an approved version and accountable owner, and links those definitions to underlying tables and views. Agents are served the approved definition rather than competing interpretations. Organizations can migrate terms from third-party catalogs through a guided CoCo session, and changes are logged so an auditor can establish what a metric meant in a past quarter and who approved it. <a href="https://www.snowflake.com/en/news/press-releases/expedition-2026-business-ontology-ai-gateway/">Snowflake’s announcement</a> describes the system as a way to ground agents in shared business meaning.</p> <p>The implication is broader than one vendor’s product: <strong>definitions need ownership, lineage, and change history</strong> if agents will use them to inform decisions. As dentsu’s data product leader noted in the announcement, once agents become a way people query data, definitions cannot remain only in a static data dictionary.</p> <h2>Build a Three-Part Context Stack</h2> <p>For executive use, separate context into three layers. Each answers a different question; none alone resolves the context gap.</p> <ol> <li><strong>What does the term officially mean? Governed ontology.</strong> Assign an owner and approve a canonical business definition.</li> <li><strong>How is the number calculated? Semantic layer.</strong> Record the calculation logic, filters, time period, and source data used to produce it.</li> <li><strong>What operational knowledge was never written down? Inferred operational knowledge.</strong> Capture relevant workflow signals and constraints, then keep human-approved definitions authoritative where they exist.</li> </ol> <p>This structure makes the answer testable. When two systems disagree, leaders can examine the definition, calculation, and underlying data instead of attributing the discrepancy to “AI being unpredictable.”</p> <p><img src="https://cdn.marblism.com/6j_h-oaZ-Lq.webp" alt="One approved business definition flowing to every AI agent through governed data" style="max-width: 100%; height: auto;"></p> <p>A low-cost first step is a <strong>definition census</strong>: list the <strong>10,20 metrics</strong> executives most often dispute, assign an accountable owner to each, and approve one official calculation. This is the unglamorous groundwork before scaling agents, and it maps to the first stage of Mahlum Innovations’ RAPID Framework, where goals and data readiness are established before any system is built. Agents deployed against inconsistent meanings scale the inconsistency; definitions approved before deployment scale the trust.</p> <h2>Govern the Agent’s Access and Spend</h2> <p>Context also includes what an agent is permitted to access and do. Snowflake’s November 3 announcement positions <strong>Cortex AI Gateway</strong>, generally available as of the announcement, as a single front door for agents. The company says its gateway applies existing human-user policies to agent calls through MCP servers, records agent activity in the customer’s Snowflake account, and supports spend attribution and hard limits. Several associated capabilities are identified as private preview, including Agent Abuse Detection, Agent Trace Intelligence, and Dynamic Model Routing; executives should read availability status carefully before committing production plans to any of them.</p> <p><img src="https://cdn.marblism.com/f4aCoWx8J7V.webp" alt="An AI agent governed by access policies, spending caps, and a human-approved stop control" style="max-width: 100%; height: auto;"></p> <p>Other announced controls include account-wide runtime guardrails against prompt injection and sandboxes that run agent code with the permissions of the person running it. Agent Abuse Detection is designed to analyze complete agent trajectories, not individual prompts, to identify potential overreach or use of agents to bypass approvals or exfiltrate data. The principle for executives is straightforward: <strong>agent calls should inherit controls already established for human users, not operate under a parallel permission system.</strong></p> <p>Cost governance should extend beyond models to individual agents. Kargo VP of Data and AI Andy Owens said the team previously could not identify which teams and agents were driving spend; he described the value of spend breakdowns and hard limits that prevent a runaway process from quietly consuming a budget, plus model routing that keeps routine work off the most expensive models. Establish per-agent attribution and budget caps before increasing the number of agents in production.</p> <p>Use this executive test: <strong>If two agents and a dashboard disagree on net revenue right now, can we say in one hour which answer is correct and why?</strong> If not, adding agents scales inconsistency along with capacity.</p> <h2>Complete the Checklist: Emergency Stop and Sovereign Deployment</h2> <p>Two further November 3 announcements fill out the executive checklist for trusted agents.</p> <p>On November 3, CrowdStrike announced an AI kill switch in Falcon Guardian that lets security teams stop internal agents that go rogue, block relaunch, and cut network access. The action requires a human decision to cut off and reinstate access; the mechanism does not delete the agent’s code but uses connected AI gateways to sever access to the models behind its decisions. <a href="https://www.cnbc.com/2026/11/03/crowdstrike-ai-kill-switch-rogue-agents.html">CNBC’s report</a> also notes that CrowdStrike announced a Claude Cowork integration and acquisitions of Fiddler AI, an AI observability startup, and Dreadnode, an AI security testing firm. An emergency stop is not a substitute for good context, but it gives authorized personnel a decisive response when an agent’s behavior or access is unacceptable.</p> <p>A separate November 3 <a href="https://news.sap.com/2026/11/sap-cohere-empower-regulated-industries-innovate-deploy-ai-faster/">SAP announcement</a> made Cohere’s agentic platform North generally available on SAP Business AI Platform. SAP positions the offering for organizations with sovereignty and data-residency requirements, including regulated industries and the public sector, with agents and workflow automations deployed within the customer’s own infrastructure. Deployment location can matter for compliance; it does not, by itself, settle what a business term means.</p> <p>Together, the three announcements cover the three questions every executive should be able to answer about a production agent: <strong>What does it know</strong> (governed definitions), <strong>what may it do</strong> (unified access and spending controls), and <strong>how do we stop it</strong> (a named human decision path). An agent program missing any one of the three is exposed, regardless of which vendor supplies the components.</p> <h2>The Executive Priority: Govern Meaning Before Scaling</h2> <p>MIT Technology Review Insights found that only <strong>24%</strong> of surveyed executives can evaluate all insights their agents generate, and <strong>52%</strong> of organizations receive more insights than they can properly evaluate. That gap makes shared definitions and traceable calculations more consequential: teams cannot effectively validate every result if they lack a common reference for what “correct” means. The math is unforgiving. When more than half of agent insights already fail to improve the business, each ungoverned definition multiplies the share of output that looks authoritative but is not.</p> <p>Start with business-critical definitions and accountable owners, then establish the calculation and data lineage behind each metric. Apply human access controls to agent activity, attribute spend to specific agents, and set hard limits before expanding usage. Confirm a named human decision path for stopping an agent that misbehaves. This is an operational foundation, not a model upgrade.</p> <p>The survey’s own roadmap supports this sequence. The initiatives surveyed companies prioritize are governance (<strong>60%</strong>) and semantic layers (<strong>45%</strong>), not larger models. “Insight leaders” that achieve more than <strong>70%</strong> business-improving agent actions are the proof: better context converts more agent output into decisions worth acting on, which is where the return on agentic analytics is actually earned.</p> <p>Mahlum Innovations’ <strong>3.5× average ROI is a company-stated average, not a guarantee</strong>; results depend on the use case, baseline, and implementation. For organizations defining that baseline, <a href="https://mahluminnovations.com/ai-strategy-consulting">AI strategy consulting</a> can help connect priorities to measurable outcomes. For teams ready to build, explore <a href="https://mahluminnovations.com/ai-implementation-services">AI implementation services</a>, <a href="https://mahluminnovations.com/ai-automation-consulting">AI automation consulting</a>, or the full <a href="https://mahluminnovations.com/services">services overview</a>.</p> <p>The most capable model cannot resolve a business disagreement that the organization has not resolved itself. The enterprises that scale reliable agents will define what their data means, govern who can act on it, and make the reason behind each answer inspectable.</p> <p><em>Sources: MIT Technology Review Insights, “Raising the quality of insights in agentic analytics,” November 2, 2026; Snowflake press release, November 3, 2026; CNBC, November 3, 2026; SAP News Center, November 3, 2026.</em></p>