A sequel to “Neuro-Symbolic AI: Why the Future Belongs to Hybrids.”
In the summer of 2020, we published a bet: the future of AI belonged to hybrids: systems that learn like neural networks and reason like logic engines. GPT-3 had been announced only weeks earlier. Most of the industry was about to spend the next several years betting the other way, i.e. on scale alone.
We were correct. And we were early. What we couldn’t do in 2020 was build the thing we were envisioning. The neural half wasn’t good enough yet, and nobody had worked out the plumbing between the two halves. Today, both problems are solved, and the plumbing has a name: ontology-based retrieval-augmented generation.
Here are a few words about why it’s all relevant to the two things our clients ask about first: accuracy and security.
What We Wanted in 2020
We were young. We were starry-eyed. The dream was supereasy to state and tantalizingly hard to build. A system that could understand a question the way a person actually asks it (that is: messy, ambiguous, half-formed) and answer it the way an human expert would: from verified knowledge, with reasons, and within the rules.
Symbolic systems could already do the second half. Expert systems and knowledge graphs had been answering precise questions precisely for decades, but only if you spoke their query language and had a deep understanding of their schema. Neural systems could do the first half, sort of badly—well, what in retrospect looks sort of badly. In 2020, a language model could write a convincing paragraph about your contracts, for example; what it could not do was tell you which of them actually existed. Hallucinations were rampant. What was missing was a good bridge: a trustworthy way for the neural side to understand people while the symbolic side held the source-of-truth, with a clean, inspectable handoff between.
Vector RAG Got Close (Close Not Exactly Being Good Enough for the Proverbial Cigar)
The first bridge the industry built was vector RAG. Chop your documents into chunks, turn each chunk into an embedding, and when a question arrives, retrieve the chunks that look most similar and hand them to the model. It was a bonafide leap because suddenly an LLM could talk about your data instead of just the internet’s.
But, as we are now reminded daily, similarity isn’t truth. Vector search retrieves text that resembles the question, not facts that answer it. Ask “Which of our vendor contracts renew in Q3 and carry an automatic price escalator?” and a vector store hands back passages that mention renewals and escalators from the wrong contracts, from superseded drafts, from a sales email that happened to use the same words. Back then, the model did what models do well: which is to compose a fluent, confident-sounding answer from whatever it happened to have been handed. Rather good for the CEO’s doing demos, pretty damn disappointing and dangerous for the CFO’s.
What an Ontology Adds
An ontology is a formal model of a domain: the kinds of things that exist in it, the relationships between them, and the rules they obey. In our contract example, a Contract has exactly one Counterparty, a Term with a start and an end date, and any number of Clauses. An EscalatorClause is a kind of Clause, with a rate and a trigger. Load your actual data into that model and you have a knowledge graph, a structure that knows which escalator belongs to which contract, because the relationship is stored, not guessed.
Ontology-based RAG changes the division of labor:
- The model interprets. The LLM reads the user’s question and translates it into a structured query against the ontology (which classes, which relationships, which constraints).
- The graph answers. The query runs deterministically. You ask the same question with the same data and you get the same result…every time. No sampling temperature, no creative interpretation, etc.
- The model explains. The LLM turns the exact result set into readable prose, citing the specific records it came from.
That is the 2020 hybrid to the freckle. Neural at the edges, where the people are; symbolic at the core, where the facts live.
An Old Idea, Finally in Its Element
If the vocabulary of ontologies, relationships, knowledge graphs sounds familiar, it’s because it harkens back to the early oughts’ Semantic Web and to innovations we were big believers in way before 2020. In May 2001, Tim Berners-Lee, James Hendler, and Ora Lassila described in Scientific American a web whose pages carried machine-readable meaning, so that software agents could reason across them. The W3C spent the following years building the machinery: RDF to express facts as subject–predicate–object triples, OWL (2004) to define ontologies, and SPARQL (2008) to query them.
In 2002 and 2003, few ideas excited us more. OWL was still a working draft, and the promise was intoxicating: software that didn’t just store data but understood what it meant and could show its reasoning (which ended up being one of the biggest problems with LLMs early on: they were total blackboxes and you never knew exactly why you were being given the answers you were). For a few heady years there, it looked like the next web, and we were almost positive we’d be building on it.
Then, as so often happens in the History of AI, the early enthusiasm drained away. Ontologies were slow and expensive to build, someone had to mark up the data by hand, and almost nobody (besides us) wanted to learn SPARQL just to ask a question. The industry chose JSON APIs and search boxes instead. But, instead of dying, the ideas went into hibernation, retreated to the places that could afford them: Stephan Wolfram had a thing; Google had its Knowledge Graph (2012, pitched as “things, not strings”); there was Wikidata; schema.org markup; and the life-sciences knowledge bases that quietly ran on RDF all along.
Which is why today’s moment feels so personal and so cool. Large language models remove precisely the two obstacles that stalled the Semantic Web. Authoring, meaning a model can draft an ontology and extract triples from a document set in hours, not months. And Access, meaning good people no longer have to write SPARQL, because the model writes the query from a plain-English question. The machinery we were so psyched about in 2002 turns out to be exactly the deterministic core that generative AI was missing. The neural half didn’t replace the Semantic Web. It completed it.
Accuracy: Let the Graph Answer, Let the Model Talk
Determinism is the whole point. When the answer comes from a graph query, you can inspect the query, rerun it, and get the same rows back. Accuracy stops depending on how well the model read a pile of text that day and starts depending on whether your data is right…a problem businesses and the engineers inside of them already know how to manage.
So, three accuracy properties fall out of the architecture almost for free:
- Provenance. Every fact in the answer traces to a node in the graph, and every node traces to a source document or system of record. “Where did that number come from?” always has an answer.
- Validation. The ontology’s rules become automated checks. If the model’s draft names a contract that isn’t in the result set, or asserts a relationship the ontology forbids, a validator catches it before the user ever sees it. (W3C’s SHACL standard exists for this.)
- Multi-hop reasoning. “Which customers are exposed if this supplier fails?” means walking from supplier to part to product to order to customer. Vector search can’t hop like that but a good graph query does it effortlessly.
Sure, the model can still misread a question or phrase its answer in a clumsy way, but its mistakes become visible. And checkable. Because the structured query sits in the middle as an audit point. This is the difference between AI you can demo and AI you can deploy.
Security: Put the Rules Where the Model Can’t Argue With Them
Security, where pure-LLM systems are most exposed, is where the deterministic core shows up to save the day.
In a typical vector RAG system, permissions are an afterthought: metadata tags on chunks, filtered (one hopes) before retrieval. Everything else rests on the prompt: “Only answer using documents this user is allowed to see.” That instruction is a suggestion, given to a system that can be talked out of suggestions. Prompt injection exists precisely because an LLM cannot reliably tell the difference between an instruction and a piece of text that looks like one.
An ontology lets you move the rules out of the prompt and right into the data model:
- Access control as structure. Sensitivity and ownership become properties in the ontology — a Document hasClassification Confidential; a Matter isRestrictedTo a team. Queries are rewritten to respect those properties before they execute. The model never receives data the user isn’t entitled to, so no prompt (whether clever or malicious) can coax it into revealing that data.
- A narrow, typed interface. The model doesn’t get free-text access to your corpus. It gets the ability to request specific classes and relationships through a query layer you control. A narrower interface is a smaller attack surface.
- Auditability. Every question produces a log-able, human-readable query. Security reviews stop being archaeological expeditions on model output and become just ordinary log review.
The principle is old and sound: never enforce a rule in the component most likely to be fooled. In an AI system, the LLM is the most gullible component in the stack. The ontology is the least.
Walking Through a Real Request
In the original post, we described an AI-assisted document system for a financial services firm. Here is how the ontology-based version handles an actual request.
A compliance analyst asks: “Show me every client account opened in the last 90 days that is still missing beneficial-ownership documentation.”
- The LLM maps the question onto the ontology: Accounts opened within 90 days, their Clients, and each Client’s required BeneficialOwnershipRecord where the status is missing.
- The query layer adds the analyst’s entitlements as mandatory filters. The model has no say in this step.
- The graph returns 14 accounts. Exactly 14. The same 14 you’re going to see if the analyst asks again tomorrow.
- The LLM drafts a summary. A validator confirms that every account it mentions is in the result set.
- The analyst sees the answer, the list, and a link to the query that produced it.
No step depends on the model being perfect. Every step that must be right is a deterministic one.
What It Costs and When It’s Worth It
Ontologies aren’t free. Someone has to model the domain, and that someone needs to understand the business, not just the technology. This is where Desert Willow’s first pillar — domain expertise up front — stops being a slogan and becomes a line item.
The good news: the modeling is substantially cheaper than it was in 2020. LLMs are excellent at drafting candidate ontologies from existing documents and database schemas. A human expert still has to review and ratify the result, but a first working model for a focused domain now takes days or weeks instead of quarters. You start with 20 to 50 core classes built around the questions that matter most, and you grow from there.
The Dream, Realized
In 2020 we wrote that neuro-symbolic hybrids would play a central role in shaping the future of intelligent systems. Six years later, we’d put it more precisely: the neural half makes AI usable; the symbolic half makes it trustworthy. Ontology-based RAG is the first architecture we’ve built where each half does exactly the job it’s best at, and where the parts that must never be wrong are the parts that can’t be.
That’s the hybrid we dreamed about that summer. It’s no longer a thesis; it’s a design pattern. And we build it.
Desert Willow Digital Architectures designs and builds AI-integrated software platforms — including ontology-based RAG systems — for startups and growing businesses. If accuracy and security aren’t optional in your domain, the consult is free.
