Agent memory is just writing JSON to SQLite: building Deterministic Scaffolding

There is a deeply frustrating semantic drift happening right now regarding the concept of "agent memory." A massive chunk of the industry has started treating massive vector databases shoved into expanding context windows as if they represent literal cognitive recall. We are fundamentally conflating cosine similarity with factual truth. The absolute fragility of this architecture was brought into sharp relief for me recently while reading Jordy Zomer's excellent Lemmalog post exploring how AI models handle memory during program analysis.

Zomer outlines precisely how these models completely lose the plot when attempting to track strict logical state over time during deep code inspection. Following the resulting Hacker News discussion about his findings, a recurring theme surfaced among engineers who are currently trying to deploy these systems in production environments. Any time you demand a large language model to juggle complex, shifting application state and historical facts simultaneously, the system inevitably collapses under its own mathematical weight.

The enterprise ecosystem is flooded with startups selling opaque memory endpoints, aggressively marketing emergent reasoning and complex cognitive capabilities. We need to immediately strip away this academic marketing fluff. These language models are fundamentally calculators for words—predicting the most statistically probable next token based on their training weights. They cannot remember anything; they only generate text based on the immediate strings you feed them. We can fix this architectural anti-pattern by adopting Deterministic Scaffolding. In this model, you completely demote the AI. You treat the model purely as a fuzzy text parser—a highly disposable mechanism whose sole job is to read messy, unstructured input and reliably output strictly typed JSON into a boring, verifiable database.

This approach completely changes the economics of background processing. Checking the official OpenAI pricing page for their GPT-4o-mini model, the current rate sits at $0.150 per 1M input tokens. If we imagine a hypothetical system that extracts hard facts from a messy 1,000-token chunk of transcribed meeting notes, the mathematical cost calculates out to roughly $0.00015 per operation. This is so economically trivial that you can deploy it indiscriminately! You could theoretically run this extraction process across every single raw text file in your entire personal namespace for just a few pennies, transforming a highly disorganized hard drive into a deeply structured, queryable knowledge graph.

To see what this actually looks like in practice, you don't need a heavy enterprise framework. We can observe the absolute simplest, dumbest version of genuine agent memory by stringing together a minimal shell pipeline. Using the llm CLI utility and passing that output down to the sqlite-utils command-line tool, you can achieve the architectural proof directly in the terminal:

cat messy_notes.txt | llm -s "Extract a strict JSON array of facts: [{'entity': 'string', 'fact': 'string'}]" | sqlite-utils insert memory.db facts -

That one-liner is fun, but to build a robust system, we need to wrap this in actual application logic. Conceptually, we can write a lightweight Python script that loops over a directory of text files, calls out to the AI API to generate those JSON arrays, and uses a standard database driver to log the results. Here is exactly how you would wire that up using the Python APIs for llm and sqlite-utils:

import llm
import sqlite_utils
import json
import glob

# Initialize our deterministic memory store
db = sqlite_utils.Database("memory.db")

# Load the model (using a fast, cheap model for fuzzy parsing)
model = llm.get_model("gpt-4o-mini")
model.key = "YOUR_API_KEY"

system_prompt = """
You are a strict data extraction tool.
Read the provided text and output ONLY a valid JSON array of objects.
Schema: [{"entity": "Name or concept", "fact": "Stated truth"}]
"""

for filepath in glob.glob("notes/*.txt"):
    with open(filepath, "r") as f:
        content = f.read()

    # Execute the fuzzy parse
    response = model.prompt(content, system=system_prompt)

    try:
        # Parse the string back into strict JSON
        facts = json.loads(response.text())
        
        # Insert directly into the SQLite table
        db["facts"].insert_all(facts, alter=True)
        print(f"Processed {filepath}")
    except json.JSONDecodeError:
        print(f"Failed to parse JSON for {filepath}")

Let's examine what that script actually accomplishes. The raw text goes into the language model heavily constrained by a system prompt dictating a rigid schema. The model performs its specific role as the fuzzy parser, discarding the conversational fluff and converting natural language into strict JSON syntax. Forcing the LLM to translate the meeting into a static artifact entirely bypasses the need for complex vector embeddings. The script then passes that resulting array directly into a table named facts inside a local SQLite database file.

It usually spits out a JSON array looking something like this:

[
  {"entity": "Project Phoenix", "fact": "Launch delayed to Q3"},
  {"entity": "Sarah", "fact": "Handling the new API documentation"}
]

Once that process completes, your agent memory relies entirely on writing strings to disk. If you want to know what the agent has logged, you simply run a standard SELECT * FROM facts; statement to instantly review the extracted entities. If the model hallucinated a detail during the extraction phase, you can just delete the offending row. You aren't guessing what the context window might contain or worrying about prompt injection silently polluting a vector index.

Contrast this highly visible, verifiable data structure with the opaque vibes-based search currently dominating the industry. Relying on vector databases for memory means hoping the cosine distance algorithm happens to align with factual reality at query time. Using deterministic extraction locks the reality down into a traditional table before the agent ever takes its next action.

Zooming out from this single Python script, the broader implications for how we build AI-assisted software are incredibly stark. The industry wants you to build autonomous agents that operate entirely inside the latent space of a proprietary model. They want to sell you subscriptions to their hosted memory endpoints, obfuscating your application state behind their API keys and leaving you permanently vulnerable to their unpredictable failure modes.

Looking back at that brief block of code, it proves we have a much safer, vastly cheaper alternative readily available. The massive cloud providers want you to buy into their complex, proprietary memory services, but we cannot outsource our application logic to black-box APIs! If you want an agent you can actually trust, you have to strip away the magic: demote the LLM to a mere parser, and let a boring, deterministic database handle the truth.