Close Menu
    Facebook X (Twitter) Instagram
    TRENDING :
    • This Is the Reason Your AI Actually Remembers Anything
    • Kenny Johnson Cleared to Play Saturday
    • The future of live entertainment
    • Charlotte Terbrack Gets Showered With Love From Adoring Fans
    • People are using Meta’s Muse to purge who they’re following on Instagram
    • Kalshi Promo CBSSPORTS55 Offers $55 for Top College Football Games
    • The poster for the Sam Altman movie looks like AI slop. It’s actually brilliant marketing
    • Rachaad White Not Spotted at Saturday Practice
    Populist Bulletin
    • Home
    • US Politics
    • World Politics
    • Economy
    • Business
    • Headline News
    Populist Bulletin
    Home»Business»This Is the Reason Your AI Actually Remembers Anything
    Business 7 Mins Read

    This Is the Reason Your AI Actually Remembers Anything

    Business 7 Mins Read
    Share Facebook Twitter Pinterest LinkedIn Tumblr Telegram Email Copy Link
    Follow Us
    Google News Flipboard
    Share
    Facebook Twitter LinkedIn Pinterest Email


    Opinions expressed by Entrepreneur contributors are their own.

    Key Takeaways

    • It’s usually smarter to start simple. A basic cache is cheap and fast, and it teaches you what people actually want before you build anything more advanced.
    • A simple cache has one weakness: It only recognizes a question if it’s asked the exact same way. The moment someone rephrases things, which happens constantly, it can miss answers it technically already has.
    • A smarter memory system costs a little more to run and can feel slightly slower, but that trade is worth it. It understands people however they phrase things, remembers for longer and can be reused across other projects.

    During the World Cup this year, I built Rune Score, a sports analytics chatbot. From the outside, it looks like a simple chatbot. Under the hood, it’s a distributed system that has to handle the freshness of sports data, user rephrasing, extraction failures, LLM call costs, streaming, memory, fallbacks and security. What I learned building it applies to almost any AI product that has to stay fast, cheap and accurate at the same time.

    Two quick definitions before we go further, since neither tool is exactly a household name. Redis is “an open-source, in-memory data structure store used as a database, cache, message broker and streaming engine,” most commonly used in AI products as a fast cache that stores one exact answer under one exact label. RAG, short for retrieval-augmented generation, is defined by AWS as “a technique used to augment a large language model (LLM) with external data, such as a company’s internal documents.” Instead of matching an exact label, it searches for a broader knowledge base using meaning rather than exact wording. Redis is precise and cheap but rigid. RAG is more flexible and durable, but costs more to run.

    Why I didn’t start with RAG

    My first move wasn’t a vector database, it was a Redis cache built around a simple labeling rule: Identical requests get an identical key, something like the football matchup Argentina vs. Spain. The savings were immediate. As I documented in my LinkedIn article on Rune Score, “without the cache, a fresh extraction followed by an analysis can consume up to 23,000 input tokens: scraped content, context, history. With a cache that hits the same key, that number drops to about 5,000 tokens,” a 4.5-fold reduction in the single most expensive part of the pipeline.

    AI cost usually comes from what you repeat unnecessarily. If 10 people ask your product the same question within a reasonable time window, rerunning the full, expensive pipeline 10 times is money burned for no reason.

    Where Redis breaks down

    People don’t talk in fixed labels. They might write “FC Barcelona vs Real Madrid,” or “Barça – Madrid,” and a strict cache treats each variant as a completely different request. That means duplicate work, a missed answer that was already sitting there or an unnecessary rerun of the whole expensive pipeline just because the wording didn’t match.

    This is where RAG earns its place, through a technique called embeddings. According to Google Cloud’s documentation, “the closer objects are in the embedding space, the more semantically similar they are, and greater distances indicate greater semantic dissimilarity.” In plain terms, a knowledge base built this way can retrieve a past answer even when the question is worded completely differently, because it searches by meaning rather than exact wording.

    The part that actually breaks quality

    Here’s a lesson worth internalizing regardless of what you’re building: The biggest quality bottleneck in a RAG system usually isn’t the retrieval technology itself; it’s what feeds it. This is the classic computing idea that TechTarget defines as garbage in, garbage out, or GIGO, “the idea that in any system, the quality of output is determined by the quality of the input.” In my pipeline, every time I cut corners on how much content got extracted from a source, the resulting analysis got visibly worse downstream.

    The same logic applies to how the system ranks relevance. Trusting a single relevance score is fragile. Redis’s own engineering blog puts it plainly: “the consistent takeaway across these studies is that combining retrievers tends to surface more relevant results in the top positions than either method on its own.” Combining exact keyword matches, meaning-based search and how recent the information is, then merging those into one ranking, beats relying on any single signal alone.

    RAG costs more, and that’s fine

    In my experiment with Rune Score, RAG hit brings a request down from 20,000 to 8,000 tokens — a 60% reduction. Compared to the 80% reduction from an exact Redis hit, that’s less economical and slightly slower, but that gap isn’t a strike against RAG. What it buys instead is tolerance for how people actually phrase things, memory that outlives a cache’s expiration window and an architecture that’s reusable well beyond the one product you built it for.

    The bottom line

    Redis and RAG aren’t competing choices; they’re sequential ones. The best refactor isn’t the one that’s theoretically right; it’s the one that lets you roll back in seconds if the real metrics don’t hold up. Build the cheap, fast cache first, let it teach you what your users actually ask for and only then invest in a full retrieval system for the cases a cache structurally can’t handle.

    Key Takeaways

    • It’s usually smarter to start simple. A basic cache is cheap and fast, and it teaches you what people actually want before you build anything more advanced.
    • A simple cache has one weakness: It only recognizes a question if it’s asked the exact same way. The moment someone rephrases things, which happens constantly, it can miss answers it technically already has.
    • A smarter memory system costs a little more to run and can feel slightly slower, but that trade is worth it. It understands people however they phrase things, remembers for longer and can be reused across other projects.

    During the World Cup this year, I built Rune Score, a sports analytics chatbot. From the outside, it looks like a simple chatbot. Under the hood, it’s a distributed system that has to handle the freshness of sports data, user rephrasing, extraction failures, LLM call costs, streaming, memory, fallbacks and security. What I learned building it applies to almost any AI product that has to stay fast, cheap and accurate at the same time.

    Two quick definitions before we go further, since neither tool is exactly a household name. Redis is “an open-source, in-memory data structure store used as a database, cache, message broker and streaming engine,” most commonly used in AI products as a fast cache that stores one exact answer under one exact label. RAG, short for retrieval-augmented generation, is defined by AWS as “a technique used to augment a large language model (LLM) with external data, such as a company’s internal documents.” Instead of matching an exact label, it searches for a broader knowledge base using meaning rather than exact wording. Redis is precise and cheap but rigid. RAG is more flexible and durable, but costs more to run.

    Why I didn’t start with RAG

    My first move wasn’t a vector database, it was a Redis cache built around a simple labeling rule: Identical requests get an identical key, something like the football matchup Argentina vs. Spain. The savings were immediate. As I documented in my LinkedIn article on Rune Score, “without the cache, a fresh extraction followed by an analysis can consume up to 23,000 input tokens: scraped content, context, history. With a cache that hits the same key, that number drops to about 5,000 tokens,” a 4.5-fold reduction in the single most expensive part of the pipeline.



    Source link

    Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

    Related Posts

    The future of live entertainment

    November 21, 2026

    People are using Meta’s Muse to purge who they’re following on Instagram

    November 21, 2026

    The poster for the Sam Altman movie looks like AI slop. It’s actually brilliant marketing

    November 21, 2026
    Top News
    US Politics 7 Mins Read

    Lindsey Graham’s Defense of Brett Kavanaugh Told Us Everything

    US Politics 7 Mins Read

    The senator’s infamous cry of rage spoke not only to his cruelty but his larger…

    Cinnamon Mania Boosts Cinnabon’s Best Growth Year Ever

    July 24, 2026

    The US Needs India to Buy Coal. Who Pays the Cost?

    January 15, 2026

    Controversial changes to the definition of a ‘professional’ degree were just blocked. What does it mean for student borrowing caps?

    June 27, 2026
    Top Trending
    Business 7 Mins Read

    This Is the Reason Your AI Actually Remembers Anything

    Business 7 Mins Read

    Opinions expressed by Entrepreneur contributors are their own. Key Takeaways It’s usually…

    World Politics 1 Min Read

    Kenny Johnson Cleared to Play Saturday

    World Politics 1 Min Read

    Good news for the lineup card: Kenny Johnson has been cleared to…

    Business 5 Mins Read

    The future of live entertainment

    Business 5 Mins Read

    Change is in the air. Digital saturation is reshaping nearly every dimension…

    Categories
    • Business
    • Economy
    • Headline News
    • Top News
    • US Politics
    • World Politics
    About us

    The Populist Bulletin was founded with a fervent commitment to inform, inspire, empower and spark meaningful conversations about the economy, business, politics, government accountability, globalization, and the preservation of American cultural heritage.

    We are devoted to delivering straightforward, unfiltered, compelling, relatable stories that resonate with the majority of the American public, while boldly challenging false mainstream narratives that seem to only serve entrenched elitists, and foreign interests.

    Top Picks

    This Is the Reason Your AI Actually Remembers Anything

    November 21, 2026

    Kenny Johnson Cleared to Play Saturday

    November 21, 2026

    The future of live entertainment

    November 21, 2026
    Categories
    • Business
    • Economy
    • Headline News
    • Top News
    • US Politics
    • World Politics
    Copyright © 2025 Populist Bulletin. All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.