How I use Hermes as an external memory system

I do not need another chatbot.

I have enough boxes I can type into. I have search bars, notebooks, calendars, task lists, email, Notion, browser history, Slack, text messages, documents, and a pile of half-finished thoughts scattered across systems that all technically “store information.”

That is not the same thing as memory.

Storage is easy.

Memory is being able to recover the right context at the right time.

That is why I have been building a program called Hermes into something more useful than a chat window. I am using it as an external memory system.

Not because I want AI to think for me.

I want it to help me find the thread again after I have dropped it.

The problem is not lack of information

I covered some of this another post about building systems because my brain drops details.

The short version is this: I can understand complex systems, but I do not reliably remember every small detail that matters later.

That creates a strange problem.

I might remember how a network is designed, why a server failed, what a customer needs, or how a compliance requirement connects to a business process.

But I may not remember the name of the person I talked to, what we decided in a meeting, whether I already checked something, or where I put the note that explains all of it.

So I started building systems outside my head.

Calendars help.

Checklists help.

Notion helps.

Automation helps.

But eventually the problem changes. Once you capture enough information, the issue is no longer “where do I write this down?”

The issue becomes:

“Where did I put it?”

“What did I say last time?”

“What do we already know?”

“Is this a new problem or the same problem wearing a hat?”

That is where Hermes started to matter.

What Hermes is for me

Hermes is an open-source AI agent framework from Nous Research.

That sentence is technically true and also not very useful by itself.

For me, Hermes is the machinery behind Holly.

Holly is my ship’s computer.

Yes, for otehr geeks out there, its a Red Dwarf reference. Specifically the Norman Lovett version of Holly: calm, dry, strangely competent, and slightly damaged by spending too long alone with the machinery.

That tone matters more than it sounds like it should.

I did not want a cheerful productivity assistant. I did not want an inspirational life coach. I did not want an AI pretending every calendar event was an exciting opportunity.

I wanted a practical system that could help me keep track of things, answer directly, use tools, search files, inspect data, remember stable facts, and occasionally point out when something has gone peculiar.

So I configured Hermes around that job.

It remembers durable facts about me and my environment.

It can search past conversations.

It can read and write files.

It can use tools.

It can crawl my blog.

It can help draft posts.

It can keep track of preferences.

It can create reusable skills when we figure out a workflow worth keeping.

That is very different from opening a blank chat with a model that knows nothing about me.

A blank chat is a conversation.

Hermes is becoming infrastructure.

Memory has to be intentional

The dangerous thing about AI memory is that it sounds like magic.

It is not magic.

It is a database with judgment problems if you are careless.

That means memory needs rules.

I do not want Hermes remembering every stray sentence forever. That would turn memory into a junk drawer. I already have enough of those, some of them digital and some of them probably in my RV.

The useful memories are stable facts that reduce future friction.

Things like:

• how I prefer to communicate
• what kind of answers help me
• what systems I use
• what environment Hermes is running in
• what projects matter
• what context should not be mixed together
• what details I will probably forget and ask about later

For example, Hermes knows that I prefer direct answers. It knows that I would rather it act on obvious low-risk steps than ask for permission every three minutes. It knows not to give me vague advice when it can inspect the actual system.

That matters.

If I ask, “When was my last blog post?” Hermes does not need to give me a lecture about blogging platforms. It can check the blog and answer.

When I asked it to crawl my personal blog and get to know me, it pulled the public WordPress API, read the posts, summarized what it learned, and stored the stable parts that would help later.

It did not store everything.

That distinction matters.

Memory should be useful, not invasive.

I separate personal memory from system memory

One thing I have learned is that not all memory belongs in the same bucket.

There are facts about me as a person.

There are facts about my work.

There are facts about my home and RV systems.

There are facts about routers, NAS devices, Home Assistant, Docker, websites, databases, automations, and whatever else I have bolted together because apparently relaxation was not available.

Those should not all be treated the same.

If I ask Hermes what it knows about me, I do not want the first answer to be the IP address of a router.

That is not “me.”

That is equipment context.

Useful equipment context, yes. But still equipment context.

So part of building Hermes as an external memory system has been teaching it the difference.

Personal context belongs in one place.

Infrastructure context belongs in another.

Temporary task progress should usually not become permanent memory.

Procedures should become skills, not random remembered facts.

That last one is important.

If Hermes and I work through a repeatable process, the useful result is not just “remember that we did this once.” The useful result is a reusable procedure.

That is what skills are for.

Memory is for facts.

Skills are for workflows.

Session history is for what happened.

The database knows the difference if you make it. Otherwise it becomes soup.

Session search is underrated

One of the most useful parts of Hermes is not just memory.

It is being able to search past sessions.

Persistent memory is small on purpose. It holds compact durable facts.

Session history is bigger. Messier. More detailed and where the witness statements live.

If I know we talked about something before, Hermes can search previous conversations and recover the relevant context.

That is useful because I often remember that a conversation happened, but not the exact phrase, date, or decision.

Without session search, I am stuck trying to reconstruct the past from whatever scraps are still floating around in my head.

With session search, I can ask.

That changes how I use AI.

The assistant is not just answering from the current chat. It can help reconnect work across time.

That is the part most normal chatbots still fail at. They treat every conversation like it was born five minutes ago with no childhood and no responsibilities.

Tools make the memory useful

Memory by itself is not enough.

Hermes also needs tools.

If I ask about my blog, it can check the blog.

If I ask about a file, it can read the file.

If I ask about a database, it can query the database.

If I ask about a system, it can inspect the machinery.

This matters because memory can be stale.

A remembered fact is not proof that something is true right now.

Hermes may remember that my blog exists, but if I ask for the latest post, it should check the live site. It should not guess from memory.

That is one of the reasons I like this setup. The memory gives context, but the tools provide evidence.

The memory says, “Peter has a personal blog at peterzendzian.com.”

The tool says, “The latest post is this one, published at this time.”

That is the difference between being helpful and confidently making things up.

I have enough of that already. It is called the internet.

How I use Hermes day to day

I am still building the system, but the pattern is becoming clear.

I use Hermes for things like:

• recovering context from past conversations
• remembering stable preferences
• keeping track of systems I do not want to re-explain
• drafting blog posts from raw notes
• crawling my own site and summarizing what changed
• turning repeated workflows into skills
• checking files, databases, logs, and web pages
• helping me decide what should be automated
• acting as a calm second brain when I have too many threads open

That last one is the big one.

I do not need Hermes to be clever for its own sake.

I need it to reduce the number of things I am carrying in my head.

If I have to remember every detail of every system, every preference, every note, every prior decision, and every next step, the system has failed.

The whole point is to move some of that load outside my head and into something I can query.

What I had to define

Hermes did not become useful just because I installed it.

I had to define what useful meant.

For me, that meant giving it rules.

Answer directly.

Do not be corporate.

Do not cheerlead.

Use tools instead of guessing.

Ask questions only when the ambiguity matters.

Keep personal facts separate from equipment facts.

Do not store secrets.

Do not turn temporary task progress into permanent memory.

Save durable facts compactly.

Create skills for repeatable workflows.

Be blunt when something is uncertain.

That sounds like personality work, but it is really interface design.

The way an assistant communicates changes whether I can use it when I am tired, overloaded, or trying to solve a real problem.

If it gives me five paragraphs of motivational fog when I need one direct answer, it is not helping.

If it asks me for permission on every obvious low-risk step, it is adding friction.

If it remembers the wrong things, it becomes a liability.

So I shaped Holly around the way I actually work.

Calm. Direct. Practical. Mildly suspicious of unsupported claims.

A ship’s computer, not a golden retriever with a login screen.

AI memory needs boundaries

I do not think AI memory should remember everything.

That sounds useful until you think about it for more than twelve seconds.

Some things are sensitive.

Some things are temporary.

Some things will be wrong next week.

Some things belong in a document, not memory.

Some things should be verified every time.

A good external memory system needs boundaries. Otherwise it becomes clutter with confidence.

For me, the rule is simple:

Memory should reduce future explanation.

If I will have to tell the assistant the same stable fact again and again, it probably belongs in memory.

If the information is task progress, a one-time decision, a temporary detail, or something likely to go stale, it probably does not.

If it is a procedure, it belongs in a skill.

If it is raw source material, it belongs in a file or note.

If it is secret, it does not belong in memory at all.

That last one should be obvious, which means it probably needs to be written down twice.

What this changes

Using Hermes this way has not made me perfectly organized.

That is not the goal.

The goal is to have a reliable place to ask:

“What do we know?”

“What did I decide?”

“What did I say I wanted?”

“What system am I working with?”

“What should we check before acting?”

“What am I forgetting?”

That is what external memory is for.

It is not there to replace thinking.

It is there to reduce the cost of recovering context.

That matters for people like me because the hard part is not always solving the problem. Sometimes the hard part is remembering where I left the problem, what I already tried, and why I cared about it in the first place.

Hermes gives me a way back into the thread.

Advice if you build something like this

If you want to use Hermes or any AI assistant as external memory, start small.

Do not try to build a perfect second brain.

That phrase has caused enough trouble.

Start with a few stable facts.

Tell it how you like answers.

Tell it what not to do.

Tell it what systems matter.

Correct it when it gets something wrong.

Store only what will help later.

Use tools to verify live facts.

Keep secrets out of memory.

Turn repeated workflows into reusable instructions.

And most importantly, build the system around your actual behavior.

If you already keep perfect notes, never forget follow-up, always use your calendar, and never lose track of context, you may not need this.

Also, congratulations on being imaginary.

For the rest of us, external memory helps.

Why I am writing about this

I plan to write more about how I use Hermes.

This post is the why.

The how comes next.

I want to show how I structure memory, how I use skills, how I create specialized agents, how I use Hermes for writing, research, troubleshooting, and system management, and how I decide what should be automated instead of trusted to my attention span.

Because Hermes is not just another AI chat box for me.

It is becoming part of how I manage complexity.

It is where I put context I cannot afford to keep dropping.

And if I build it well, it gives me something I have needed for a long time:

A system I can ask when my brain says, “I know we had this handled, but I have misplaced the entire plot.”