I Built My Own AI Agent. It Lives in My Notes.

This isn’t a product. It’s a pattern. I’ve been refining it for the last 12 months and it’s now the operating system underneath most of my workday. If you build something similar, it’ll look different from mine because it’ll be shaped by your work. That’s the point.

the agent

I built an AI agent that lives in my Obsidian vault. Not in someone else’s product. Not behind a chat window I have to remember to open. In my notes. On my own machine. Reading the same files I read.

It runs scheduled tasks on a cron. Some of them run when I’m asleep. Some run before my first meeting of the day. By the time I open my laptop in the morning, today’s daily note already exists, with my calendar pulled, my unread inbox triaged, my carry-forward items inherited from yesterday, and a pre-meeting brief sitting in my inbox for the first call of the day.

I didn’t open a chatbot to ask for any of that. The agent had the work done before I asked.

why a personal agent

The popular AI assistants are general. They know the internet. They don’t know my projects, my team, my one-on-one history with Steve, the vendor I escalate to on the hotel-phone work, the audit cycle that ran in three hours last week, the specific tone I use when I’m drafting an email to my manager.

I don’t want a general assistant. I want one that knows my work. The pattern shift the last 18 months made possible is that you can have both. Use the general models for general work. Build a personal agent for the specific stuff that only matters in your context.

The personal agent is just a small set of cron’d tasks, each one with a clear, narrow job, reading and writing into a vault that’s already structured around how I think. The model is the engine. The vault is the memory. The cron is the loop.

what’s actually in the vault

An Obsidian vault, synced via OneDrive, with folders for:

• Daily notes — one file per day, generated by a 6 AM scheduled task
• Weekly notes — one per week, generated Sunday with priorities and carry-forward
• Project notes — one per active project, updated nightly with what changed
• CRM — one note per person I interact with, with a snapshot, recent interaction log, and personal-rapport details
• Meeting notes — one per meeting, generated from Teams + Fireflies transcripts
• Pre-meeting briefs — one per day, generated 30 minutes before the day’s first call, emailed to me
• Memory — running daily log files capturing what happened, what was decided, what’s open
• Wiki / reference — long-lived knowledge: how-to’s, patterns, decisions, organizational context
• Scripts — Python and PowerShell helpers that the agent and I both call

None of that is novel. People have been building knowledge vaults like this for years. What changed is the agent on top.

what the agent does

A handful of scheduled tasks, each focused on one job:

Morning daily note generator (weekdays 6 AM). Pulls my Outlook calendar, my unread inbox, my Teams chat highlights from the last 24 hours. Reads yesterday’s daily note to inherit carry-forward items. Reads the current week’s weekly note for priorities and deadlines. Writes today’s daily note as a structured markdown file: today’s schedule, inbox highlights with priority sorting, carry forward, focus block. Done by 6:10 AM.

Pre-meeting brief (weekdays 6:35 AM). Reads today’s calendar. For each meeting with attendees, looks up each attendee in the CRM, surfaces their snapshot + most recent interaction with me + a personal-rapport nugget. Detects project IDs and SD ticket references in the meeting subject; surfaces the relevant project’s recent activity. Writes the brief and emails it to me. By the time I sit down for my first meeting, I have the people loaded.

Meeting transcript ingest (daily 7 PM). Pulls today’s Teams transcripts via Graph API and Fireflies transcripts via their MCP. Saves the VTT files to the vault. Creates meeting notes with the transcript linked. Dedups across sources.

End-of-day close (weekdays 6:20 PM). Reads today’s daily note, today’s meetings, recent activity in projects I own. Extracts substantive interactions from today’s Teams chats and emails. Appends dated entries to the relevant CRM pages. Updates project notes with status changes. Appends an EOD block to the current weekly note. Read-only on all communications — never sends.

Nightly memory update (daily 10 PM). Reads everything that changed in the vault today. Writes a structured summary into the daily memory file. Evaluates the long-term memory for durable updates. Keeps the long-term memory under a hard size cap.

Nightly code review (daily 10:30 PM). Walks my watched repositories. For each one: fetches latest from origin (read-only), reviews the diff of anything new, flags security / quality / dependency / style issues. Writes the review into a dated review file. By morning, I know if a script I wrote yesterday looks shaky in retrospect.

Vault health check (daily 8:05 AM). A dead-man’s-switch monitor. Confirms today’s daily note exists. Confirms the git snapshot is fresh. Confirms the M365 token is valid. If any of that’s wrong, emails me a single concise alert. If everything’s fine, silent.

And a few more — saturday-morning checkout-email prep, weekly CRM enrichment from the week’s transcripts, weekly vault linting, etc.

the discretion principle

The most important thing I learned building this: private capture is fine. External artifacts are different.

The vault captures everything — including things I’d never want surfaced in a status update to my manager. My honest read on a candidate. My frustrations with a vendor. Notes from a conversation with a friend about whether to take a different job. The vault doesn’t judge any of it.

But routines that produce external artifacts — the saturday-morning checkout email to my manager, any draft I’d publish on LinkedIn or this blog — have an explicit discretion filter. They skip the categories that should stay private (compensation conversations, career evaluation thoughts, anything I wouldn’t want a specific audience to read). That filter isn’t an afterthought. It’s baked into the prompts for each routine that touches an external surface.

If you build something like this, build the discretion filter before you ship the first external artifact. The cost of getting it wrong is real, and the cost of getting it right is small.

how the day changes

The biggest shift isn’t time saved. It’s arrival shape. I show up to my desk in the morning and the day is already organized. Yesterday’s open loops are surfaced. Today’s meetings have context loaded. The week’s priorities are visible. My inbox has been triaged for noise. If something in my infrastructure broke overnight, I already have an email about it.

The dread number goes down. The first 10 minutes of the day aren’t a “what’s on fire” scramble — they’re a glance at the daily note that the agent already wrote for me, plus a decision about which of today’s three priorities I’m starting with.

That arrival shape compounds. By the time I’m in the first meeting, I’m prepped. By the time the first meeting ends, the agent has already ingested any new transcripts and surfaced action items. By end of day, the EOD close has organized everything that happened. By the time I open my laptop the next morning, the agent has already done the consolidation.

I never feel behind on my own information. That’s the real product.

where it doesn’t work

It does not work if you don’t have a vault. If you’re not already writing notes in a structured way, building the agent first won’t save you. The agent’s value comes from operating on a well-organized substrate. Start with the vault. Start writing notes. Build the agent later.

It does not work for free. The model calls cost money. The cloud infrastructure (if you use any) costs money. The time investment to set up the routines and refine the prompts is real. Mine cost maybe $20-40 a month in API calls plus a few hundred hours of evening/weekend setup time across a year. If you’re not getting at least an hour a day back from it, you’re underinvesting in the loop.

It does not work if you don’t trust the discretion filter. The whole pattern depends on private capture being actually private. If your vault gets read by your employer, an auditor, or a partner, you need to know what’s in it and what isn’t. Treat the vault like a personal journal, not a corporate document.

the pattern, summarized

Vault (the memory) + scheduled tasks (the loop) + AI models (the engine) + discretion filter (the boundary) = a personal agent that gets stronger every week.

You can buy parts of this. Microsoft Copilot is shipping a version. ChatGPT has memory now. Some of the AI-first note apps are building toward it. None of them are quite this — yet. They’re getting closer.

What you build yourself has one advantage the products don’t: it knows you. Not the cohort of people like you. Not the average user. You. Your vendors, your team, your habits, your vocabulary, your tone with your manager versus your tone with your reports. That’s the layer the products are still figuring out.

It took me a year to get to here. The first month was clunky. By the third I’d shipped most of the daily routines. By the sixth I’d stopped having to think about it. It just ran.

If you build something like this, expect a similar shape. The first month is the hardest. After that, the agent compounds.

If you’re building something similar and want to compare notes — I’d love to. The patterns I described above are mine. They’re not the only ones that work. The interesting conversations are about the edges where they break.