Ch 14: Knowledge layer: Karpathy's LLM Wiki in Obsidian

Collection: Nate Herk: AIOS course · Creator: Nate Herk (@nateherk)

Original: raw source · demo-lab/research/nate-herk-course/transcript.txt @ 2:04:30 · 4,737 words · 13 sections

Content by Nate Herk (@nateherk). Chapter starts at 2:04:30. Auto-transcribed; names may be misheard.

Summary

• So if you guys are interested in figuring out how you can turn all of your business context, all of your meeting transcripts, all of your YouTube video transcripts into a visual thing like this using the LLM Wiki, then I'm going to head into a video right here where I explain exactly how to set this up.
• What you're looking at right here is 36 of my most recent YouTube videos organized into an actual knowledge system that makes sense.
• So as this continues to fill up, we can start to see patterns and relationships between every tool or every skill or every MCP server that I might've talked about in a YouTube video and I can just query it in a really efficient way now that we have this actual system set up.
• Or after that, I have all the different techniques, agent teams, sub-agents, permission modes, the WAT framework, and then we've got different concepts, MCP servers, RAG, vibe coding.
• Because it's interesting, in my personal kind of second brain, the wiki is literally just marked on files.

Entities mentioned

YouTube · LLM · Claude Code · Obsidian · AI 2027 · Andrej Karpathy · MCP · WAT · AIOS · Visual Studio Code · GitHub · VS Code · Nate

Topics

Topic: second brain · wiki · youtube · Topic: scheduled tasks · routines · run · Topic: youtube video · google · gave

Key points by section

[2:04:30]

• Maybe you've got a deadline due at the end of the week and you need a three day sprint to be constantly checking in on the team and checking in on progress.
• Whether that is a local routine or a cloud routine, you understand how you can set them up.
• So if I open up Obsidian real quick, you guys remember how I talked about putting all of my YouTube transcripts into my AIOS.
• Now, Obsidian giving me this visual layer doesn't change anything fundamentally about how my AI OS actually uses the data and is able to look at it and talk to it and edit it.
• Topic: Topic: scheduled tasks · routines · run

[2:06:00]

• So if you guys are interested in figuring out how you can turn all of your business context, all of your meeting transcripts, all of your YouTube video transcripts into a visual thing like this using the LLM Wiki, then I'm going to head into a video right here where I explain exactly how to set this up.
• What you're looking at right here is 36 of my most recent YouTube videos organized into an actual knowledge system that makes sense.
• It also has techniques like the WT framework or bypass permissions mode or human review checkpoint.
• So as this continues to fill up, we can start to see patterns and relationships between every tool or every skill or every MCP server that I might've talked about in a YouTube video and I can just query it in a really efficient way now that we have this actual system set up.
• Topic: Topic: second brain · wiki · youtube

[2:07:30]

• So I've got one second brain here and then I've got one basically YouTube knowledge system and I could combine these or I could keep them separate and I can just keep building more knowledge systems and plug them all into other AI agents that I need to have this context.
• So Andrej Karpathy just released this little post about LLM knowledge bases and explaining what he's been doing with them.
• So let's do a quick breakdown and then I'm gonna show you guys how you can get this set up in basically five minutes.
• Something I've been finding very useful recently is using LLMs to build personal knowledge bases for various topics of research interest.
• Topic: Topic: second brain · wiki · youtube

[2:09:00]

• He said here, I thought that I had to reach for fancy rag, but the LLM has been pretty good about auto maintaining index files and brief summaries of all documents, and it reads all the important related data fairly easily at this small scale.
• So there's a few other things that we'll cover later, but the TLDR is you give raw data to Claude Code, it compares it, it organizes it, and then it puts it into the right spots with relationships, and then you can query it about anything.
• But this method using Karpathy's LLM Wiki makes knowledge compound like interest in a bank.
• You don't need a fancy vector database embeddings or complex infrastructure.
• Topic: Topic: second brain · wiki · youtube

[2:10:30]

• Or after that, I have all the different techniques, agent teams, sub-agents, permission modes, the WAT framework, and then we've got different concepts, MCP servers, RAG, vibe coding.
• So in this case, in the YouTube project, the log isn't huge, because I only ran one huge batch of the initial 36 YouTube videos.
• One X user turned 383 scattered files and over a hundred meeting transcripts into a compact Wiki and dropped token usage by 95% when querying with Claude.
• And Karpathy even said, hey, you know I left this prompt vague so that you guys can customize And I'll show you the ways in my two different vaults right now that it changed things a little bit based on the context and understanding of what the project is actually for.
• Topic: Topic: second brain · wiki · youtube

[2:12:00]

• So we're gonna come right back to this in a sec, but the first prereq that we're gonna do, it's not necessary, but I like to have a nice little front end to see the relationships, is we're gonna go to Obsidian and download it.
• So if you just go to obsidian.md, you can see this is the completely free tool, and you're going to go ahead and download it.
• So I'm just going to call this one demo vault, and you're going to choose a location where you want to put this.
• So I'm just throwing this on my desktop, and I'm going to go ahead and create this vault.
• Topic: Topic: youtube video · google · gave

[2:13:30]

• So anyways, on the right we have this Claude Code running, and on the left we have our Obsidian Vault, and you can see it just created those two folders.
• Because it's interesting, in my personal kind of second brain, the wiki is literally just marked on files.
• But you can see that it went ahead and it created a claw.md, it created an index and a log, and then a few different folders in our wiki.
• But now it's saying, hey, let's go ahead and try it out, drop in your first source into the raw folder and tell me to ingest it.
• Topic: Topic: second brain · wiki · youtube

[2:15:00]

• Or we can just get an obsidian extension, which lets us basically take articles right from the web and just put it right into our vault, super easy.
• So then when you're in the article that you want, you basically just click on your extensions, you open up Obsidian Web Clipper, and then you can just chuck it into your vault.
• So you can go ahead and click Add to Obsidian, open Obsidian, and then now you can see in my raw section, we have this AI 2027 source with the title, the source, and it's not super, super populated yet because the LLM in Claude Code is gonna do that.
• So there's different ways that you can set up the project as you saw with mine, one for YouTube, one for just personal second brain.
• Topic: Topic: second brain · wiki · youtube

[2:16:30]

• Now, one thing I wanna call out real quick is with this extension, if you go here and you open up the options for it, you can see that you can actually change where by default the folders are dropped, which is in the location section.
• So that's just a quick example of what it might look like for you to give it some more context to continuously build your project.
• So I'm going to switch over over here to the graph view, because I think it will be interesting to see as it is starting to go through and create those different Wiki files, it's going to go ahead and it's going to create all those relationships and we'll be able to watch it in real time.
• All right, so it's creating all of the wiki pages now, and you can see that it said it's going to make about 25 because there's so much stuff going on in the original AI 2027 article.
• Topic: Topic: second brain · wiki · youtube

[2:18:00]

• When I uploaded the 36 YouTube transcripts in batch, it took about 14 minutes.
• So it kind of just depends, but it created 23 Wiki pages.
• And then it asks some questions about it so that it can help make the relationships and make the structure even better.
• So let's just see, if I click into the main source, we can see the tags that it got, we can see the authors and we can click around.
• Topic: Topic: second brain · wiki · youtube

[2:19:30]

• And whenever I wanna ask questions about YouTube, or if I wanna turn this into like a website, I can just do that from here.
• And it can crawl through the Wiki, it can read the index and it knows how this stuff works because you can give it the claw.md so it understands the project as well.
• So for example, in this one, which is just my second brain where we have all of the different things about like, I drop in my meeting recordings, I drop in, you know, click up channels, summaries and things like that.
• Now, if you guys remember, if you watched my video on setting up an executive assistant, I used to do this with context files inside of this project.
• Topic: Topic: second brain · wiki · youtube

[2:21:00]

• So it basically helps you run a lint, you know, every day, every week, whenever you want, which helps make sure that everything is scalable and structured in the right way.
• So now the final question about this that I wanted to cover is like, does this kill semantic search rag?
• I was in my hurt brain where I dumped in a bunch of information about Karpathy's LLM knowledge.
• And I just said, hey, can you please explain Karpathy knowledge as simple as possible, keep it super concise and compare it to typical semantic search rag.
• Topic: Topic: second brain · wiki · youtube

[2:22:30]

• and it compares it here to the actual semantic search rag.
• So we're getting a deeper understanding of relationships because they're links rather than just saying, hey, these chunks seem similar.
• Whereas with semantic search, you need an embedding model, you need a vector database, and a chunking pipeline.
• You clean up things, you add more articles, you give it more context, rather than having to re-embed when things change.
• Topic: Topic: second brain · wiki · youtube

Backlinks

AI 2027 · AIOS · Andrej Karpathy · Claude Code · GitHub · LLM · MCP · Nate · Obsidian · VS Code · Visual Studio Code · WAT · YouTube · Index · Topic: scheduled tasks · routines · run · Topic: second brain · wiki · youtube · Topic: youtube video · google · gave