Andrej Karpathy said manual data ingest for AI agents is too slow. I built the fix.
Andrej Karpathy said manual data ingest for AI agents is too slow. I built the fix. Last week Andrej Karpathy posted about building personal knowledge bases for LLM agents. He described his workflo...

Source: DEV Community
Andrej Karpathy said manual data ingest for AI agents is too slow. I built the fix. Last week Andrej Karpathy posted about building personal knowledge bases for LLM agents. He described his workflow: manually indexing source documents into a raw/ directory, writing custom search tools, building a naive search engine over his wiki. Then he wrote this: "I think there is room here for an incredible new product instead of a hacky collection of scripts." He was right. So I built it. The Problem He Identified Karpathy's workflow is brilliant but it requires him to manually curate every source. He clips articles with Obsidian Web Clipper, downloads images locally, feeds them one-by-one to his LLM agent. For a researcher at his level that works. For a developer building production AI agents for clients, it doesn't scale. Here's the specific failure mode I kept hitting: You build a RAG pipeline. It works. A user asks about a Python library. Your retriever finds a Stack Overflow answer with cosi