Nanochat: Build Your Own ChatGPT for $100

At Emelia, a B2B prospecting platform that integrates artificial intelligence into its enrichment and cold emailing features, we closely follow breakthroughs that make AI more accessible. Through Bridgers, our digital agency specializing in AI, we help companies understand and leverage these technologies. When Andrej Karpathy publishes a project that lets anyone build their own ChatGPT for $100, it is exactly the kind of innovation that matters to our clients and community. Here is everything you need to know about nanochat and its most recent extension, autoresearch.

Andrej Karpathy is not your average developer. He holds a PhD in computer science from Stanford, where he studied under Fei-Fei Li. He co-founded OpenAI in 2015 before joining Tesla as Senior Director of AI, where he led the development of Autopilot and Full Self-Driving (FSD). He later returned to OpenAI before dedicating himself to education and open source projects.

His YouTube channel and courses (notably the acclaimed "Neural Networks: Zero to Hero") have become global references for anyone wanting to understand deep learning. Karpathy has a rare ability to combine cutting-edge research expertise with exceptional teaching talent. Nanochat is a direct expression of that philosophy: making the construction of a large language model (LLM) understandable and reproducible by anyone.

Nanochat is an open source project published on October 13, 2026 on GitHub, where it has already accumulated over 42,900 stars. Its objective is simple yet ambitious: provide the complete pipeline to train a ChatGPT-style chatbot, from the tokenizer to the web interface, all runnable on a single 8xH100 GPU node for approximately $100.

The project does not pretend to rival GPT-4 or Claude. It is an educational and experimental tool. Karpathy describes it as "the best ChatGPT that $100 can buy." The source code is roughly 8,000 lines (stripped down to about 1,000 for the core), written primarily in Python (75.8%), with Jupyter Notebook (17.2%), HTML (3.9%), and Shell (3.1%).

The central script, speedrun.sh, launches the entire training and inference cycle with a single command. This simplicity is what makes the project so powerful.

The first step converts raw text into tokens, the numerical units the model can process. Nanochat uses a custom BPE (Byte Pair Encoding) tokenizer written in Rust for performance. This step is often glossed over in tutorials, but Karpathy makes it transparent and fully modifiable.

The model is pre-trained on the FineWeb-EDU dataset, consisting of 1,822 randomly shuffled data shards, each about 100 MB. More recent versions of the project have switched to the NVIDIA ClimbMix dataset, which yields better results at a similar cost. Pre-training accounts for the bulk of the compute time.

After pre-training, the model goes through a mid-training phase on Smoltalk+ chat data, which takes roughly ten minutes. This is followed by supervised fine-tuning (SFT) that transforms the base model into a conversational assistant capable of answering questions.

An optional reinforcement learning (RL) step allows further refinement of the model's responses. Nanochat also integrates evaluation benchmarks like HumanEval (for code) and GSM8K (for mathematics), enabling objective measurement of the model's progress at each stage.

The pipeline concludes with a ChatGPT-style web interface, allowing you to interact with the trained model directly in a browser. The end result is a functional chatbot, built entirely from scratch.

Nanochat offers three depth levels, each with a different cost-to-performance ratio:

The first tier, called "Kindergarten," produces a functional but limited chatbot. It can hold a simple conversation and answer basic general knowledge questions, but it hallucinates frequently and lacks nuance. The second tier, at $300, is significantly more impressive: it surpasses the performance of GPT-2, the model that made headlines when OpenAI published it in 2019. The third tier, at $1,000, is still under development.

This is the major development of March 2026 and arguably the most revolutionary aspect of the entire project. On March 7, 8, and 9, Karpathy published a series of tweets and a dedicated GitHub repository for "autoresearch": a system where AI agents autonomously optimize nanochat training.

The principle is elegant. Instead of a human researcher manually testing modifications to the architecture, hyperparameters, or data pipeline, an AI agent performs these iterations automatically. Each "dot" in the results represents a 5-minute training run. Over two days, the system made more than 700 changes autonomously.

The results published on March 9 are remarkable. Karpathy left autoresearch running for two days on a depth-12 model. The system found approximately 20 changes that improved validation loss. The most striking result: the time required to reach GPT-2 level dropped from 2.02 hours to 1.80 hours, an improvement of roughly 11%. These optimizations had been missed by human researchers.

The autoresearch code fits in 630 lines and runs on a single GPU. It is a striking demonstration of what AI agents can accomplish in machine learning research, even with modest resources.

Karpathy went further by experimenting with multi-agent organizations. He configured 8 agents (4 Claude, 4 Codex), each with a dedicated GPU. He tested two configurations: 8 independent researchers working in parallel, and a hierarchical structure with 1 "chief scientist" coordinating 8 juniors.

Each research program is managed as a Git branch, allowing discoveries to be tracked and merged. As Karpathy puts it: "The goal is that you are now programming an organization." This vision of autonomous agent-driven research may be more significant than nanochat itself.

Shopify CEO Tobi Lutke was among the first to test autoresearch, demonstrating that interest in this approach extends well beyond the academic community.

Nanochat is an unparalleled educational tool. If you run a data science or AI engineering training program, this project allows your students or employees to understand every step of LLM construction. Rather than using black-box APIs, they can observe and modify the tokenizer, pre-training, fine-tuning, and RLHF stages. A training program could integrate nanochat as a one-week practical project, with a hardware budget of a few hundred dollars.

For a startup or SMB considering building a specialized conversational assistant, nanochat provides an ideal testing ground. Before investing tens of thousands of dollars in training a custom model, you can test your hypothesis for $100 to $300. The pipeline is modular enough to swap in your own dataset and observe the results.

If you are a CTO or VP of Engineering at a company considering developing its own model, nanochat gives you a concrete basis for estimating costs. By observing the relationship between model depth, training time, and performance, you can make informed extrapolations about the investment required for a production-quality model.

Autoresearch opens a new dimension. AI research teams can now let agents explore the hyperparameter and architecture space overnight or over the weekend. A research lab with a limited budget can multiply its exploration capacity by a significant factor. Karpathy's example (700 iterations in 2 days) illustrates a pace that no human team could sustain.

For executives who need to make strategic decisions about AI, nanochat offers valuable demystification. Understanding that chatbot training involves tokenization, pre-training, fine-tuning, and RLHF allows you to ask the right questions of technical teams and evaluate vendor proposals with an informed perspective.

It is essential to temper expectations. The $100 model is called "Kindergarten" for a good reason. It produces often incoherent responses, hallucinates regularly, and cannot be used in production. Even the $300 model, while surpassing GPT-2, remains far behind GPT-4, Claude, or Gemini in performance. We are talking about a 1.9 billion parameter model, while commercial models have hundreds of billions.

Nanochat is not designed to be deployed to end users. The lack of robust safety guardrails, the tendency to hallucinate, and limited performance make it a learning and research tool, not a product. If you need a chatbot for customer service or your application, look to existing APIs (OpenAI, Anthropic, Google) or mature open source models like Llama or Mistral.

Even though the cost is "only" $100, you need access to an 8xH100 GPU node, which runs about $24 per hour in cloud rental. This is not consumer hardware. Users need to be comfortable renting GPUs in the cloud (Lambda Labs, AWS, etc.) and working in a Linux environment.

Autoresearch is an impressive advancement, but it remains experimental. The agents find local optimizations, not necessarily fundamental breakthroughs. The reproducibility of some improvements is not yet guaranteed, and the system requires human oversight to validate changes before integration.

If you follow Karpathy's projects, you may know nanoGPT, his previous effort. The main difference lies in completeness. NanoGPT focused on pre-training a GPT-style model. Nanochat covers the entire pipeline, from the BPE tokenizer to the web interface, including fine-tuning and RLHF. It is the difference between building an engine and building an entire car.

Compared to commercial LLMs (ChatGPT, Claude, Gemini), the comparison does not really apply. These models were trained with budgets of tens of millions of dollars, on thousands of GPUs over months. Nanochat is a microscope for understanding how these models work, not a competitor.

You are a student or researcher in AI and want to understand the full LLM training pipeline. You are a developer looking to experiment with language model architecture. You are a CTO who needs to estimate costs and feasibility for an internal LLM project. You are a trainer looking for practical material to teach deep learning. You are passionate about AI and want to see from the inside how a ChatGPT works.

You are looking for a ready-to-use chatbot for your business. You have no experience in programming or machine learning. You need a reliable and secure model for end users. You do not have a budget for cloud GPU rental.

Nanochat alone would already be a remarkable project: the democratization of LLM training, made accessible for the price of a nice dinner. But it is autoresearch that marks a real turning point. The idea that AI agents can autonomously iterate on the training process, discover optimizations that humans missed, and organize their research like a structured team redefines how we think about artificial intelligence research.

With 42,900 stars on GitHub, an active community, and endorsement from figures like Shopify's CEO, nanochat has become much more than an educational project. It is an open laboratory where tomorrow's AI research methods are being invented. And at $100 for a ticket in, the barrier to entry has never been lower.