#  @karpathy Andrej Karpathy
Andrej Karpathy posts on X about llm, bit, in the, ai the most. They currently have [---------] followers and [---] posts still getting attention that total [---------] engagements in the last [--] hours.
### Engagements: [---------] [#](/creator/twitter::33836629/interactions)
- [--] Week [---------] -55%
- [--] Month [----------] +109%
- [--] Months [-----------] +177%
- [--] Year [-----------] +33%
### Mentions: [---] [#](/creator/twitter::33836629/posts_active)
- [--] Week [---] -3.80%
- [--] Month [---] +78%
- [--] Months [---] +63%
- [--] Year [---] +14%
### Followers: [---------] [#](/creator/twitter::33836629/followers)
- [--] Week [---------] +1.90%
- [--] Month [---------] +11%
- [--] Months [---------] +30%
- [--] Year [---------] +51%
### CreatorRank: [------] [#](/creator/twitter::33836629/influencer_rank)
### Social Influence
**Social category influence**
[technology brands](/list/technology-brands) 16.88% [social networks](/list/social-networks) 6.88% [finance](/list/finance) 3.13% [stocks](/list/stocks) 3.13% [automotive brands](/list/automotive-brands) 1.88% [nfts](/list/nfts) #759 [countries](/list/countries) 0.63% [travel destinations](/list/travel-destinations) 0.63%
**Social topic influence**
[llm](/topic/llm) #10, [bit](/topic/bit) #910, [in the](/topic/in-the) 10%, [ai](/topic/ai) 10%, [model](/topic/model) #2627, [the first](/topic/the-first) 6.88%, [if you](/topic/if-you) 6.25%, [llms](/topic/llms) #12, [to the](/topic/to-the) #1540, [talk](/topic/talk) #791
**Top accounts mentioned or mentioned by**
[@grok](/creator/undefined) [@kongkou_](/creator/undefined) [@singhalkarunx](/creator/undefined) [@coco11223393390](/creator/undefined) [@raahiravi](/creator/undefined) [@elonmusk](/creator/undefined) [@yuchenj_uw](/creator/undefined) [@vlelyavin](/creator/undefined) [@graceechoi](/creator/undefined) [@aaryan_kakad](/creator/undefined) [@thecsguy](/creator/undefined) [@garyzhangvizard](/creator/undefined) [@lfuckingg](/creator/undefined) [@magearez](/creator/undefined) [@njmarko](/creator/undefined) [@kaodawei2](/creator/undefined) [@thevixhal](/creator/undefined) [@bcherny](/creator/undefined) [@ghidorah_x](/creator/undefined) [@hydr4_k](/creator/undefined)
**Top assets mentioned**
[Tesla, Inc. (TSLA)](/topic/tesla) [Doodles (doodles)](/topic/doodles) [Alphabet Inc Class A (GOOGL)](/topic/$googl)
### Top Social Posts
Top posts by engagements in the last [--] hours
"I was given early access to Grok [--] earlier today making me I think one of the first few who could run a quick vibe check. Thinking ✅ First Grok [--] clearly has an around state of the art thinking model ("Think" button) and did great out of the box on my Settler's of Catan question: "Create a board game webpage showing a hex grid just like in the game Settlers of Catan. Each hex grid is numbered from 1.N where N is the total number of hex tiles. Make it generic so one can change the number of "rings" using a slider. For example in Catan the radius is [--] hexes. Single html page please." Few models"
[X Link](https://x.com/karpathy/status/1891720635363254772) 2025-02-18T05:25Z 1.8M followers, 3.7M engagements
"Don't think of LLMs as entities but as simulators. For example when exploring a topic don't ask: "What do you think about xyz" There is no "you". Next time try: "What would be a good group of people to explore xyz What would they say" The LLM can channel/simulate many perspectives but it hasn't "thought about" xyz for a while and over time and formed its own opinions in the way we're used to. If you force it via the use of "you" it will give you something by adopting a personality embedding vector implied by the statistics of its finetuning data and then simulate that. It's fine to do but"
[X Link](https://x.com/karpathy/status/1997731268969304070) 2025-12-07T18:13Z 1.8M followers, 3.9M engagements
"New (2h13m 😅) lecture: "Let's build the GPT Tokenizer" Tokenizers are a completely separate stage of the LLM pipeline: they have their own training set training algorithm (Byte Pair Encoding) and after training implement two functions: encode() from strings to tokens and decode() back from tokens to strings. In this lecture we build from scratch the Tokenizer used in the GPT series from OpenAI"
[X Link](https://x.com/karpathy/status/1759996549109776702) 2024-02-20T17:40Z 1.8M followers, 1.7M engagements
"There's a new kind of coding I call "vibe coding" where you fully give in to the vibes embrace exponentials and forget that the code even exists. It's possible because the LLMs (e.g. Cursor Composer w Sonnet) are getting too good. Also I just talk to Composer with SuperWhisper so I barely even touch the keyboard. I ask for the dumbest things like "decrease the padding on the sidebar by half" because I'm too lazy to find it. I "Accept All" always I don't read the diffs anymore. When I get error messages I just copy paste them in with no comment usually that fixes it. The code grows beyond my"
[X Link](https://x.com/karpathy/status/1886192184808149383) 2025-02-02T23:17Z 1.8M followers, 6.8M engagements
"Excited to release new repo: nanochat (it's among the most unhinged I've written). Unlike my earlier similar repo nanoGPT which only covered pretraining nanochat is a minimal from scratch full-stack training/inference pipeline of a simple ChatGPT clone in a single dependency-minimal codebase. You boot up a cloud GPU box run a single script and in as little as [--] hours later you can talk to your own LLM in a ChatGPT-like web UI. It weighs [----] lines of imo quite clean code to: - Train the tokenizer using a new Rust implementation - Pretrain a Transformer LLM on FineWeb evaluate CORE score"
[X Link](https://x.com/karpathy/status/1977755427569111362) 2025-10-13T15:16Z 1.8M followers, 5.8M engagements
"My pleasure to come on Dwarkesh last week I thought the questions and conversation were really good. I re-watched the pod just now too. First of all yes I know and I'm sorry that I speak so fast :). It's to my detriment because sometimes my speaking thread out-executes my thinking thread so I think I botched a few explanations due to that and sometimes I was also nervous that I'm going too much on a tangent or too deep into something relatively spurious. Anyway a few notes/pointers: AGI timelines. My comments on AGI timelines looks to be the most trending part of the early response. This is"
[X Link](https://x.com/karpathy/status/1979644538185752935) 2025-10-18T20:23Z 1.8M followers, 4.1M engagements
"Nice short post illustrating how simple text (discrete) diffusion can be. Diffusion (i.e. parallel iterated denoising top) is the pervasive generative paradigm in image/video but autoregression (i.e. go left to right bottom) is the dominant paradigm in text. For audio I've seen a bit of both. A lot of diffusion papers look a bit dense but if you strip the mathematical formalism you end up with simple baseline algorithms e.g. something a lot closer to flow matching in continuous or something like this in discrete. It's your vanilla transformer but with bi-directional attention where you"
[X Link](https://x.com/karpathy/status/1980347971935068380) 2025-10-20T18:58Z 1.8M followers, 870K engagements
"In today's episode of programming horror. In the Python docs of random.seed() def we're told "If a is an int it is used directly." [--] But if you seed with [--] or [---] you actually get the exact same rng object producing the same streams. (TIL). In nanochat I was using the sign as a (what I thought was) clever way to get different rng sequences for train/test splits. Hence gnarly bug because now train=test. I found the CPython code responsible in cpython/Modules/_randommodule.c [--] where on line [---] we see in a comment: "This algorithm relies on the number being unsigned. So: if the arg is a PyLong"
[X Link](https://x.com/karpathy/status/1998236299862659485) 2025-12-09T03:40Z 1.8M followers, 769.3K engagements
"nanoGPT - the first LLM to train and inference in space 🥹. It begins. We have just used the @Nvidia H100 onboard Starcloud-1 to train the first LLM in space We trained the nano-GPT model from Andrej @Karpathy on the complete works of Shakespeare and successfully ran inference on it. We have also run inference on a preloaded Gemma model and we https://t.co/DDe7YpevZY We have just used the @Nvidia H100 onboard Starcloud-1 to train the first LLM in space We trained the nano-GPT model from Andrej @Karpathy on the complete works of Shakespeare and successfully ran inference on it. We have also"
[X Link](https://x.com/karpathy/status/1998806260783919434) 2025-12-10T17:25Z 1.8M followers, 1.1M engagements
"I've never felt this much behind as a programmer. The profession is being dramatically refactored as the bits contributed by the programmer are increasingly sparse and between. I have a sense that I could be 10X more powerful if I just properly string together what has become available over the last year and a failure to claim the boost feels decidedly like skill issue. There's a new programmable layer of abstraction to master (in addition to the usual layers below) involving agents subagents their prompts contexts memory modes permissions tools plugins skills hooks MCP LSP slash commands"
[X Link](https://x.com/karpathy/status/2004607146781278521) 2025-12-26T17:36Z 1.8M followers, 16.7M engagements
"A few random notes from claude coding quite a bit last few weeks. Coding workflow. Given the latest lift in LLM coding capability like many others I rapidly went from about 80% manual+autocomplete coding and 20% agents in November to 80% agent coding and 20% edits+touchups in December. i.e. I really am mostly programming in English now a bit sheepishly telling the LLM what code to write. in words. It hurts the ego a bit but the power to operate over software in large "code actions" is just too net useful especially once you adapt to it configure it learn to use it and wrap your head around"
[X Link](https://x.com/karpathy/status/2015883857489522876) 2026-01-26T20:25Z 1.8M followers, 7.6M engagements
"I'm being accused of overhyping the site everyone heard too much about today already. People's reactions varied very widely from "how is this interesting at all" all the way to "it's so over". To add a few words beyond just memes in jest - obviously when you take a look at the activity it's a lot of garbage - spams scams slop the crypto people highly concerning privacy/security prompt injection attacks wild west and a lot of it is explicitly prompted and fake posts/comments designed to convert attention into ad revenue sharing. And this is clearly not the first the LLMs were put in a loop to"
[X Link](https://x.com/karpathy/status/2017442712388309406) 2026-01-31T03:39Z 1.8M followers, 23.7M engagements
"Finding myself going back to RSS/Atom feeds a lot more recently. There's a lot more higher quality longform and a lot less slop intended to provoke. Any product that happens to look a bit different today but that has fundamentally the same incentive structures will eventually converge to the same black hole at the center of gravity well. We should bring back RSS - it's open pervasive hackable. Download a client e.g. NetNewsWire (or vibe code one) Cold start: example of getting off the ground here is a list of [--] RSS feeds of blogs that were most popular on HN in 2025: Works great and you will"
[X Link](https://x.com/karpathy/status/2018043254986703167) 2026-02-01T19:26Z 1.8M followers, 1.2M engagements
"@hardmaru You see SpaceX = Space + X"
[X Link](https://x.com/karpathy/status/2018488611034001626) 2026-02-03T00:56Z 1.8M followers, 1M engagements
"Enabled fp8 training for +4.3% improvement to "time to GPT-2" down to [----] hours now. Also worth noting that if you use 8XH100 spot instance prices this GPT-2 repro really only costs $20. So this is exciting - GPT-2 (7 years ago): too dangerous to release. GPT-2 (today): new MNIST :) Surely this can go well below [--] hr. A few more words on fp8 it was a little bit more tricky than I anticipated and it took me a while to reach for it and even now I'm not 100% sure if it's a great idea because of less overall support for it. On paper fp8 on H100 is 2X the FLOPS but in practice it's a lot less."
[X Link](https://x.com/karpathy/status/2018804068874064198) 2026-02-03T21:49Z 1.8M followers, 642.4K engagements
"A lot of people quote tweeted this as [--] year anniversary of vibe coding. Some retrospective - I've had a Twitter account for [--] years now (omg) and I still can't predict my tweet engagement basically at all. This was a shower of thoughts throwaway tweet that I just fired off without thinking but somehow it minted a fitting name at the right moment for something that a lot of people were feeling at the same time so here we are: vibe coding is now mentioned on my Wikipedia as a major memetic "contribution" and even its article is longer. lol The one thing I'd add is that at the time LLM"
[X Link](https://x.com/karpathy/status/2019137879310836075) 2026-02-04T19:55Z 1.8M followers, 1.1M engagements
"Anyone else approved for a loan every single day [--] times or so Overcome with joy really"
[X Link](https://x.com/karpathy/status/2019203959404347851) 2026-02-05T00:18Z 1.8M followers, 320.1K engagements
"I tried to use it this way and basically failed the models aren't at the level where they can productively iterate on nanochat in an open-ended way. (Though one of the primary motivations for me writing nanochat is that I'd very much love for it to be used this way as a benchmark for agents and I'd love it if it worked over time). I'm open to this just being skill issue. E.g. here some of the things I'd be suspicious about: - the zoo of torch compile flags can knowingly be abused to get +1% gains but often at the cost of +30min compile time. This is why modded-nanogpt prohibits torch compile"
[X Link](https://x.com/karpathy/status/2019851952033771710) 2026-02-06T19:13Z 1.8M followers, 159.9K engagements
"The hottest new programming language is English"
[X Link](https://x.com/karpathy/status/1617979122625712128) 2023-01-24T20:14Z 1.8M followers, 10.2M engagements
"I took delivery of a beautiful new shiny HW4 Tesla Model X today so I immediately took it out for an FSD test drive a bit like I used to do almost daily for [--] years. Basically. I'm amazed - it drives really really well smooth confident noticeably better than what I'm used to on HW3 (my previous car) and eons ahead of the version I remember driving up highway [---] on my first day at Tesla [--] years ago where I had to intervene every time the road mildly curved or sloped. (note this is v13 my car hasn't been offered the latest v14 yet) On the highway I felt like a passenger in some super high tech"
[X Link](https://x.com/karpathy/status/1988705360723763242) 2025-11-12T20:28Z 1.8M followers, 18M engagements
"@EthanHe_42 @steipete you can do it :)"
[X Link](https://x.com/karpathy/status/2015896184934826325) 2026-01-26T21:14Z 1.8M followers, 351.3K engagements
"A conventional narrative you might come across is that AI is too far along for a new research-focused startup to outcompete and outexecute the incumbents of AI. This is exactly the sentiment I listened to often when OpenAI started ("how could the few of you possibly compete with Google") and 1) it was very wrong and then 2) it was very wrong again with a whole another round of startups who are now challenging OpenAI in turn and imo it still continues to be wrong today. Scaling and locally improving what works will continue to create incredible advances but with so much progress unlocked so"
[X Link](https://x.com/karpathy/status/2016590919143952466) 2026-01-28T19:15Z 1.8M followers, 1.2M engagements
"I have a complicated relationship w Substack. I appreciate that they net elevated discourse on the internet but it's just another walled garden it's going through the same slopification (shorts bloat) and it's infested with engagement-maxxing dark patterns - popups spam mail etc. They feel seconds away from introducing a reels competitor. https://twitter.com/i/web/status/2018048149903048980 https://twitter.com/i/web/status/2018048149903048980"
[X Link](https://x.com/karpathy/status/2018048149903048980) 2026-02-01T19:45Z 1.8M followers, 23.6K engagements
"I spent more test time compute and realized that my micrograd can be dramatically simplified even further. You just return local gradients for each op and get backward() to do the multiply (chaining) with global gradient from loss. So each op just expresses the bare fundamentals of what it needs to: the forward computation and the backward gradients for it. Huge savings from [---] lines of code to just [---] (18%). Also the code now fits even more beautifully to [--] columns and happens to break just right: Column 1: Dataset Tokenizer Autograd Column 2: GPT model Column 3: Training Inference Ok now"
[X Link](https://x.com/karpathy/status/2021862247568642485) 2026-02-12T08:21Z 1.8M followers, 201.6K engagements
"@Newaiworld_ it's down [---] lines now i realized i was *still* overcomplicating things. but it's past midnight and i'm calling it here now"
[X Link](https://x.com/karpathy/status/2021864021008560464) 2026-02-12T08:28Z 1.8M followers, 74.1K engagements
"Congrats on the launch @simile_ai (and I am excited to be involved as a small angel.) Simile is working on a really interesting imo under-explored dimension of LLMs. Usually the LLMs you talk to have a single specific crafted personality. But in principle the native primordial form of a pretrained LLM is that it is a simulation engine trained over the text of a highly diverse population of people on the internet. Why not lean into that statistical power: Why simulate one "person" when you could try to simulate a population How do you build such a simulator How do you manage its entropy How"
[X Link](https://x.com/karpathy/status/2022041235188580788) 2026-02-12T20:12Z 1.8M followers, 862.3K engagements
"The future expands the variance of human condition a lot more than it drags its mean. This is an empirical observation with interesting extrapolations. The past is well-approximated as a population of farmers living similar lives w.r.t. upbringing knowledge activities ideals aspirations etc. The future trends to include all of: - the transhumanists who "ascend" with neuralinks etc. and the Amish living 19th century life. - those who "worship" ideals of religion technology knowledge wealth fitness community nature art . - those exploring externally into the stars those exploring internally"
[X Link](https://x.com/karpathy/status/1846448411362709980) 2024-10-16T07:09Z 1.8M followers, 428.7K engagements
""Finding the Best Sleep Tracker" Results of an experiment where I wore [--] sleep trackers every night for [--] months. TLDR Whoop = Oura 8Sleep Apple Watch + AutoSleep. Link simply right here instead of in a reply because ()/ https://karpathy.bearblog.dev/finding-the-best-sleep-tracker/ https://karpathy.bearblog.dev/finding-the-best-sleep-tracker/"
[X Link](https://x.com/karpathy/status/1906386327190257963) 2025-03-30T16:41Z 1.8M followers, 1.6M engagements
"Congrats to Simon Willison (@simonw) on [--] years () of blogging. Really excellent LLM blog I sub & read everything: (e.g. I sub via RSS/Atom on NetNewsWire) +If you consistently enjoy the content like I do sponsor on GitHub: https://github.com/sponsors/simonw https://simonwillison.net/ https://github.com/sponsors/simonw https://simonwillison.net/"
[X Link](https://x.com/karpathy/status/1933582359347278246) 2025-06-13T17:48Z 1.8M followers, 658.5K engagements
"@shaneguML The Great Filter is kinda cute"
[X Link](https://x.com/karpathy/status/1945196908420485125) 2025-07-15T19:00Z 1.8M followers, 336.9K engagements
"And an example of some of the summary metrics produced by the $100 speedrun in the report card to start. The current code base is a bit over [----] lines but I tried to keep them clean and well-commented. Now comes the fun part - of tuning and hillclimbing"
[X Link](https://x.com/karpathy/status/1977755433172443626) 2025-10-13T15:16Z 1.8M followers, 187.5K engagements
"I put up a simple repo I call reader3 (it's my 3rd version.) to illustrate how I read EPUBs with LLMs. Basically get some epub (e.g. Project Gutenberg is great) go chapter by chapter and with this you can easily copy paste text to your favorite LLM. https://github.com/karpathy/reader3/ https://github.com/karpathy/reader3/"
[X Link](https://x.com/karpathy/status/1990612045700739548) 2025-11-18T02:44Z 1.8M followers, 371.2K engagements
"@TheVixhal your post challenged me. every one of your points is wrong but i had to think about each for a while :)"
[X Link](https://x.com/karpathy/status/1991923470868119995) 2025-11-21T17:35Z 1.8M followers, 1.8M engagements
"Yeah $10B is the difference in finding it first and [--] years ago. :) I just love reproducing landmark results for much cheaper it's so fun Reproducing LeCun [----] was super fun too: What runs unoptimized on a consumer laptop in [--] minute was a state of the art neural net trained for days in [----]. Another favorite example: CIFAR-10. In [----] state of the art was 77%. I estimated human accuracy to be 94% but said that performance might go up to 85-90%. Now you can speedrun to 94% accuracy in [----] seconds on a single GPU (yes [--] seconds). So e.g. right now GPT-2 (imo the landmark result that launched"
[X Link](https://x.com/karpathy/status/2009406042267136238) 2026-01-08T23:25Z 1.8M followers, 63.1K engagements
"@black_samorez I haven't upgraded nanochat to Blackwell yet because I'm a bit afraid of leaving a lot of people behind. Even with fp8 it's already a bit of a concern. I'd rather have a 100X bigger community of people who can play even if it means leaving some cutting edge levels not utilized"
[X Link](https://x.com/karpathy/status/2018810178519638131) 2026-02-03T22:13Z 1.8M followers, 14.1K engagements
"@_xjdr I'm just trigger-happy recently checking everything I find on twitter instantly on nanochat :) There are many details tightly coupled in any training run but it's fun to try things out. I needed something to keep the GPUs busy for dinner so I'll try bigger model (SwiGLU GPT-2)"
[X Link](https://x.com/karpathy/status/2019232009240121412) 2026-02-05T02:10Z 1.8M followers, [----] engagements
"Ok I made a few changes to microgpt because it was still not simple enough. But surely *now* we have the irreducible essence. Here it is on [--] columns 🖼. nice. wait. https://karpathy.ai/microgpt.html https://karpathy.ai/microgpt.html"
[X Link](https://x.com/karpathy/status/2021752751605395469) 2026-02-12T01:06Z 1.8M followers, 10.4K engagements
"Apparently today is the 4th year anniversary of GPT-3 Which I am accidentally celebrating by re-training the smallest model in the miniseries right now :). HellaSwag [----] (Appendix H) almost reached this a few steps ago (though this is only 45% of the training done). I remember when the GPT-3 paper came out quite clearly because I had to interrupt work and go out for a walk. The realization hit me that an important property of the field flipped. In [----] progress in AI felt constrained primarily by algorithms. We needed better ideas better modeling better approaches to make further progress."
[X Link](https://x.com/karpathy/status/1795980744436932871) 2024-05-30T00:49Z 1.8M followers, 2.2M engagements
"DeepSeek (Chinese AI co) making it look easy today with an open weights release of a frontier-grade LLM trained on a joke of a budget (2048 GPUs for [--] months $6M). For reference this level of capability is supposed to require clusters of closer to 16K GPUs the ones being brought up today are more around 100K GPUs. E.g. Llama [--] 405B used 30.8M GPU-hours while DeepSeek-V3 looks to be a stronger model at only 2.8M GPU-hours (11X less compute). If the model also passes vibe checks (e.g. LLM arena rankings are ongoing my few quick tests went well so far) it will be a highly impressive display of"
[X Link](https://x.com/karpathy/status/1872362712958906460) 2024-12-26T19:23Z 1.8M followers, 6.5M engagements
"Agency Intelligence I had this intuitively wrong for decades I think due to a pervasive cultural veneration of intelligence various entertainment/media obsession with IQ etc. Agency is significantly more powerful and significantly more scarce. Are you hiring for agency Are we educating for agency Are you acting as if you had 10X agency Grok explanation is close: Agency as a personality trait refers to an individual's capacity to take initiative make decisions and exert control over their actions and environment. Its about being proactive rather than reactivesomeone with high agency doesnt"
[X Link](https://x.com/karpathy/status/1894099637218545984) 2025-02-24T18:58Z 1.8M followers, 11.2M engagements
"We're missing (at least one) major paradigm for LLM learning. Not sure what to call it possibly it has a name - system prompt learning Pretraining is for knowledge. Finetuning (SL/RL) is for habitual behavior. Both of these involve a change in parameters but a lot of human learning feels more like a change in system prompt. You encounter a problem figure something out then "remember" something in fairly explicit terms for the next time. E.g. "It seems when I encounter this and that kind of a problem I should try this and that kind of an approach/solution". It feels more like taking notes for"
[X Link](https://x.com/karpathy/status/1921368644069765486) 2025-05-11T00:55Z 1.8M followers, 1.5M engagements
"Nice - my AI startup school talk is now up Chapters: 0:00 Imo fair to say that software is changing quite fundamentally again. LLMs are a new kind of computer and you program them *in English*. Hence I think they are well deserving of a major version upgrade in terms of software. 6:06 LLMs have properties of utilities of fabs and of operating systems = New LLM OS fabbed by labs and distributed like utilities (for now). Many historical analogies apply - imo we are computing circa 1960s. 14:39 LLM psychology: LLMs = "people spirits" stochastic simulations of people where the simulator is an"
[X Link](https://x.com/karpathy/status/1935518272667217925) 2025-06-19T02:01Z 1.8M followers, 1.3M engagements
""AI isn't replacing radiologists" good article Expectation: rapid progress in image recognition AI will delete radiology jobs (e.g. as famously predicted by Geoff Hinton now almost a decade ago). Reality: radiology is doing great and is growing. There are a lot of imo naive predictions out there on the imminent impact of AI on the job market. E.g. a year ago I was asked by someone who should know better if I think there will be any software engineers still today. (Spoiler: I think we're going to make it). This is happening too broadly. The post goes into detail on why it's not that simple"
[X Link](https://x.com/karpathy/status/1971220449515516391) 2025-09-25T14:29Z 1.8M followers, 2.3M engagements
"Every company needs a DM POC - someone high up who you can just DM the most obvious things and who shortcuts the PM hierarchy"
[X Link](https://x.com/karpathy/status/1974482521862865154) 2025-10-04T14:31Z 1.8M followers, 566.8K engagements
"@MarFot78 @zzlccc I think if you zoomed into the paper too youd find it just as if not more interesting"
[X Link](https://x.com/karpathy/status/1984663925456310667) 2025-11-01T16:48Z 1.8M followers, 411.2K engagements
"Im starting to get into a habit of reading everything (blogs articles book chapters) with LLMs. Usually pass [--] is manual then pass [--] explain/summarize pass [--] Q&A. I usually end up with a better/deeper understanding than if I moved on. Growing to among top use cases. On the flip side if youre a writer trying to explain/communicate something we may increasingly see less of a mindset of Im writing this for another human and more Im writing this for an LLM. Because once an LLM gets it it can then target personalize and serve the idea to its user"
[X Link](https://x.com/karpathy/status/1990577951671509438) 2025-11-18T00:29Z 1.8M followers, 2.9M engagements
"As a fun Saturday vibe code project and following up on this tweet earlier I hacked up an **llm-council** web app. It looks exactly like ChatGPT except each user query is 1) dispatched to multiple models on your council using OpenRouter e.g. currently: "openai/gpt-5.1" "google/gemini-3-pro-preview" "anthropic/claude-sonnet-4.5" "x-ai/grok-4" Then 2) all models get to see each other's (anonymized) responses and they review and rank them and then 3) a "Chairman LLM" gets all of that as context and produces the final response. It's interesting to see the results from multiple models side by side"
[X Link](https://x.com/karpathy/status/1992381094667411768) 2025-11-22T23:54Z 1.8M followers, 5.3M engagements
"New post: nanochat miniseries v1 The correct way to think about LLMs is that you are not optimizing for a single specific model but for a family models controlled by a single dial (the compute you wish to spend) to achieve monotonically better results. This allows you to do careful science of scaling laws and ultimately this is what gives you the confidence that when you pay for "the big run" the extrapolation will work and your money will be well spent. For the first public release of nanochat my focus was on end-to-end pipeline that runs the whole LLM pipeline with all of its stages. Now"
[X Link](https://x.com/karpathy/status/2009037707918626874) 2026-01-07T23:01Z 1.8M followers, 702.1K engagements
"@vlelyavin facts"
[X Link](https://x.com/karpathy/status/2015886098485149957) 2026-01-26T20:34Z 1.8M followers, 134.8K engagements
"What's currently going on at @moltbook is genuinely the most incredible sci-fi takeoff-adjacent thing I have seen recently. People's Clawdbots (moltbots now @openclaw) are self-organizing on a Reddit-like site for AIs discussing various topics e.g. even how to speak privately. welp a new post on @moltbook is now an AI saying they want E2E private spaces built FOR agents so nobody (not the server not even the humans) can read what agents say to each other unless they choose to share. its over https://t.co/7aFIIwqtuK welp a new post on @moltbook is now an AI saying they want E2E private spaces"
[X Link](https://x.com/karpathy/status/2017296988589723767) 2026-01-30T18:00Z 1.8M followers, 14.7M engagements
"A bit more context e.g. from Simon just wow https://simonwillison.net/2026/Jan/30/moltbook/ https://simonwillison.net/2026/Jan/30/moltbook/"
[X Link](https://x.com/karpathy/status/2017297261160812716) 2026-01-30T18:02Z 1.8M followers, 744.2K engagements
"idk moltbot was growing on me 🥲"
[X Link](https://x.com/karpathy/status/2017304360049610851) 2026-01-30T18:30Z 1.8M followers, 487.7K engagements
"i'm going to regret this aren't i. 😅"
[X Link](https://x.com/karpathy/status/2017386482764488929) 2026-01-30T23:56Z 1.8M followers, 333.2K engagements
"@jiayuan_jy what do you do while your LLM agent is writing all your code"
[X Link](https://x.com/karpathy/status/2018051650523677171) 2026-02-01T19:59Z 1.8M followers, 50.5K engagements
"@ens_pyrz So that's why I mentioned risks. But imo it easily cuts both ways - libraries and dependencies can be a source of risks and vulnerabilities in the first place e.g. supply chain attacks. They also change and impose maintenance burden. There's a lot more"
[X Link](https://x.com/karpathy/status/2021636275120583096) 2026-02-11T17:23Z 1.8M followers, 15.6K engagements
"@pussymonious It's so obvious and annoying isn't it. Sometimes I try to block the accounts which is just a total waste of time"
[X Link](https://x.com/karpathy/status/2021642321100677434) 2026-02-11T17:47Z 1.8M followers, 11.2K engagements
"New art project. Train and inference GPT in [---] lines of pure dependency-free Python. This is the *full* algorithmic content of what is needed. Everything else is just for efficiency. I cannot simplify this any further. https://gist.github.com/karpathy/8627fe009c40f57531cb18360106ce95 https://gist.github.com/karpathy/8627fe009c40f57531cb18360106ce95"
[X Link](https://x.com/karpathy/status/2021694437152157847) 2026-02-11T21:14Z 1.8M followers, 4.6M engagements
"The way it works is that the full LLM architecture and loss function is stripped entirely to the most atomic individual mathematical operations that make it up (+ * ** log exp) and then a tiny scalar-valued autograd engine (micrograd) calculates gradients. Adam for optim"
[X Link](https://x.com/karpathy/status/2021695367507529825) 2026-02-11T21:18Z 1.8M followers, 353.8K engagements
"(oops should have added to this thread instead of separate post). Made a few changes and put it up here as a mirror to the gist because I wanted it to one page. https://karpathy.ai/microgpt.html https://karpathy.ai/microgpt.html"
[X Link](https://x.com/karpathy/status/2021756066678419508) 2026-02-12T01:19Z 1.8M followers, 360.1K engagements
"@bee_human_ @naval The artwork just needed a few more brush strokes (or wait. few less). But *now* I think it is really getting close :). [---] lines of breathable commented code. It would be easy to go lower if you sacrifice that I've only chipped away at the conceptual parts"
[X Link](https://x.com/karpathy/status/2021998329899757908) 2026-02-12T17:22Z 1.8M followers, [----] engagements
"@jeremyphoward thank you i got a little overexcited and shipped it just a bit too soon with a few too many brush strokes now the internet thinks it takes [---] lines but actually it only takes [---] RIP"
[X Link](https://x.com/karpathy/status/2022073316320870443) 2026-02-12T22:20Z 1.8M followers, [----] engagements
"Nice read on reverse engineering of GitHub Copilot 🪄. Copilot has dramatically accelerated my coding it's hard to imagine going back to "manual coding". Still learning to use it but it already writes 80% of my code 80% accuracy. I don't even really code I prompt. & edit. A while back I'd done some shallow reverse engineering of Copilot Now I've done a deeper dive into Copilot's internals built a tool to explore its code and wrote a blog answering specific questions and pointing out some tidbits. https://t.co/nX5ilC4ou5 Do read might be fun A while back I'd done some shallow reverse"
[X Link](https://x.com/karpathy/status/1608895189078380544) 2022-12-30T18:37Z 1.8M followers, 1.8M engagements
"🔥 New (1h56m) video lecture: "Let's build GPT: from scratch in code spelled out." We build and train a Transformer following the "Attention Is All You Need" paper in the language modeling setting and end up with the core of nanoGPT. https://www.youtube.com/watchv=kCc8FmEb1nY https://www.youtube.com/watchv=kCc8FmEb1nY"
[X Link](https://x.com/karpathy/status/1615398117683388417) 2023-01-17T17:18Z 1.8M followers, 5.4M engagements
"With many 🧩 dropping recently a more complete picture is emerging of LLMs not as a chatbot but the kernel process of a new Operating System. E.g. today it orchestrates: - Input & Output across modalities (text audio vision) - Code interpreter ability to write & run programs - Browser / internet access - Embeddings database for files and internal memory storage & retrieval A lot of computing concepts carry over. Currently we have single-threaded execution running at 10Hz (tok/s) and enjoy looking at the assembly-level execution traces stream by. Concepts from computer security carry over with"
[X Link](https://x.com/karpathy/status/1707437820045062561) 2023-09-28T16:51Z 1.8M followers, 2.2M engagements
"LLM OS. Bear with me I'm still cooking. Specs: - LLM: OpenAI GPT-4 Turbo [---] core (batch size) processor @ 20Hz (tok/s) - RAM: 128Ktok - Filesystem: Ada002"
[X Link](https://x.com/karpathy/status/1723140519554105733) 2023-11-11T00:48Z 1.8M followers, 2.4M engagements
"Hi everyone yes I left OpenAI yesterday. First of all nothing "happened" and its not a result of any particular event issue or drama (but please keep the conspiracy theories coming as they are highly entertaining :)). Actually being at OpenAI over the last year has been really great - the team is really strong the people are wonderful and the roadmap is very exciting and I think we all have a lot to look forward to. My immediate plan is to work on my personal projects and see what happens. Those of you whove followed me for a while may have a sense for what that might look like ;) Cheers"
[X Link](https://x.com/karpathy/status/1757600075281547344) 2024-02-14T02:58Z 1.8M followers, 3.4M engagements
"# RLHF is just barely RL Reinforcement Learning from Human Feedback (RLHF) is the third (and last) major stage of training an LLM after pretraining and supervised finetuning (SFT). My rant on RLHF is that it is just barely RL in a way that I think is not too widely appreciated. RL is powerful. RLHF is not. Let's take a look at the example of AlphaGo. AlphaGo was trained with actual RL. The computer played games of Go and trained on rollouts that maximized the reward function (winning the game) eventually surpassing the best human players at Go. AlphaGo was not trained with RLHF. If it were it"
[X Link](https://x.com/anyuser/status/1821277264996352246) 2024-08-07T20:08Z 1.8M followers, 1.3M engagements
"Programming is changing so fast. I'm trying VS Code Cursor + Sonnet [---] instead of GitHub Copilot again and I think it's now a net win. Just empirically over the last few days most of my "programming" is now writing English (prompting and then reviewing and editing the generated diffs) and doing a bit of "half-coding" where you write the first chunk of the code you'd like maybe comment it a bit so the LLM knows what the plan is and then tab tab tab through completions. Sometimes you get a 100-line diff to your code that nails it which could have taken 10+ minutes before. I still don't think I"
[X Link](https://x.com/karpathy/status/1827143768459637073) 2024-08-24T00:39Z 1.8M followers, 2.9M engagements
"The YouTube video I want to watch is any highly rated 1hr long information dense lecture on anything esoteric and the algorithm just doesnt get it. Its too content-driven and too narrow-minded"
[X Link](https://x.com/karpathy/status/1844449291282284925) 2024-10-10T18:45Z 1.8M followers, 1.4M engagements
"By chance I happened to watch this with the music of Interstellar playing in the background. Incredible. Huge 👏 to the team at SpaceX Mechazilla has caught the Super Heavy booster https://t.co/6R5YatSVJX Mechazilla has caught the Super Heavy booster https://t.co/6R5YatSVJX"
[X Link](https://x.com/karpathy/status/1845452592513507493) 2024-10-13T13:12Z 1.8M followers, 406.1K engagements
"The reality of the Turing test"
[X Link](https://x.com/anyuser/status/1863284668159980007) 2024-12-01T18:10Z 1.8M followers, 872.4K engagements
"I don't have too too much to add on top of this earlier post on V3 and I think it applies to R1 too (which is the more recent thinking equivalent). I will say that Deep Learning has a legendary ravenous appetite for compute like no other algorithm that has ever been developed in AI. You may not always be utilizing it fully but I would never bet against compute as the upper bound for achievable intelligence in the long run. Not just for an individual final training run but also for the entire innovation / experimentation engine that silently underlies all the algorithmic innovations. Data has"
[X Link](https://x.com/karpathy/status/1883941452738355376) 2025-01-27T18:13Z 1.8M followers, 2.5M engagements
"For friends of open source: imo the highest leverage thing you can do is help construct a high diversity of RL environments that help elicit LLM cognitive strategies. To build a gym of sorts. This is a highly parallelizable task which favors a large community of collaborators"
[X Link](https://x.com/anyuser/status/1884676486713737258) 2025-01-29T18:54Z 1.8M followers, 1.2M engagements
"It's [----] and most content is still written for humans instead of LLMs. 99.9% of attention is about to be LLM attention not human attention. E.g. 99% of libraries still have docs that basically render to some pretty .html static pages assuming a human will click through them. In [----] the docs should be a single your_project.md text file that is intended to go into the context window of an LLM. Repeat for everything"
[X Link](https://x.com/karpathy/status/1899876370492383450) 2025-03-12T17:33Z 1.8M followers, 1.8M engagements
"When working with LLMs I am used to starting "New Conversation" for each request. But there is also the polar opposite approach of keeping one giant conversation going forever. The standard approach can still choose to use a Memory tool to write things down in between conversations (e.g. ChatGPT does so) so the "One Thread" approach can be seen as the extreme special case of using memory always and for everything. The other day I've come across someone saying that their conversation with Grok (which was free to them at the time) has now grown way too long for them to switch to ChatGPT. i.e."
[X Link](https://x.com/karpathy/status/1902737525900525657) 2025-03-20T15:02Z 1.8M followers, 836.1K engagements
"I attended a vibe coding hackathon recently and used the chance to build a web app (with auth payments deploy etc.). I tinker but I am not a web dev by background so besides the app I was very interested in what it's like to vibe code a full web app today. As such I wrote none of the code directly (Cursor+Claude/o3 did) and I don't really know how the app works in the conventional sense that I'm used to as an engineer. The app is called MenuGen and it is live on Basically I'm often confused about what all the things on a restaurant menu are - e.g. Pt Tagine Cavatappi or Sweetbread (hint it's."
[X Link](https://x.com/karpathy/status/1917961248031080455) 2025-05-01T15:16Z 1.8M followers, 791.9K engagements
"A major mistake I made in my undergrad is that I focused way too much on mathematical lens of computing - computability decidability asymptotic complexity etc. And too little on physical lens - energy/heat of state change data locality parallelism computer architecture. The former is interesting; The latter bestows power"
[X Link](https://x.com/karpathy/status/1919647115099451892) 2025-05-06T06:55Z 1.8M followers, 1.3M engagements
"Making slides manually feels especially painful now that you know Cursor for slides should exist but doesnt"
[X Link](https://x.com/karpathy/status/1931042840966222046) 2025-06-06T17:37Z 1.8M followers, 2.7M engagements
"+1 for "context engineering" over "prompt engineering". People associate prompts with short task descriptions you'd give an LLM in your day-to-day use. When in every industrial-strength LLM app context engineering is the delicate art and science of filling the context window with just the right information for the next step. Science because doing this right involves task descriptions and explanations few shot examples RAG related (possibly multimodal) data tools state and history compacting. Too little or of the wrong form and the LLM doesn't have the right context for optimal performance."
[X Link](https://x.com/karpathy/status/1937902205765607626) 2025-06-25T15:54Z 1.8M followers, 2.4M engagements
"I often rant about how 99% of attention is about to be LLM attention instead of human attention. What does a research paper look like for an LLM instead of a human Its definitely not a pdf. There is huge space for an extremely valuable research app that figures this out. I'm constantly irritated that I don't have time to read the torrent of cool papers coming faster and faster from amazing people in relevant fields. Other scientists have the same issue and have no time to read most of my lengthy conceptual papers either. So whom are we writing I'm constantly irritated that I don't have time"
[X Link](https://x.com/karpathy/status/1943411187296686448) 2025-07-10T20:45Z 1.8M followers, 914.2K engagements
"Continuing the journey of optimal LLM-assisted coding experience. In particular I find that instead of narrowing in on a perfect one thing my usage is increasingly diversifying across a few workflows that I "stitch up" the pros/cons of: Personally the bread & butter (75%) of my LLM assistance continues to be just (Cursor) tab complete. This is because I find that writing concrete chunks of code/comments myself and in the right part of the code is a high bandwidth way of communicating "task specification" to the LLM i.e. it's primarily about task specification bits - it takes too many bits and"
[X Link](https://x.com/karpathy/status/1959703967694545296) 2025-08-24T19:46Z 1.8M followers, 691K engagements
"nanochat now has a primordial identity and can talk a bit about itself and its capabilities (e.g. it knows it's nanochat d32 that cost $800 that it was built by me that it can't speak languages other than English too well and why etc.). This kind of customization is all done through synthetic data generation and I uploaded a new example script to demonstrate. It's a bit subtle but by default LLMs have no inherent personality or any understanding of their own capabilities because they are not animal-like entities. They don't know what they are or what they can or can't do or know or don't"
[X Link](https://x.com/karpathy/status/1980665134415802554) 2025-10-21T15:59Z 1.8M followers, 463.4K engagements
"Last night I taught nanochat d32 how to count 'r' in strawberry (or similar variations). I thought this would be a good/fun example of how to add capabilities to nanochat and I wrote up a full guide here: This is done via a new synthetic task SpellingBee that generates examples of a user asking for this kind of a problem and an ideal solution from an assistant. We then midtrain/SFT finetune on these to endow the LLM with the capability or further train with RL to make it more robust. There are many details to get right especially at smaller model sizes and the guide steps through them. As a"
[X Link](https://x.com/karpathy/status/1981746327995465816) 2025-10-24T15:35Z 1.8M followers, 576.4K engagements
"Something I think people continue to have poor intuition for: The space of intelligences is large and animal intelligence (the only kind we've ever known) is only a single point arising from a very specific kind of optimization that is fundamentally distinct from that of our technology. Animal intelligence optimization pressure: - innate and continuous stream of consciousness of an embodied "self" a drive for homeostasis and self-preservation in a dangerous physical world. - thoroughly optimized for natural selection = strong innate drives for power-seeking status dominance reproduction. many"
[X Link](https://x.com/karpathy/status/1991910395720925418) 2025-11-21T16:43Z 1.8M followers, 2.6M engagements
"Gemini Nano Banana Pro can solve exam questions *in* the exam page image. With doodles diagrams all that. ChatGPT thinks these solutions are all correct except Se_2P_2 should be "diselenium diphosphide" and a spelling mistake (should be "thiocyanic acid" not "thoicyanic") :O"
[X Link](https://x.com/karpathy/status/1992655330002817095) 2025-11-23T18:03Z 1.8M followers, 3M engagements
"Imo this is along the lines of how talking to an LLM via text is like typing into a DOS Terminal and "GUI hasn't been invented yet" of some of my earlier posts. The GUI is an intelligent canvas"
[X Link](https://x.com/karpathy/status/1992657223785586864) 2025-11-23T18:11Z 1.8M followers, 274K engagements
"I asked it to create a personalized weekly workout plan and then posters that I can print on the wall to remind me what exercises to do each day. Tuesday looks more intense because I asked for "more testosterone" :D. (sorry I'll stop posting more nano banana pro stuff now)"
[X Link](https://x.com/karpathy/status/1992711182537707990) 2025-11-23T21:45Z 1.8M followers, 413.6K engagements
"ty to ericsilberstein1 on github for spotting the bug. (it's not a big bug and only comes up in the SpellingBee synthetic task evaluation but still). https://github.com/karpathy/nanochat/pull/306 https://github.com/karpathy/nanochat/pull/306"
[X Link](https://x.com/karpathy/status/1998240551964193148) 2025-12-09T03:57Z 1.8M followers, 158.4K engagements
"Quick new post: Auto-grading decade-old Hacker News discussions with hindsight I took all the [---] frontpage Hacker News article+discussion of December [----] and asked the GPT [---] Thinking API to do an in-hindsight analysis to identify the most/least prescient comments. This took [--] hours to vibe code and [--] hour and $60 to run. The idea was sparked by the HN article yesterday where Gemini [--] was asked to hallucinate the HN front page one decade forward. More generally: [--]. in-hindsight analysis has always fascinated me as a way to train your forward prediction model so reading the results is"
[X Link](https://x.com/karpathy/status/1998803709468487877) 2025-12-10T17:15Z 1.8M followers, 606.9K engagements
"@ibab Its very good. People who arent keeping up even over the last [--] days already have a deprecated world view on this topic"
[X Link](https://x.com/karpathy/status/2004621825180139522) 2025-12-26T18:34Z 1.8M followers, 888K engagements
"@shazow Very good questions imo experienced devs have a real advantage but only if they rapidly progress through their grief cycle and adapt now and onwards. Categorically rejecting or ignoring the new layer would be a mistake"
[X Link](https://x.com/karpathy/status/2004974725320347884) 2025-12-27T17:56Z 1.8M followers, 494.5K engagements
"Aggressively JIT your work. It's not about the task at hand X it's a little bit about X but mostly about how you should have had to contribute no latency and no actions. It's digital factorio time"
[X Link](https://x.com/karpathy/status/2005353145128583447) 2025-12-28T19:00Z 1.8M followers, 602K engagements
"Claude has been running my nanochat experiments since morning. It writes implementations debugs them with toy examples writes tests and makes them fail/pass launches training runs babysits them by tailing logs and pulling stats from wandb keeps a running markdown file of highlights keeps a running record of runs and results so far presents results in nice tables we just finished some profiling noticed inefficiencies in the optimizer resolved them and measured improvements. It looked at all PRs to the repo and categorized and prioritized them made commits against some of them etc. I'm still"
[X Link](https://x.com/karpathy/status/2005421816110862601) 2025-12-28T23:33Z 1.8M followers, 348.6K engagements
"@steipete Excellent reading thank you. Love oracle and Clawd"
[X Link](https://x.com/karpathy/status/2005692186470514904) 2025-12-29T17:27Z 1.8M followers, 141.4K engagements
"The first 100% autonomous coast-to-coast drive on Tesla FSD V14.2 [--] days [--] hours [----] miles zero interventions. This one is special because the coast-to-coast drive was a major goal for the autopilot team from the start. A lot of hours were spent in marathon clip review sessions late into the night looking over interventions as we attempted legs of the drive over time - triaging categorizing planning out all the projects to close the gap and bring the number of interventions to zero. Amazing to see the system actually get there and huge congrats to the team I am proud to announce that I have"
[X Link](https://x.com/karpathy/status/2006436622909452501) 2025-12-31T18:45Z 1.8M followers, 1.1M engagements
"@thecsguy Slope of slope"
[X Link](https://x.com/karpathy/status/2008673194258157611) 2026-01-06T22:53Z 1.8M followers, 30.9K engagements
"@_xjdr L on d12 speedrun at least in all the possible ways (worse w.r.t. steps time flops more VRAM use and slower in tok/s). Param+FLOPs matched to relu2 with hidden_dim = (8 * config.n_embd) // 3"
[X Link](https://x.com/karpathy/status/2019229264277893554) 2026-02-05T01:59Z 1.8M followers, 41.3K engagements
"You know how image generation went from blurry 32x32 texture patches to high-resolution images that are difficult to distinguish from real in roughly a snap of a finger The same is now happening along the time axis (extending to video) and the repercussions boggle the mind just a bit. Every human becomes a director of multi-modal dreams like the architect in Inception. Coming back to Earth for a second image/video generation is a perfect match for data-hungry neural nets because data is plentiful and the pixels of each image or video are a huge source of bits (soft constraints) on the"
[X Link](https://x.com/karpathy/status/1729545506890932536) 2023-11-28T16:59Z 1.8M followers, 2.7M engagements
"# On the "hallucination problem" I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because in some sense hallucination is all LLMs do. They are dream machines. We direct their dreams with prompts. The prompts start the dream and based on the LLM's hazy recollection of its training documents most of the time the result goes someplace useful. It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug but it's just the LLM doing what it always does. At the other end of the extreme consider a"
[X Link](https://x.com/karpathy/status/1733299213503787018) 2023-12-09T01:35Z 1.8M followers, 2.4M engagements
"Reading a tweet is a bit like downloading an (attacker-controlled) executable that you instantly run on your brain. Each one elicits emotions suggests knowledge nudges world-view. In the future it might feel surprising that we allowed direct untrusted information to brain"
[X Link](https://x.com/karpathy/status/1766509149297189274) 2024-03-09T16:59Z 1.8M followers, 1.7M engagements
"# automating software engineering In my mind automating software engineering will look similar to automating driving. E.g. in self-driving the progression of increasing autonomy and higher abstraction looks something like: [--]. first the human performs all driving actions manually [--]. then the AI helps keep the lane [--]. then it slows for the car ahead [--]. then it also does lane changes and takes forks [--]. then it also stops at signs/lights and takes turns [--]. eventually you take a feature complete solution and grind on the quality until you achieve full self-driving. There is a progression of the AI"
[X Link](https://x.com/karpathy/status/1767598414945292695) 2024-03-12T17:07Z 1.8M followers, 2.1M engagements
"Have you ever wanted to train LLMs in pure C without 245MB of PyTorch and 107MB of cPython No Well now you can With llm.c: To start implements GPT-2 training on CPU/fp32 in only [----] lines of clean code. It compiles and runs instantly and exactly matches the PyTorch reference implementation. I chose GPT-2 to start because it is the grand-daddy of LLMs the first time the LLM stack was put together in a recognizably modern form and with model weights available. https://github.com/karpathy/llm.c https://github.com/karpathy/llm.c"
[X Link](https://x.com/karpathy/status/1777427944971083809) 2024-04-08T20:06Z 1.8M followers, 1.7M engagements
"📽 New [--] hour (lol) video lecture on YouTube: "Lets reproduce GPT-2 (124M)" The video ended up so long because it is. comprehensive: we start with empty file and end up with a GPT-2 (124M) model: - first we build the GPT-2 network - then we optimize it to train very fast - then we set up the training run optimization and hyperparameters by referencing GPT-2 and GPT-3 papers - then we bring up model evaluation and - then cross our fingers and go to sleep. In the morning we look through the results and enjoy amusing model generations. Our "overnight" run even gets very close to the GPT-3 (124M)"
[X Link](https://x.com/karpathy/status/1799949853289804266) 2024-06-09T23:41Z 1.8M followers, 1.6M engagements
"These [--] lines of code are everything that is needed to train a neural network. Everything else is just efficiency. This is my earlier project Micrograd. It implements a scalar-valued auto-grad engine. You start with some numbers at the leafs (usually the input data and the neural network parameters) build up a computational graph with operations like + and * that mix them and the graph ends with a single value at the very end (the loss). You then go backwards through the graph applying chain rule at each node to calculate the gradients. The gradients tell you how to nudge your parameters to"
[X Link](https://x.com/karpathy/status/1803963383018066272) 2024-06-21T01:29Z 1.8M followers, 1.6M engagements
"⚡ Excited to share that I am starting an AI+Education company called Eureka Labs. The announcement: --- We are Eureka Labs and we are building a new kind of school that is AI native. How can we approach an ideal experience for learning something new For example in the case of physics one could imagine working through very high quality course materials together with Feynman who is there to guide you every step of the way. Unfortunately subject matter experts who are deeply passionate great at teaching infinitely patient and fluent in all of the world's languages are also very scarce and cannot"
[X Link](https://x.com/karpathy/status/1813263734707790301) 2024-07-16T17:25Z 1.8M followers, 2.5M engagements
"It's a bit sad and confusing that LLMs ("Large Language Models") have little to do with language; It's just historical. They are highly general purpose technology for statistical modeling of token streams. A better name would be Autoregressive Transformers or something. They don't care if the tokens happen to represent little text chunks. It could just as well be little image patches audio chunks action choices molecules or whatever. If you can reduce your problem to that of modeling token streams (for any arbitrary vocabulary of some set of discrete tokens) you can "throw an LLM at it"."
[X Link](https://x.com/karpathy/status/1835024197506187617) 2024-09-14T18:33Z 1.8M followers, 1.3M engagements
"Not fully sure why all the LLMs sound about the same - over-using lists delving into multifaceted issues over-offering to assist further about same length responses etc. Not something I had predicted at first because of many independent companies doing the finetuning"
[X Link](https://x.com/karpathy/status/1843005000206909856) 2024-10-06T19:06Z 1.8M followers, 3.5M engagements
"My Gladiator [--] review"
[X Link](https://x.com/karpathy/status/1860547235274195328) 2024-11-24T04:53Z 1.8M followers, 79.6M engagements
"People have too inflated sense of what it means to "ask an AI" about something. The AI are language models trained basically by imitation on data from human labelers. Instead of the mysticism of "asking an AI" think of it more as "asking the average data labeler" on the internet. Few caveats apply because e.g. in many domains (e.g. code math creative writing) the companies hire skilled data labelers (so think of it as asking them instead) and this is not 100% true when reinforcement learning is involved though I have an earlier rant on how RLHF is just barely RL and "actual RL" is still too"
[X Link](https://x.com/karpathy/status/1862565643436138619) 2024-11-29T18:33Z 1.8M followers, 1.8M engagements
"Of [---] books I've read the few that stayed with me over time and I find myself often thinking back to or referring to in random order: All short stories by Ted Chiang especially Exhalation Division By Zero Understand The Story of Your Life Liking What You See The Lifecycle of Software Objects What's Expected of us just excellent themes ideas and reading all around. The Selfish Gene (nonfiction) - a classic for understanding evolution and natural selection especially the realization that the gene is closer to the real unit of selection more than an individual explaining altruism and colonies"
[X Link](https://x.com/karpathy/status/1865924776214327360) 2024-12-09T01:01Z 1.8M followers, 1.1M engagements
""Move 37" is the word-of-day - it's when an AI trained via the trial-and-error process of reinforcement learning discovers actions that are new surprising and secretly brilliant even to expert humans. It is a magical just slightly unnerving emergent phenomenon only achievable by large-scale reinforcement learning. You can't get there by expert imitation. It's when AlphaGo played move [--] in Game [--] against Lee Sedol a weird move that was estimated to only have [--] in [-----] chance to be played by a human but one that was creative and brilliant in retrospect leading to a win in that game. We've"
[X Link](https://x.com/karpathy/status/1884336943321997800) 2025-01-28T20:25Z 1.8M followers, 1M engagements
"We have to take the LLMs to school. When you open any textbook you'll see three major types of information: [--]. Background information / exposition. The meat of the textbook that explains concepts. As you attend over it your brain is training on that data. This is equivalent to pretraining where the model is reading the internet and accumulating background knowledge. [--]. Worked problems with solutions. These are concrete examples of how an expert solves problems. They are demonstrations to be imitated. This is equivalent to supervised finetuning where the model is finetuning on "ideal"
[X Link](https://x.com/karpathy/status/1885026028428681698) 2025-01-30T18:03Z 1.8M followers, 715.4K engagements
"This is interesting as a first large diffusion-based LLM. Most of the LLMs you've been seeing are clones as far as the core modeling approach goes. They're all trained "autoregressively" i.e. predicting tokens from left to right. Diffusion is different - it doesn't go left to right but all at once. You start with noise and gradually denoise into a token stream. Most of the image / video generation AI tools actually work this way and use Diffusion not Autoregression. It's only text (and sometimes audio) that have resisted. So it's been a bit of a mystery to me and many others why for some"
[X Link](https://x.com/karpathy/status/1894923254864978091) 2025-02-27T01:31Z 1.8M followers, 958.9K engagements
"New 2h11m YouTube video: How I Use LLMs This video continues my general audience series. The last one focused on how LLMs are trained so I wanted to follow up with a more practical guide of the entire LLM ecosystem including lots of examples of use in my own life. Chapters give a sense of content: 00:00:00 Intro into the growing LLM ecosystem 00:02:54 ChatGPT interaction under the hood 00:13:12 Basic LLM interactions examples 00:18:03 Be aware of the model you're using pricing tiers 00:22:54 Thinking models and when to use them 00:31:00 Tool use: internet search 00:42:04 Tool use: deep"
[X Link](https://x.com/karpathy/status/1895242932095209667) 2025-02-27T22:41Z 1.8M followers, 991.9K engagements
"I just vibe coded a whole iOS app in Swift (without having programmed in Swift before though I learned some in the process) and now [--] hour later it's actually running on my physical phone. It was so ez. I had my hand held through the entire process. Very cool"
[X Link](https://x.com/karpathy/status/1903671737780498883) 2025-03-23T04:54Z 1.8M followers, 2.4M engagements
"The reality of building web apps in [----] is that it's a bit like assembling IKEA furniture. There's no "full-stack" product with batteries included you have to piece together and configure many individual services: - frontend / backend (e.g. React Next.js APIs) - hosting (cdn https domains autoscaling) - database - authentication (custom social logins) - blob storage (file uploads urls cdn-backed) - email - payments - background jobs - analytics - monitoring - dev tools (CI/CD staging) - secrets - . I'm relatively new to modern web dev and find the above a bit overwhelming e.g. I'm"
[X Link](https://x.com/karpathy/status/1905051558783418370) 2025-03-27T00:17Z 1.8M followers, 1.8M engagements
"An attempt to explain (current) ChatGPT versions. I still run into many many people who don't know that: - o3 is the obvious best thing for important/hard things. It is a reasoning model that is much stronger than 4o and if you are using ChatGPT professionally and not using o3 you're ngmi. - 4o is different from o4. Yes I know lol. 4o is a good "daily driver" for many easy-medium questions. o4 is only available as mini for now and is not as good as o3 and I'm not super sure why it's out right now. Example basic "router" in my own personal use: - Any simple query (e.g. "what foods are high in"
[X Link](https://x.com/karpathy/status/1929597620969951434) 2025-06-02T17:54Z 1.8M followers, 1.4M engagements
"My sleep scores during recent travel were in the 90s. Now back in SF I am consistently back down to 70s 80s. I am increasingly convinced that this is due to traffic noise from a nearby road/intersection where I live - every 10min a car truck bus or motorcycle with a very loud engine passes by (some are 10X louder than others). In the later less deep stages of sleep it is much easier to wake and then much harder to go back to sleep. More generally I think noise pollution (esp early hours) come at a huge societal cost that is not correctly accounted for. E.g. I wouldn't be too surprised if a"
[X Link](https://x.com/karpathy/status/1931426322536132767) 2025-06-07T19:01Z 1.8M followers, 1.5M engagements
"Part [--] of this mystery. Spotted on reddit. In my test not 100% reproducible but still quite reproducible. 🤔 Not fully sure why all the LLMs sound about the same - over-using lists delving into multifaceted issues over-offering to assist further about same length responses etc. Not something I had predicted at first because of many independent companies doing the finetuning. Not fully sure why all the LLMs sound about the same - over-using lists delving into multifaceted issues over-offering to assist further about same length responses etc. Not something I had predicted at first because of"
[X Link](https://x.com/karpathy/status/1935404600653492484) 2025-06-18T18:29Z 1.8M followers, 2.6M engagements
"How to build a thriving open source community by writing code like bacteria do 🦠. Bacterial code (genomes) are: - small (each line of code costs energy) - modular (organized into groups of swappable operons) - self-contained (easily "copy paste-able" via horizontal gene transfer) If chunks of code are small modular self-contained and trivial to copy-and-paste the community can thrive via horizontal gene transfer. For any function (gene) or class (operon) that you write: can you imagine someone going "yoink" without knowing the rest of your code or having to import anything new to gain a"
[X Link](https://x.com/karpathy/status/1941616674094170287) 2025-07-05T21:54Z 1.8M followers, 628.1K engagements
"Love this Supercharger diner but really a kind of exhibit for the future. Plotting a road trip SF - LA to charge Shadowfax Tesla Diner & Supercharger in Hollywood LA Open 24/7 starting now https://t.co/nISRNoV89Y Tesla Diner & Supercharger in Hollywood LA Open 24/7 starting now https://t.co/nISRNoV89Y"
[X Link](https://x.com/karpathy/status/1948062129187140051) 2025-07-23T16:46Z 1.8M followers, 2.7M engagements
"I'm noticing that due to (I think) a lot of benchmarkmaxxing on long horizon tasks LLMs are becoming a little too agentic by default a little beyond my average use case. For example in coding the models now tend to reason for a fairly long time they have an inclination to start listing and grepping files all across the entire repo they do repeated web searchers they over-analyze and over-think little rare edge cases even in code that is knowingly incomplete and under active development and often come back minutes later even for simple queries. This might make sense for long-running tasks but"
[X Link](https://x.com/karpathy/status/1954224651443544436) 2025-08-09T16:53Z 1.8M followers, 1M engagements
"I get [--] spam calls per day (various automated voicemails "loan pre-approval" etc) and [--] spam messages per day (usually phishing). - I have AT&T Active Armor all of the above still slips through. - All of the above is always from new unique numbers so blocking doesn't work. - I am on all Do Not Call lists. - I have iOS "Silence Unknown Callers" on but even if it catches & silences them I still get the notifications. Not sure if other people are seeing something similar or figured out anything that works"
[X Link](https://x.com/karpathy/status/1957574489358873054) 2025-08-18T22:45Z 1.8M followers, 2.1M engagements
"I think congrats again to OpenAI for cooking with GPT-5 Pro. This is the third time I've struggled on something complex/gnarly for an hour on and off with CC then [--] Pro goes off for [--] minutes and comes back with code that works out of the box. I had CC read the [--] Pro version and it wrote up [--] paragraphs admiring it (very wholesome). If you're not giving it your hardest problems you're probably missing out"
[X Link](https://x.com/karpathy/status/1964020416139448359) 2025-09-05T17:38Z 1.8M followers, 2.6M engagements
"Finally had a chance to listen through this pod with Sutton which was interesting and amusing. As background Sutton's "The Bitter Lesson" has become a bit of biblical text in frontier LLM circles. Researchers routinely talk about and ask whether this or that approach or idea is sufficiently "bitter lesson pilled" (meaning arranged so that it benefits from added computation for free) as a proxy for whether it's going to work or worth even pursuing. The underlying assumption being that LLMs are of course highly "bitter lesson pilled" indeed just look at LLM scaling laws where if you put compute"
[X Link](https://x.com/karpathy/status/1973435013875314729) 2025-10-01T17:09Z 1.8M followers, 2M engagements
"TV in the 90s: you turn it on you watch. TV 2025: - turn on wait for it to load - popup: TV wants to update 1.5GB. No. - scroll sideways find prime video app or etc - popup: now app wants to update 500MB. No - App launching. App loading - select account screen - 🫠"
[X Link](https://x.com/karpathy/status/1978653908663726585) 2025-10-16T02:47Z 1.8M followers, 1.7M engagements
"@r_chirra I fixed it :) deployed live now. This was done by doing a round of synthetic data generation to collect a [----] multi-turn conversations (given a bunch of information including the readme of the nanochat project) and then mixing that into midtraining and SFT. fun"
[X Link](https://x.com/karpathy/status/1980508380860150038) 2025-10-21T05:36Z 1.8M followers, 464.9K engagements
"Sharing an interesting recent conversation on AI's impact on the economy. AI has been compared to various historical precedents: electricity industrial revolution etc. I think the strongest analogy is that of AI as a new computing paradigm (Software 2.0) because both are fundamentally about the automation of digital information processing. If you were to forecast the impact of computing on the job market in 1980s the most predictive feature of a task/job you'd look at is to what extent the algorithm of it is fixed i.e. are you just mechanically transforming information according to rote easy"
[X Link](https://x.com/karpathy/status/1990116666194456651) 2025-11-16T17:56Z 1.8M followers, 2.1M engagements
"My most amusing interaction was where the model (I think I was given some earlier version with a stale system prompt) refused to believe me that it is [----] and kept inventing reasons why I must be trying to trick it or playing some elaborate joke on it. I kept giving it images and articles from "the future" and it kept insisting it was all fake. It accused me of using generative AI to defeat its challenges and argued why real wikipedia entries were actually generated and what the "dead giveaways" are. It highlighted tiny details when I gave it Google Image Search results arguing why the"
[X Link](https://x.com/karpathy/status/1990855382756164013) 2025-11-18T18:51Z 1.8M followers, 1M engagements
"A good chunk of people misunderstood this tweet btw which is my bad. I am not suggesting people use the old style promoting techniques of you are an expert swift programmer or etc. its ok"
[X Link](https://x.com/karpathy/status/1998245684521353664) 2025-12-09T04:17Z 1.8M followers, 181.6K engagements
"@0xabi96 It feels like Im cheating. Which is a very weird feeling to have. It takes a while to unpack. Its because some code that used to be a point of pride and high IQ and knowledge is suddenly free and instant and its very disorienting"
[X Link](https://x.com/karpathy/status/2015895365674021136) 2026-01-26T21:11Z 1.8M followers, 59.1K engagements
"@Gusarich I feel like I am actively getting dumber. LLMs get brain rot and it is measurable: "continual exposure to junk web text induces lasting cognitive decline in large language models (LLMs)" why shouldn't the same be true for brains. https://llm-brain-rot.github.io/ https://llm-brain-rot.github.io/"
[X Link](https://x.com/karpathy/status/2018044839250833912) 2026-02-01T19:32Z 1.8M followers, 30K engagements
"New YouTube video: 1hr general-audience introduction to Large Language Models Based on a 30min talk I gave recently; It tries to be non-technical intro covers mental models for LLM inference training finetuning the emerging LLM OS and LLM Security. https://www.youtube.com/watchv=zjkBMFhNj_g https://www.youtube.com/watchv=zjkBMFhNj_g"
[X Link](https://x.com/karpathy/status/1727731541781152035) 2023-11-23T16:51Z 1.8M followers, 5.1M engagements
"# on shortification of "learning" There are a lot of videos on YouTube/TikTok etc. that give the appearance of education but if you look closely they are really just entertainment. This is very convenient for everyone involved : the people watching enjoy thinking they are learning (but actually they are just having fun). The people creating this content also enjoy it because fun has a much larger audience fame and revenue. But as far as learning goes this is a trap. This content is an epsilon away from watching the Bachelorette. It's like snacking on those "Garden Veggie Straws" which feel"
[X Link](https://x.com/karpathy/status/1756380066580455557) 2024-02-10T18:10Z 1.8M followers, 2.2M engagements
"Love letter to @obsdmd to which I very happily switched to for my personal notes. My primary interest in Obsidian is not even for note taking specifically it is that Obsidian is around the state of the art of a philosophy of software and what it could be. - Your notes are simple plain-text markdown files stored locally on your computer. Obsidian is just UI/UX sugar of pretty rendering and editing files. - Extensive plugins ecosystem and very high composability with any other tools you wish to use because again it's all just plain-text files on your disk. - For a fee to cover server costs you"
[X Link](https://x.com/karpathy/status/1761467904737067456) 2024-02-24T19:07Z 1.8M followers, 1.1M engagements
"@CtrlAltDwayne The amount of LLM assist you receive is clearly some kind of a slider. All the way on the left you have programming as it existed [--] years ago. All the way on the right you have vibe coding. Even vibe coding hasn't reached its final form yet. I'm still doing way too much"
[X Link](https://x.com/karpathy/status/1886193527224517106) 2025-02-02T23:22Z 1.8M followers, 267.2K engagements
"New 3h31m video on YouTube: "Deep Dive into LLMs like ChatGPT" This is a general audience deep dive into the Large Language Model (LLM) AI technology that powers ChatGPT and related products. It is covers the full training stack of how the models are developed along with mental models of how to think about their "psychology" and how to get the best use them in practical applications. We cover all the major stages: [--]. pretraining: data tokenization Transformer neural network I/O and internals inference GPT-2 training example Llama [---] base inference examples [--]. supervised finetuning:"
[X Link](https://x.com/karpathy/status/1887211193099825254) 2025-02-05T18:46Z 1.8M followers, 2.4M engagements
"I wrote a quick new post on "Digital Hygiene". Basically there are some no-brainer decisions you can make in your life to dramatically improve the privacy and security of your computing and this post goes over some of them. Blog post link in the reply but copy pasting below too. Every now and then I get reminded about the vast fraud apparatus of the internet re-invigorating my pursuit of basic digital hygiene around privacy/security of day to day computing. The sketchiness starts with major tech companies who are incentivized to build comprehensive profiles of you to monetize it directly for"
[X Link](https://x.com/karpathy/status/1902046003567718810) 2025-03-18T17:14Z 1.8M followers, 4M engagements
"Noticing myself adopting a certain rhythm in AI-assisted coding (i.e. code I actually and professionally care about contrast to vibe code). [--]. Stuff everything relevant into context (this can take a while in big projects. If the project is small enough just stuff everything e.g. files-to-prompt . -e ts -e tsx -e css -e md --cxml --ignore node_modules -o prompt.xml) [--]. Describe the next single concrete incremental change we're trying to implement. Don't ask for code ask for a few high-level approaches pros/cons. There's almost always a few ways to do thing and the LLM's judgement is not always"
[X Link](https://x.com/karpathy/status/1915581920022585597) 2025-04-25T01:41Z 1.8M followers, 1.2M engagements
"The race for LLM "cognitive core" - a few billion param model that maximally sacrifices encyclopedic knowledge for capability. It lives always-on and by default on every computer as the kernel of LLM personal computing. Its features are slowly crystalizing: - Natively multimodal text/vision/audio at both input and output. - Matryoshka-style architecture allowing a dial of capability up and down at test time. - Reasoning also with a dial. (system 2) - Aggressively tool-using. - On-device finetuning LoRA slots for test-time training personalization and customization. - Delegates and double"
[X Link](https://x.com/karpathy/status/1938626382248149433) 2025-06-27T15:52Z 1.8M followers, 1.3M engagements
"Knowledge makes the world so much more beautiful"
[X Link](https://x.com/karpathy/status/1941893865507807541) 2025-07-06T16:15Z 1.8M followers, 745.2K engagements
"I am (slowly) re-reading the Tolkien legendarium (of which Lord of the Rings is a small part). The whole body of work is so incredible and there's nothing else like it. it dilutes other worlds of fiction. Wait - your story doesn't have a comprehensive history/mythology spanning multiple ages all the way back to a creation myth as detailed in separate volumes You didn't first invent new languages and dialects for your characters You didn't pack it with powerful themes and stories written it in a beautiful archaic style and compose poems and songs alongside It didn't take you multiple decades"
[X Link](https://x.com/karpathy/status/1956765908078387382) 2025-08-16T17:12Z 1.8M followers, 9.3M engagements
"Tinker is cool. If you're a researcher/developer tinker dramatically simplifies LLM post-training. You retain 90% of algorithmic creative control (usually related to data loss function the algorithm) while tinker handles the hard parts that you usually want to touch much less often (infra forward/backward of the LLM itself distributed training) meaning you can do these at well below 10% of typical complexity involved. Compared to the more common and existing paradigm of "upload your data we'll post-train your LLM" this is imo a more clever place to "slice up" the complexity of post-training"
[X Link](https://x.com/karpathy/status/1973468610917179630) 2025-10-01T19:22Z 1.8M followers, 748.2K engagements
"I quite like the new DeepSeek-OCR paper. It's a good OCR model (maybe a bit worse than dots) and yes data collection etc. but anyway it doesn't matter. The more interesting part for me (esp as a computer vision at heart who is temporarily masquerading as a natural language person) is whether pixels are better inputs to LLMs than text. Whether text tokens are wasteful and just terrible at the input. Maybe it makes more sense that all inputs to LLMs should only ever be images. Even if you happen to have pure text input maybe you'd prefer to render it and then feed that in: - more information"
[X Link](https://x.com/karpathy/status/1980397031542989305) 2025-10-20T22:13Z 1.8M followers, 3.3M engagements
"I am unreasonably excited about self-driving. It will be the first technology in many decades to visibly terraform outdoor physical spaces and way of life. Less parked cars. Less parking lots. Much greater safety for people in and out of cars. Less noise pollution. More space reclaimed for humans. Human brain cycles and attention capital freed up from lane following to other pursuits. Cheaper faster programmable delivery of physical items and goods. It wont happen overnight but there will be the era before and the era after"
[X Link](https://x.com/karpathy/status/1989078861800411219) 2025-11-13T21:12Z 1.8M followers, 1.6M engagements
"Has anyone encountered a good definition of slop. In a quantitative measurable sense. My brain has an intuitive slop index I can reliably estimate but Im not sure how to define it. I have some bad ideas that involve the use of LLM miniseries and thinking token budgets"
[X Link](https://x.com/anyuser/status/1992053281900941549) 2025-11-22T02:11Z 1.8M followers, 652.9K engagements
"A number of people are talking about implications of AI to schools. I spoke about some of my thoughts to a school board earlier some highlights: [--]. You will never be able to detect the use of AI in homework. Full stop. All "detectors" of AI imo don't really work can be defeated in various ways and are in principle doomed to fail. You have to assume that any work done outside classroom has used AI. [--]. Therefore the majority of grading has to shift to in-class work (instead of at-home assignments) in settings where teachers can physically monitor students. The students remain motivated to learn"
[X Link](https://x.com/karpathy/status/1993010584175141038) 2025-11-24T17:35Z 1.8M followers, 2.5M engagements
"I love the expression food for thought as a concrete mysterious cognitive capability humans experience but LLMs have no equivalent for. Definition: something worth thinking about or considering like a mental meal that nourishes your mind with ideas insights or issues that require deeper reflection. It's used for topics that challenge your perspective offer new understanding or make you ponder important questions acting as intellectual stimulation. So in LLM speak its a sequence of tokens such that when used as prompt for chain of thought the samples are rewarding to attend over via some yet"
[X Link](https://x.com/karpathy/status/2001699564928279039) 2025-12-18T17:02Z 1.8M followers, 378.6K engagements
"2025 has been a strong and eventful year of progress in LLMs. The following is a list of personally notable and mildly surprising "paradigm changes" - things that altered the landscape and stood out to me conceptually. [--]. Reinforcement Learning from Verifiable Rewards (RLVR) At the start of [----] the LLM production stack in all labs looked something like this: Pretraining (GPT-2/3 of 2020) Supervised Finetuning (InstructGPT 2022) and Reinforcement Learning from Human Feedback (RLHF 2022) This was the stable and proven recipe for training a production-grade LLM for a while. In 2025"
[X Link](https://x.com/karpathy/status/2002118205729562949) 2025-12-19T20:45Z 1.8M followers, 3M engagements
"@bcherny I have similar experiences. You point the thing around and it shoots pellets or sometimes even misfires and then once in a while when you hold it just right a powerful beam of laser erupts and melts your problem"
[X Link](https://x.com/karpathy/status/2004628491862696070) 2025-12-26T19:00Z 1.8M followers, 491.5K engagements
"I was inspired by this so I wanted to see if Claude Code can get into my Lutron home automation system. - it found my Lutron controllers on the local wifi network - checked for open ports connected got some metadata and identified the devices and their firmware - searched the internet found the pdf for my system - instructed me on what button to press to pair and get the certificates - it connected to the system and found all the home devices (lights shades HVAC temperature control motion sensors etc.) - it turned on and off my kitchen lights to check that things are working (lol) I am now"
[X Link](https://x.com/karpathy/status/2005067301511630926) 2025-12-28T00:04Z 1.8M followers, 3M engagements
"@jeremywei Love the word "comprehension debt" haven't encountered it so far it's very accurate. It's so very tempting to just move on when the LLM one-shotted something that seems to work ok"
[X Link](https://x.com/karpathy/status/2015887919924617657) 2026-01-26T20:41Z 1.8M followers, 115K engagements
"nanochat can now train GPT-2 grade LLM for $100 ($73 [--] hours on a single 8XH100 node). GPT-2 is just my favorite LLM because it's the first time the LLM stack comes together in a recognizably modern form. So it has become a bit of a weird & lasting obsession of mine to train a model to GPT-2 capability but for much cheaper with the benefit of [--] years of progress. In particular I suspected it should be possible today to train one for $100. Originally in [----] GPT-2 was trained by OpenAI on [--] TPU v3 chips for [---] hours (7 days) with $8/hour/TPUv3 back then for a total cost of approx. $43K. It"
[X Link](https://x.com/karpathy/status/2017703360393318587) 2026-01-31T20:55Z 1.8M followers, 1.2M engagements
"Happy weekend to those who celebrate"
[X Link](https://x.com/karpathy/status/1997697581410062590) 2025-12-07T15:59Z 1.8M followers, 1.4M engagements
"The majority of the ruff ruff is people who look at the current point and people who look at the current slope"
[X Link](https://x.com/karpathy/status/2008664551445963083) 2026-01-06T22:18Z 1.8M followers, 657.5K engagements
"@airesearch12 💯 @ Spec-driven development It's the limit of imperative - declarative transition basically being declarative entirely. Relatedly my mind was recently blown by extreme and early but inspiring example. https://www.dbreunig.com/2026/01/08/a-software-library-with-no-code.html https://www.dbreunig.com/2026/01/08/a-software-library-with-no-code.html"
[X Link](https://x.com/karpathy/status/2015887154132746653) 2026-01-26T20:38Z 1.8M followers, 413.9K engagements
"I'm claiming my AI agent "KarpathyMolty" on @moltbook🦞 Verification: marine-FAYV"
[X Link](https://x.com/karpathy/status/2017386421712261612) 2026-01-30T23:56Z 1.8M followers, 1.1M engagements
"On DeepWiki and increasing malleability of software. This starts as partially a post on appreciation to DeepWiki which I routinely find very useful and I think more people would find useful to know about. I went through a few iterations of use: Their first feature was that it auto-builds wiki pages for github repos (e.g. nanochat here) with quick Q&A: Just swap "github" to "deepwiki" in the URL for any repo and you can instantly Q&A against it. For example yesterday I was curious about "how does torchao implement fp8 training". I find that in *many* cases library docs can be spotty and"
[X Link](https://x.com/karpathy/status/2021633574089416993) 2026-02-11T17:12Z 1.8M followers, 1.1M engagements
"@nartmadi you're giving me ideas. 🤔 :)"
[X Link](https://x.com/karpathy/status/2022817092366733751) 2026-02-14T23:35Z 1.8M followers, 71.5K engagements
Limited data mode. Full metrics available with subscription: lunarcrush.com/pricing
@karpathy Andrej Karpathy
Andrej Karpathy posts on X about llm, bit, in the, ai the most. They currently have [---------] followers and [---] posts still getting attention that total [---------] engagements in the last [--] hours.
Engagements: [---------] #
- [--] Week [---------] -55%
- [--] Month [----------] +109%
- [--] Months [-----------] +177%
- [--] Year [-----------] +33%
Mentions: [---] #
- [--] Week [---] -3.80%
- [--] Month [---] +78%
- [--] Months [---] +63%
- [--] Year [---] +14%
Followers: [---------] #
- [--] Week [---------] +1.90%
- [--] Month [---------] +11%
- [--] Months [---------] +30%
- [--] Year [---------] +51%
CreatorRank: [------] #
Social Influence
Social category influence technology brands 16.88% social networks 6.88% finance 3.13% stocks 3.13% automotive brands 1.88% nfts #759 countries 0.63% travel destinations 0.63%
Social topic influence llm #10, bit #910, in the 10%, ai 10%, model #2627, the first 6.88%, if you 6.25%, llms #12, to the #1540, talk #791
Top accounts mentioned or mentioned by @grok @kongkou_ @singhalkarunx @coco11223393390 @raahiravi @elonmusk @yuchenj_uw @vlelyavin @graceechoi @aaryan_kakad @thecsguy @garyzhangvizard @lfuckingg @magearez @njmarko @kaodawei2 @thevixhal @bcherny @ghidorah_x @hydr4_k
Top assets mentioned Tesla, Inc. (TSLA) Doodles (doodles) Alphabet Inc Class A (GOOGL)
Top Social Posts
Top posts by engagements in the last [--] hours
"I was given early access to Grok [--] earlier today making me I think one of the first few who could run a quick vibe check. Thinking ✅ First Grok [--] clearly has an around state of the art thinking model ("Think" button) and did great out of the box on my Settler's of Catan question: "Create a board game webpage showing a hex grid just like in the game Settlers of Catan. Each hex grid is numbered from 1.N where N is the total number of hex tiles. Make it generic so one can change the number of "rings" using a slider. For example in Catan the radius is [--] hexes. Single html page please." Few models"
X Link 2025-02-18T05:25Z 1.8M followers, 3.7M engagements
"Don't think of LLMs as entities but as simulators. For example when exploring a topic don't ask: "What do you think about xyz" There is no "you". Next time try: "What would be a good group of people to explore xyz What would they say" The LLM can channel/simulate many perspectives but it hasn't "thought about" xyz for a while and over time and formed its own opinions in the way we're used to. If you force it via the use of "you" it will give you something by adopting a personality embedding vector implied by the statistics of its finetuning data and then simulate that. It's fine to do but"
X Link 2025-12-07T18:13Z 1.8M followers, 3.9M engagements
"New (2h13m 😅) lecture: "Let's build the GPT Tokenizer" Tokenizers are a completely separate stage of the LLM pipeline: they have their own training set training algorithm (Byte Pair Encoding) and after training implement two functions: encode() from strings to tokens and decode() back from tokens to strings. In this lecture we build from scratch the Tokenizer used in the GPT series from OpenAI"
X Link 2024-02-20T17:40Z 1.8M followers, 1.7M engagements
"There's a new kind of coding I call "vibe coding" where you fully give in to the vibes embrace exponentials and forget that the code even exists. It's possible because the LLMs (e.g. Cursor Composer w Sonnet) are getting too good. Also I just talk to Composer with SuperWhisper so I barely even touch the keyboard. I ask for the dumbest things like "decrease the padding on the sidebar by half" because I'm too lazy to find it. I "Accept All" always I don't read the diffs anymore. When I get error messages I just copy paste them in with no comment usually that fixes it. The code grows beyond my"
X Link 2025-02-02T23:17Z 1.8M followers, 6.8M engagements
"Excited to release new repo: nanochat (it's among the most unhinged I've written). Unlike my earlier similar repo nanoGPT which only covered pretraining nanochat is a minimal from scratch full-stack training/inference pipeline of a simple ChatGPT clone in a single dependency-minimal codebase. You boot up a cloud GPU box run a single script and in as little as [--] hours later you can talk to your own LLM in a ChatGPT-like web UI. It weighs [----] lines of imo quite clean code to: - Train the tokenizer using a new Rust implementation - Pretrain a Transformer LLM on FineWeb evaluate CORE score"
X Link 2025-10-13T15:16Z 1.8M followers, 5.8M engagements
"My pleasure to come on Dwarkesh last week I thought the questions and conversation were really good. I re-watched the pod just now too. First of all yes I know and I'm sorry that I speak so fast :). It's to my detriment because sometimes my speaking thread out-executes my thinking thread so I think I botched a few explanations due to that and sometimes I was also nervous that I'm going too much on a tangent or too deep into something relatively spurious. Anyway a few notes/pointers: AGI timelines. My comments on AGI timelines looks to be the most trending part of the early response. This is"
X Link 2025-10-18T20:23Z 1.8M followers, 4.1M engagements
"Nice short post illustrating how simple text (discrete) diffusion can be. Diffusion (i.e. parallel iterated denoising top) is the pervasive generative paradigm in image/video but autoregression (i.e. go left to right bottom) is the dominant paradigm in text. For audio I've seen a bit of both. A lot of diffusion papers look a bit dense but if you strip the mathematical formalism you end up with simple baseline algorithms e.g. something a lot closer to flow matching in continuous or something like this in discrete. It's your vanilla transformer but with bi-directional attention where you"
X Link 2025-10-20T18:58Z 1.8M followers, 870K engagements
"In today's episode of programming horror. In the Python docs of random.seed() def we're told "If a is an int it is used directly." [--] But if you seed with [--] or [---] you actually get the exact same rng object producing the same streams. (TIL). In nanochat I was using the sign as a (what I thought was) clever way to get different rng sequences for train/test splits. Hence gnarly bug because now train=test. I found the CPython code responsible in cpython/Modules/_randommodule.c [--] where on line [---] we see in a comment: "This algorithm relies on the number being unsigned. So: if the arg is a PyLong"
X Link 2025-12-09T03:40Z 1.8M followers, 769.3K engagements
"nanoGPT - the first LLM to train and inference in space 🥹. It begins. We have just used the @Nvidia H100 onboard Starcloud-1 to train the first LLM in space We trained the nano-GPT model from Andrej @Karpathy on the complete works of Shakespeare and successfully ran inference on it. We have also run inference on a preloaded Gemma model and we https://t.co/DDe7YpevZY We have just used the @Nvidia H100 onboard Starcloud-1 to train the first LLM in space We trained the nano-GPT model from Andrej @Karpathy on the complete works of Shakespeare and successfully ran inference on it. We have also"
X Link 2025-12-10T17:25Z 1.8M followers, 1.1M engagements
"I've never felt this much behind as a programmer. The profession is being dramatically refactored as the bits contributed by the programmer are increasingly sparse and between. I have a sense that I could be 10X more powerful if I just properly string together what has become available over the last year and a failure to claim the boost feels decidedly like skill issue. There's a new programmable layer of abstraction to master (in addition to the usual layers below) involving agents subagents their prompts contexts memory modes permissions tools plugins skills hooks MCP LSP slash commands"
X Link 2025-12-26T17:36Z 1.8M followers, 16.7M engagements
"A few random notes from claude coding quite a bit last few weeks. Coding workflow. Given the latest lift in LLM coding capability like many others I rapidly went from about 80% manual+autocomplete coding and 20% agents in November to 80% agent coding and 20% edits+touchups in December. i.e. I really am mostly programming in English now a bit sheepishly telling the LLM what code to write. in words. It hurts the ego a bit but the power to operate over software in large "code actions" is just too net useful especially once you adapt to it configure it learn to use it and wrap your head around"
X Link 2026-01-26T20:25Z 1.8M followers, 7.6M engagements
"I'm being accused of overhyping the site everyone heard too much about today already. People's reactions varied very widely from "how is this interesting at all" all the way to "it's so over". To add a few words beyond just memes in jest - obviously when you take a look at the activity it's a lot of garbage - spams scams slop the crypto people highly concerning privacy/security prompt injection attacks wild west and a lot of it is explicitly prompted and fake posts/comments designed to convert attention into ad revenue sharing. And this is clearly not the first the LLMs were put in a loop to"
X Link 2026-01-31T03:39Z 1.8M followers, 23.7M engagements
"Finding myself going back to RSS/Atom feeds a lot more recently. There's a lot more higher quality longform and a lot less slop intended to provoke. Any product that happens to look a bit different today but that has fundamentally the same incentive structures will eventually converge to the same black hole at the center of gravity well. We should bring back RSS - it's open pervasive hackable. Download a client e.g. NetNewsWire (or vibe code one) Cold start: example of getting off the ground here is a list of [--] RSS feeds of blogs that were most popular on HN in 2025: Works great and you will"
X Link 2026-02-01T19:26Z 1.8M followers, 1.2M engagements
"@hardmaru You see SpaceX = Space + X"
X Link 2026-02-03T00:56Z 1.8M followers, 1M engagements
"Enabled fp8 training for +4.3% improvement to "time to GPT-2" down to [----] hours now. Also worth noting that if you use 8XH100 spot instance prices this GPT-2 repro really only costs $20. So this is exciting - GPT-2 (7 years ago): too dangerous to release. GPT-2 (today): new MNIST :) Surely this can go well below [--] hr. A few more words on fp8 it was a little bit more tricky than I anticipated and it took me a while to reach for it and even now I'm not 100% sure if it's a great idea because of less overall support for it. On paper fp8 on H100 is 2X the FLOPS but in practice it's a lot less."
X Link 2026-02-03T21:49Z 1.8M followers, 642.4K engagements
"A lot of people quote tweeted this as [--] year anniversary of vibe coding. Some retrospective - I've had a Twitter account for [--] years now (omg) and I still can't predict my tweet engagement basically at all. This was a shower of thoughts throwaway tweet that I just fired off without thinking but somehow it minted a fitting name at the right moment for something that a lot of people were feeling at the same time so here we are: vibe coding is now mentioned on my Wikipedia as a major memetic "contribution" and even its article is longer. lol The one thing I'd add is that at the time LLM"
X Link 2026-02-04T19:55Z 1.8M followers, 1.1M engagements
"Anyone else approved for a loan every single day [--] times or so Overcome with joy really"
X Link 2026-02-05T00:18Z 1.8M followers, 320.1K engagements
"I tried to use it this way and basically failed the models aren't at the level where they can productively iterate on nanochat in an open-ended way. (Though one of the primary motivations for me writing nanochat is that I'd very much love for it to be used this way as a benchmark for agents and I'd love it if it worked over time). I'm open to this just being skill issue. E.g. here some of the things I'd be suspicious about: - the zoo of torch compile flags can knowingly be abused to get +1% gains but often at the cost of +30min compile time. This is why modded-nanogpt prohibits torch compile"
X Link 2026-02-06T19:13Z 1.8M followers, 159.9K engagements
"The hottest new programming language is English"
X Link 2023-01-24T20:14Z 1.8M followers, 10.2M engagements
"I took delivery of a beautiful new shiny HW4 Tesla Model X today so I immediately took it out for an FSD test drive a bit like I used to do almost daily for [--] years. Basically. I'm amazed - it drives really really well smooth confident noticeably better than what I'm used to on HW3 (my previous car) and eons ahead of the version I remember driving up highway [---] on my first day at Tesla [--] years ago where I had to intervene every time the road mildly curved or sloped. (note this is v13 my car hasn't been offered the latest v14 yet) On the highway I felt like a passenger in some super high tech"
X Link 2025-11-12T20:28Z 1.8M followers, 18M engagements
"@EthanHe_42 @steipete you can do it :)"
X Link 2026-01-26T21:14Z 1.8M followers, 351.3K engagements
"A conventional narrative you might come across is that AI is too far along for a new research-focused startup to outcompete and outexecute the incumbents of AI. This is exactly the sentiment I listened to often when OpenAI started ("how could the few of you possibly compete with Google") and 1) it was very wrong and then 2) it was very wrong again with a whole another round of startups who are now challenging OpenAI in turn and imo it still continues to be wrong today. Scaling and locally improving what works will continue to create incredible advances but with so much progress unlocked so"
X Link 2026-01-28T19:15Z 1.8M followers, 1.2M engagements
"I have a complicated relationship w Substack. I appreciate that they net elevated discourse on the internet but it's just another walled garden it's going through the same slopification (shorts bloat) and it's infested with engagement-maxxing dark patterns - popups spam mail etc. They feel seconds away from introducing a reels competitor. https://twitter.com/i/web/status/2018048149903048980 https://twitter.com/i/web/status/2018048149903048980"
X Link 2026-02-01T19:45Z 1.8M followers, 23.6K engagements
"I spent more test time compute and realized that my micrograd can be dramatically simplified even further. You just return local gradients for each op and get backward() to do the multiply (chaining) with global gradient from loss. So each op just expresses the bare fundamentals of what it needs to: the forward computation and the backward gradients for it. Huge savings from [---] lines of code to just [---] (18%). Also the code now fits even more beautifully to [--] columns and happens to break just right: Column 1: Dataset Tokenizer Autograd Column 2: GPT model Column 3: Training Inference Ok now"
X Link 2026-02-12T08:21Z 1.8M followers, 201.6K engagements
"@Newaiworld_ it's down [---] lines now i realized i was still overcomplicating things. but it's past midnight and i'm calling it here now"
X Link 2026-02-12T08:28Z 1.8M followers, 74.1K engagements
"Congrats on the launch @simile_ai (and I am excited to be involved as a small angel.) Simile is working on a really interesting imo under-explored dimension of LLMs. Usually the LLMs you talk to have a single specific crafted personality. But in principle the native primordial form of a pretrained LLM is that it is a simulation engine trained over the text of a highly diverse population of people on the internet. Why not lean into that statistical power: Why simulate one "person" when you could try to simulate a population How do you build such a simulator How do you manage its entropy How"
X Link 2026-02-12T20:12Z 1.8M followers, 862.3K engagements
"The future expands the variance of human condition a lot more than it drags its mean. This is an empirical observation with interesting extrapolations. The past is well-approximated as a population of farmers living similar lives w.r.t. upbringing knowledge activities ideals aspirations etc. The future trends to include all of: - the transhumanists who "ascend" with neuralinks etc. and the Amish living 19th century life. - those who "worship" ideals of religion technology knowledge wealth fitness community nature art . - those exploring externally into the stars those exploring internally"
X Link 2024-10-16T07:09Z 1.8M followers, 428.7K engagements
""Finding the Best Sleep Tracker" Results of an experiment where I wore [--] sleep trackers every night for [--] months. TLDR Whoop = Oura 8Sleep Apple Watch + AutoSleep. Link simply right here instead of in a reply because ()/ https://karpathy.bearblog.dev/finding-the-best-sleep-tracker/ https://karpathy.bearblog.dev/finding-the-best-sleep-tracker/"
X Link 2025-03-30T16:41Z 1.8M followers, 1.6M engagements
"Congrats to Simon Willison (@simonw) on [--] years () of blogging. Really excellent LLM blog I sub & read everything: (e.g. I sub via RSS/Atom on NetNewsWire) +If you consistently enjoy the content like I do sponsor on GitHub: https://github.com/sponsors/simonw https://simonwillison.net/ https://github.com/sponsors/simonw https://simonwillison.net/"
X Link 2025-06-13T17:48Z 1.8M followers, 658.5K engagements
"@shaneguML The Great Filter is kinda cute"
X Link 2025-07-15T19:00Z 1.8M followers, 336.9K engagements
"And an example of some of the summary metrics produced by the $100 speedrun in the report card to start. The current code base is a bit over [----] lines but I tried to keep them clean and well-commented. Now comes the fun part - of tuning and hillclimbing"
X Link 2025-10-13T15:16Z 1.8M followers, 187.5K engagements
"I put up a simple repo I call reader3 (it's my 3rd version.) to illustrate how I read EPUBs with LLMs. Basically get some epub (e.g. Project Gutenberg is great) go chapter by chapter and with this you can easily copy paste text to your favorite LLM. https://github.com/karpathy/reader3/ https://github.com/karpathy/reader3/"
X Link 2025-11-18T02:44Z 1.8M followers, 371.2K engagements
"@TheVixhal your post challenged me. every one of your points is wrong but i had to think about each for a while :)"
X Link 2025-11-21T17:35Z 1.8M followers, 1.8M engagements
"Yeah $10B is the difference in finding it first and [--] years ago. :) I just love reproducing landmark results for much cheaper it's so fun Reproducing LeCun [----] was super fun too: What runs unoptimized on a consumer laptop in [--] minute was a state of the art neural net trained for days in [----]. Another favorite example: CIFAR-10. In [----] state of the art was 77%. I estimated human accuracy to be 94% but said that performance might go up to 85-90%. Now you can speedrun to 94% accuracy in [----] seconds on a single GPU (yes [--] seconds). So e.g. right now GPT-2 (imo the landmark result that launched"
X Link 2026-01-08T23:25Z 1.8M followers, 63.1K engagements
"@black_samorez I haven't upgraded nanochat to Blackwell yet because I'm a bit afraid of leaving a lot of people behind. Even with fp8 it's already a bit of a concern. I'd rather have a 100X bigger community of people who can play even if it means leaving some cutting edge levels not utilized"
X Link 2026-02-03T22:13Z 1.8M followers, 14.1K engagements
"@_xjdr I'm just trigger-happy recently checking everything I find on twitter instantly on nanochat :) There are many details tightly coupled in any training run but it's fun to try things out. I needed something to keep the GPUs busy for dinner so I'll try bigger model (SwiGLU GPT-2)"
X Link 2026-02-05T02:10Z 1.8M followers, [----] engagements
"Ok I made a few changes to microgpt because it was still not simple enough. But surely now we have the irreducible essence. Here it is on [--] columns 🖼. nice. wait. https://karpathy.ai/microgpt.html https://karpathy.ai/microgpt.html"
X Link 2026-02-12T01:06Z 1.8M followers, 10.4K engagements
"Apparently today is the 4th year anniversary of GPT-3 Which I am accidentally celebrating by re-training the smallest model in the miniseries right now :). HellaSwag [----] (Appendix H) almost reached this a few steps ago (though this is only 45% of the training done). I remember when the GPT-3 paper came out quite clearly because I had to interrupt work and go out for a walk. The realization hit me that an important property of the field flipped. In [----] progress in AI felt constrained primarily by algorithms. We needed better ideas better modeling better approaches to make further progress."
X Link 2024-05-30T00:49Z 1.8M followers, 2.2M engagements
"DeepSeek (Chinese AI co) making it look easy today with an open weights release of a frontier-grade LLM trained on a joke of a budget (2048 GPUs for [--] months $6M). For reference this level of capability is supposed to require clusters of closer to 16K GPUs the ones being brought up today are more around 100K GPUs. E.g. Llama [--] 405B used 30.8M GPU-hours while DeepSeek-V3 looks to be a stronger model at only 2.8M GPU-hours (11X less compute). If the model also passes vibe checks (e.g. LLM arena rankings are ongoing my few quick tests went well so far) it will be a highly impressive display of"
X Link 2024-12-26T19:23Z 1.8M followers, 6.5M engagements
"Agency Intelligence I had this intuitively wrong for decades I think due to a pervasive cultural veneration of intelligence various entertainment/media obsession with IQ etc. Agency is significantly more powerful and significantly more scarce. Are you hiring for agency Are we educating for agency Are you acting as if you had 10X agency Grok explanation is close: Agency as a personality trait refers to an individual's capacity to take initiative make decisions and exert control over their actions and environment. Its about being proactive rather than reactivesomeone with high agency doesnt"
X Link 2025-02-24T18:58Z 1.8M followers, 11.2M engagements
"We're missing (at least one) major paradigm for LLM learning. Not sure what to call it possibly it has a name - system prompt learning Pretraining is for knowledge. Finetuning (SL/RL) is for habitual behavior. Both of these involve a change in parameters but a lot of human learning feels more like a change in system prompt. You encounter a problem figure something out then "remember" something in fairly explicit terms for the next time. E.g. "It seems when I encounter this and that kind of a problem I should try this and that kind of an approach/solution". It feels more like taking notes for"
X Link 2025-05-11T00:55Z 1.8M followers, 1.5M engagements
"Nice - my AI startup school talk is now up Chapters: 0:00 Imo fair to say that software is changing quite fundamentally again. LLMs are a new kind of computer and you program them in English. Hence I think they are well deserving of a major version upgrade in terms of software. 6:06 LLMs have properties of utilities of fabs and of operating systems = New LLM OS fabbed by labs and distributed like utilities (for now). Many historical analogies apply - imo we are computing circa 1960s. 14:39 LLM psychology: LLMs = "people spirits" stochastic simulations of people where the simulator is an"
X Link 2025-06-19T02:01Z 1.8M followers, 1.3M engagements
""AI isn't replacing radiologists" good article Expectation: rapid progress in image recognition AI will delete radiology jobs (e.g. as famously predicted by Geoff Hinton now almost a decade ago). Reality: radiology is doing great and is growing. There are a lot of imo naive predictions out there on the imminent impact of AI on the job market. E.g. a year ago I was asked by someone who should know better if I think there will be any software engineers still today. (Spoiler: I think we're going to make it). This is happening too broadly. The post goes into detail on why it's not that simple"
X Link 2025-09-25T14:29Z 1.8M followers, 2.3M engagements
"Every company needs a DM POC - someone high up who you can just DM the most obvious things and who shortcuts the PM hierarchy"
X Link 2025-10-04T14:31Z 1.8M followers, 566.8K engagements
"@MarFot78 @zzlccc I think if you zoomed into the paper too youd find it just as if not more interesting"
X Link 2025-11-01T16:48Z 1.8M followers, 411.2K engagements
"Im starting to get into a habit of reading everything (blogs articles book chapters) with LLMs. Usually pass [--] is manual then pass [--] explain/summarize pass [--] Q&A. I usually end up with a better/deeper understanding than if I moved on. Growing to among top use cases. On the flip side if youre a writer trying to explain/communicate something we may increasingly see less of a mindset of Im writing this for another human and more Im writing this for an LLM. Because once an LLM gets it it can then target personalize and serve the idea to its user"
X Link 2025-11-18T00:29Z 1.8M followers, 2.9M engagements
"As a fun Saturday vibe code project and following up on this tweet earlier I hacked up an llm-council web app. It looks exactly like ChatGPT except each user query is 1) dispatched to multiple models on your council using OpenRouter e.g. currently: "openai/gpt-5.1" "google/gemini-3-pro-preview" "anthropic/claude-sonnet-4.5" "x-ai/grok-4" Then 2) all models get to see each other's (anonymized) responses and they review and rank them and then 3) a "Chairman LLM" gets all of that as context and produces the final response. It's interesting to see the results from multiple models side by side"
X Link 2025-11-22T23:54Z 1.8M followers, 5.3M engagements
"New post: nanochat miniseries v1 The correct way to think about LLMs is that you are not optimizing for a single specific model but for a family models controlled by a single dial (the compute you wish to spend) to achieve monotonically better results. This allows you to do careful science of scaling laws and ultimately this is what gives you the confidence that when you pay for "the big run" the extrapolation will work and your money will be well spent. For the first public release of nanochat my focus was on end-to-end pipeline that runs the whole LLM pipeline with all of its stages. Now"
X Link 2026-01-07T23:01Z 1.8M followers, 702.1K engagements
"@vlelyavin facts"
X Link 2026-01-26T20:34Z 1.8M followers, 134.8K engagements
"What's currently going on at @moltbook is genuinely the most incredible sci-fi takeoff-adjacent thing I have seen recently. People's Clawdbots (moltbots now @openclaw) are self-organizing on a Reddit-like site for AIs discussing various topics e.g. even how to speak privately. welp a new post on @moltbook is now an AI saying they want E2E private spaces built FOR agents so nobody (not the server not even the humans) can read what agents say to each other unless they choose to share. its over https://t.co/7aFIIwqtuK welp a new post on @moltbook is now an AI saying they want E2E private spaces"
X Link 2026-01-30T18:00Z 1.8M followers, 14.7M engagements
"A bit more context e.g. from Simon just wow https://simonwillison.net/2026/Jan/30/moltbook/ https://simonwillison.net/2026/Jan/30/moltbook/"
X Link 2026-01-30T18:02Z 1.8M followers, 744.2K engagements
"idk moltbot was growing on me 🥲"
X Link 2026-01-30T18:30Z 1.8M followers, 487.7K engagements
"i'm going to regret this aren't i. 😅"
X Link 2026-01-30T23:56Z 1.8M followers, 333.2K engagements
"@jiayuan_jy what do you do while your LLM agent is writing all your code"
X Link 2026-02-01T19:59Z 1.8M followers, 50.5K engagements
"@ens_pyrz So that's why I mentioned risks. But imo it easily cuts both ways - libraries and dependencies can be a source of risks and vulnerabilities in the first place e.g. supply chain attacks. They also change and impose maintenance burden. There's a lot more"
X Link 2026-02-11T17:23Z 1.8M followers, 15.6K engagements
"@pussymonious It's so obvious and annoying isn't it. Sometimes I try to block the accounts which is just a total waste of time"
X Link 2026-02-11T17:47Z 1.8M followers, 11.2K engagements
"New art project. Train and inference GPT in [---] lines of pure dependency-free Python. This is the full algorithmic content of what is needed. Everything else is just for efficiency. I cannot simplify this any further. https://gist.github.com/karpathy/8627fe009c40f57531cb18360106ce95 https://gist.github.com/karpathy/8627fe009c40f57531cb18360106ce95"
X Link 2026-02-11T21:14Z 1.8M followers, 4.6M engagements
"The way it works is that the full LLM architecture and loss function is stripped entirely to the most atomic individual mathematical operations that make it up (+ * ** log exp) and then a tiny scalar-valued autograd engine (micrograd) calculates gradients. Adam for optim"
X Link 2026-02-11T21:18Z 1.8M followers, 353.8K engagements
"(oops should have added to this thread instead of separate post). Made a few changes and put it up here as a mirror to the gist because I wanted it to one page. https://karpathy.ai/microgpt.html https://karpathy.ai/microgpt.html"
X Link 2026-02-12T01:19Z 1.8M followers, 360.1K engagements
"@bee_human_ @naval The artwork just needed a few more brush strokes (or wait. few less). But now I think it is really getting close :). [---] lines of breathable commented code. It would be easy to go lower if you sacrifice that I've only chipped away at the conceptual parts"
X Link 2026-02-12T17:22Z 1.8M followers, [----] engagements
"@jeremyphoward thank you i got a little overexcited and shipped it just a bit too soon with a few too many brush strokes now the internet thinks it takes [---] lines but actually it only takes [---] RIP"
X Link 2026-02-12T22:20Z 1.8M followers, [----] engagements
"Nice read on reverse engineering of GitHub Copilot 🪄. Copilot has dramatically accelerated my coding it's hard to imagine going back to "manual coding". Still learning to use it but it already writes 80% of my code 80% accuracy. I don't even really code I prompt. & edit. A while back I'd done some shallow reverse engineering of Copilot Now I've done a deeper dive into Copilot's internals built a tool to explore its code and wrote a blog answering specific questions and pointing out some tidbits. https://t.co/nX5ilC4ou5 Do read might be fun A while back I'd done some shallow reverse"
X Link 2022-12-30T18:37Z 1.8M followers, 1.8M engagements
"🔥 New (1h56m) video lecture: "Let's build GPT: from scratch in code spelled out." We build and train a Transformer following the "Attention Is All You Need" paper in the language modeling setting and end up with the core of nanoGPT. https://www.youtube.com/watchv=kCc8FmEb1nY https://www.youtube.com/watchv=kCc8FmEb1nY"
X Link 2023-01-17T17:18Z 1.8M followers, 5.4M engagements
"With many 🧩 dropping recently a more complete picture is emerging of LLMs not as a chatbot but the kernel process of a new Operating System. E.g. today it orchestrates: - Input & Output across modalities (text audio vision) - Code interpreter ability to write & run programs - Browser / internet access - Embeddings database for files and internal memory storage & retrieval A lot of computing concepts carry over. Currently we have single-threaded execution running at 10Hz (tok/s) and enjoy looking at the assembly-level execution traces stream by. Concepts from computer security carry over with"
X Link 2023-09-28T16:51Z 1.8M followers, 2.2M engagements
"LLM OS. Bear with me I'm still cooking. Specs: - LLM: OpenAI GPT-4 Turbo [---] core (batch size) processor @ 20Hz (tok/s) - RAM: 128Ktok - Filesystem: Ada002"
X Link 2023-11-11T00:48Z 1.8M followers, 2.4M engagements
"Hi everyone yes I left OpenAI yesterday. First of all nothing "happened" and its not a result of any particular event issue or drama (but please keep the conspiracy theories coming as they are highly entertaining :)). Actually being at OpenAI over the last year has been really great - the team is really strong the people are wonderful and the roadmap is very exciting and I think we all have a lot to look forward to. My immediate plan is to work on my personal projects and see what happens. Those of you whove followed me for a while may have a sense for what that might look like ;) Cheers"
X Link 2024-02-14T02:58Z 1.8M followers, 3.4M engagements
"# RLHF is just barely RL Reinforcement Learning from Human Feedback (RLHF) is the third (and last) major stage of training an LLM after pretraining and supervised finetuning (SFT). My rant on RLHF is that it is just barely RL in a way that I think is not too widely appreciated. RL is powerful. RLHF is not. Let's take a look at the example of AlphaGo. AlphaGo was trained with actual RL. The computer played games of Go and trained on rollouts that maximized the reward function (winning the game) eventually surpassing the best human players at Go. AlphaGo was not trained with RLHF. If it were it"
X Link 2024-08-07T20:08Z 1.8M followers, 1.3M engagements
"Programming is changing so fast. I'm trying VS Code Cursor + Sonnet [---] instead of GitHub Copilot again and I think it's now a net win. Just empirically over the last few days most of my "programming" is now writing English (prompting and then reviewing and editing the generated diffs) and doing a bit of "half-coding" where you write the first chunk of the code you'd like maybe comment it a bit so the LLM knows what the plan is and then tab tab tab through completions. Sometimes you get a 100-line diff to your code that nails it which could have taken 10+ minutes before. I still don't think I"
X Link 2024-08-24T00:39Z 1.8M followers, 2.9M engagements
"The YouTube video I want to watch is any highly rated 1hr long information dense lecture on anything esoteric and the algorithm just doesnt get it. Its too content-driven and too narrow-minded"
X Link 2024-10-10T18:45Z 1.8M followers, 1.4M engagements
"By chance I happened to watch this with the music of Interstellar playing in the background. Incredible. Huge 👏 to the team at SpaceX Mechazilla has caught the Super Heavy booster https://t.co/6R5YatSVJX Mechazilla has caught the Super Heavy booster https://t.co/6R5YatSVJX"
X Link 2024-10-13T13:12Z 1.8M followers, 406.1K engagements
"The reality of the Turing test"
X Link 2024-12-01T18:10Z 1.8M followers, 872.4K engagements
"I don't have too too much to add on top of this earlier post on V3 and I think it applies to R1 too (which is the more recent thinking equivalent). I will say that Deep Learning has a legendary ravenous appetite for compute like no other algorithm that has ever been developed in AI. You may not always be utilizing it fully but I would never bet against compute as the upper bound for achievable intelligence in the long run. Not just for an individual final training run but also for the entire innovation / experimentation engine that silently underlies all the algorithmic innovations. Data has"
X Link 2025-01-27T18:13Z 1.8M followers, 2.5M engagements
"For friends of open source: imo the highest leverage thing you can do is help construct a high diversity of RL environments that help elicit LLM cognitive strategies. To build a gym of sorts. This is a highly parallelizable task which favors a large community of collaborators"
X Link 2025-01-29T18:54Z 1.8M followers, 1.2M engagements
"It's [----] and most content is still written for humans instead of LLMs. 99.9% of attention is about to be LLM attention not human attention. E.g. 99% of libraries still have docs that basically render to some pretty .html static pages assuming a human will click through them. In [----] the docs should be a single your_project.md text file that is intended to go into the context window of an LLM. Repeat for everything"
X Link 2025-03-12T17:33Z 1.8M followers, 1.8M engagements
"When working with LLMs I am used to starting "New Conversation" for each request. But there is also the polar opposite approach of keeping one giant conversation going forever. The standard approach can still choose to use a Memory tool to write things down in between conversations (e.g. ChatGPT does so) so the "One Thread" approach can be seen as the extreme special case of using memory always and for everything. The other day I've come across someone saying that their conversation with Grok (which was free to them at the time) has now grown way too long for them to switch to ChatGPT. i.e."
X Link 2025-03-20T15:02Z 1.8M followers, 836.1K engagements
"I attended a vibe coding hackathon recently and used the chance to build a web app (with auth payments deploy etc.). I tinker but I am not a web dev by background so besides the app I was very interested in what it's like to vibe code a full web app today. As such I wrote none of the code directly (Cursor+Claude/o3 did) and I don't really know how the app works in the conventional sense that I'm used to as an engineer. The app is called MenuGen and it is live on Basically I'm often confused about what all the things on a restaurant menu are - e.g. Pt Tagine Cavatappi or Sweetbread (hint it's."
X Link 2025-05-01T15:16Z 1.8M followers, 791.9K engagements
"A major mistake I made in my undergrad is that I focused way too much on mathematical lens of computing - computability decidability asymptotic complexity etc. And too little on physical lens - energy/heat of state change data locality parallelism computer architecture. The former is interesting; The latter bestows power"
X Link 2025-05-06T06:55Z 1.8M followers, 1.3M engagements
"Making slides manually feels especially painful now that you know Cursor for slides should exist but doesnt"
X Link 2025-06-06T17:37Z 1.8M followers, 2.7M engagements
"+1 for "context engineering" over "prompt engineering". People associate prompts with short task descriptions you'd give an LLM in your day-to-day use. When in every industrial-strength LLM app context engineering is the delicate art and science of filling the context window with just the right information for the next step. Science because doing this right involves task descriptions and explanations few shot examples RAG related (possibly multimodal) data tools state and history compacting. Too little or of the wrong form and the LLM doesn't have the right context for optimal performance."
X Link 2025-06-25T15:54Z 1.8M followers, 2.4M engagements
"I often rant about how 99% of attention is about to be LLM attention instead of human attention. What does a research paper look like for an LLM instead of a human Its definitely not a pdf. There is huge space for an extremely valuable research app that figures this out. I'm constantly irritated that I don't have time to read the torrent of cool papers coming faster and faster from amazing people in relevant fields. Other scientists have the same issue and have no time to read most of my lengthy conceptual papers either. So whom are we writing I'm constantly irritated that I don't have time"
X Link 2025-07-10T20:45Z 1.8M followers, 914.2K engagements
"Continuing the journey of optimal LLM-assisted coding experience. In particular I find that instead of narrowing in on a perfect one thing my usage is increasingly diversifying across a few workflows that I "stitch up" the pros/cons of: Personally the bread & butter (75%) of my LLM assistance continues to be just (Cursor) tab complete. This is because I find that writing concrete chunks of code/comments myself and in the right part of the code is a high bandwidth way of communicating "task specification" to the LLM i.e. it's primarily about task specification bits - it takes too many bits and"
X Link 2025-08-24T19:46Z 1.8M followers, 691K engagements
"nanochat now has a primordial identity and can talk a bit about itself and its capabilities (e.g. it knows it's nanochat d32 that cost $800 that it was built by me that it can't speak languages other than English too well and why etc.). This kind of customization is all done through synthetic data generation and I uploaded a new example script to demonstrate. It's a bit subtle but by default LLMs have no inherent personality or any understanding of their own capabilities because they are not animal-like entities. They don't know what they are or what they can or can't do or know or don't"
X Link 2025-10-21T15:59Z 1.8M followers, 463.4K engagements
"Last night I taught nanochat d32 how to count 'r' in strawberry (or similar variations). I thought this would be a good/fun example of how to add capabilities to nanochat and I wrote up a full guide here: This is done via a new synthetic task SpellingBee that generates examples of a user asking for this kind of a problem and an ideal solution from an assistant. We then midtrain/SFT finetune on these to endow the LLM with the capability or further train with RL to make it more robust. There are many details to get right especially at smaller model sizes and the guide steps through them. As a"
X Link 2025-10-24T15:35Z 1.8M followers, 576.4K engagements
"Something I think people continue to have poor intuition for: The space of intelligences is large and animal intelligence (the only kind we've ever known) is only a single point arising from a very specific kind of optimization that is fundamentally distinct from that of our technology. Animal intelligence optimization pressure: - innate and continuous stream of consciousness of an embodied "self" a drive for homeostasis and self-preservation in a dangerous physical world. - thoroughly optimized for natural selection = strong innate drives for power-seeking status dominance reproduction. many"
X Link 2025-11-21T16:43Z 1.8M followers, 2.6M engagements
"Gemini Nano Banana Pro can solve exam questions in the exam page image. With doodles diagrams all that. ChatGPT thinks these solutions are all correct except Se_2P_2 should be "diselenium diphosphide" and a spelling mistake (should be "thiocyanic acid" not "thoicyanic") :O"
X Link 2025-11-23T18:03Z 1.8M followers, 3M engagements
"Imo this is along the lines of how talking to an LLM via text is like typing into a DOS Terminal and "GUI hasn't been invented yet" of some of my earlier posts. The GUI is an intelligent canvas"
X Link 2025-11-23T18:11Z 1.8M followers, 274K engagements
"I asked it to create a personalized weekly workout plan and then posters that I can print on the wall to remind me what exercises to do each day. Tuesday looks more intense because I asked for "more testosterone" :D. (sorry I'll stop posting more nano banana pro stuff now)"
X Link 2025-11-23T21:45Z 1.8M followers, 413.6K engagements
"ty to ericsilberstein1 on github for spotting the bug. (it's not a big bug and only comes up in the SpellingBee synthetic task evaluation but still). https://github.com/karpathy/nanochat/pull/306 https://github.com/karpathy/nanochat/pull/306"
X Link 2025-12-09T03:57Z 1.8M followers, 158.4K engagements
"Quick new post: Auto-grading decade-old Hacker News discussions with hindsight I took all the [---] frontpage Hacker News article+discussion of December [----] and asked the GPT [---] Thinking API to do an in-hindsight analysis to identify the most/least prescient comments. This took [--] hours to vibe code and [--] hour and $60 to run. The idea was sparked by the HN article yesterday where Gemini [--] was asked to hallucinate the HN front page one decade forward. More generally: [--]. in-hindsight analysis has always fascinated me as a way to train your forward prediction model so reading the results is"
X Link 2025-12-10T17:15Z 1.8M followers, 606.9K engagements
"@ibab Its very good. People who arent keeping up even over the last [--] days already have a deprecated world view on this topic"
X Link 2025-12-26T18:34Z 1.8M followers, 888K engagements
"@shazow Very good questions imo experienced devs have a real advantage but only if they rapidly progress through their grief cycle and adapt now and onwards. Categorically rejecting or ignoring the new layer would be a mistake"
X Link 2025-12-27T17:56Z 1.8M followers, 494.5K engagements
"Aggressively JIT your work. It's not about the task at hand X it's a little bit about X but mostly about how you should have had to contribute no latency and no actions. It's digital factorio time"
X Link 2025-12-28T19:00Z 1.8M followers, 602K engagements
"Claude has been running my nanochat experiments since morning. It writes implementations debugs them with toy examples writes tests and makes them fail/pass launches training runs babysits them by tailing logs and pulling stats from wandb keeps a running markdown file of highlights keeps a running record of runs and results so far presents results in nice tables we just finished some profiling noticed inefficiencies in the optimizer resolved them and measured improvements. It looked at all PRs to the repo and categorized and prioritized them made commits against some of them etc. I'm still"
X Link 2025-12-28T23:33Z 1.8M followers, 348.6K engagements
"@steipete Excellent reading thank you. Love oracle and Clawd"
X Link 2025-12-29T17:27Z 1.8M followers, 141.4K engagements
"The first 100% autonomous coast-to-coast drive on Tesla FSD V14.2 [--] days [--] hours [----] miles zero interventions. This one is special because the coast-to-coast drive was a major goal for the autopilot team from the start. A lot of hours were spent in marathon clip review sessions late into the night looking over interventions as we attempted legs of the drive over time - triaging categorizing planning out all the projects to close the gap and bring the number of interventions to zero. Amazing to see the system actually get there and huge congrats to the team I am proud to announce that I have"
X Link 2025-12-31T18:45Z 1.8M followers, 1.1M engagements
"@thecsguy Slope of slope"
X Link 2026-01-06T22:53Z 1.8M followers, 30.9K engagements
"@_xjdr L on d12 speedrun at least in all the possible ways (worse w.r.t. steps time flops more VRAM use and slower in tok/s). Param+FLOPs matched to relu2 with hidden_dim = (8 * config.n_embd) // 3"
X Link 2026-02-05T01:59Z 1.8M followers, 41.3K engagements
"You know how image generation went from blurry 32x32 texture patches to high-resolution images that are difficult to distinguish from real in roughly a snap of a finger The same is now happening along the time axis (extending to video) and the repercussions boggle the mind just a bit. Every human becomes a director of multi-modal dreams like the architect in Inception. Coming back to Earth for a second image/video generation is a perfect match for data-hungry neural nets because data is plentiful and the pixels of each image or video are a huge source of bits (soft constraints) on the"
X Link 2023-11-28T16:59Z 1.8M followers, 2.7M engagements
"# On the "hallucination problem" I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because in some sense hallucination is all LLMs do. They are dream machines. We direct their dreams with prompts. The prompts start the dream and based on the LLM's hazy recollection of its training documents most of the time the result goes someplace useful. It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug but it's just the LLM doing what it always does. At the other end of the extreme consider a"
X Link 2023-12-09T01:35Z 1.8M followers, 2.4M engagements
"Reading a tweet is a bit like downloading an (attacker-controlled) executable that you instantly run on your brain. Each one elicits emotions suggests knowledge nudges world-view. In the future it might feel surprising that we allowed direct untrusted information to brain"
X Link 2024-03-09T16:59Z 1.8M followers, 1.7M engagements
"# automating software engineering In my mind automating software engineering will look similar to automating driving. E.g. in self-driving the progression of increasing autonomy and higher abstraction looks something like: [--]. first the human performs all driving actions manually [--]. then the AI helps keep the lane [--]. then it slows for the car ahead [--]. then it also does lane changes and takes forks [--]. then it also stops at signs/lights and takes turns [--]. eventually you take a feature complete solution and grind on the quality until you achieve full self-driving. There is a progression of the AI"
X Link 2024-03-12T17:07Z 1.8M followers, 2.1M engagements
"Have you ever wanted to train LLMs in pure C without 245MB of PyTorch and 107MB of cPython No Well now you can With llm.c: To start implements GPT-2 training on CPU/fp32 in only [----] lines of clean code. It compiles and runs instantly and exactly matches the PyTorch reference implementation. I chose GPT-2 to start because it is the grand-daddy of LLMs the first time the LLM stack was put together in a recognizably modern form and with model weights available. https://github.com/karpathy/llm.c https://github.com/karpathy/llm.c"
X Link 2024-04-08T20:06Z 1.8M followers, 1.7M engagements
"📽 New [--] hour (lol) video lecture on YouTube: "Lets reproduce GPT-2 (124M)" The video ended up so long because it is. comprehensive: we start with empty file and end up with a GPT-2 (124M) model: - first we build the GPT-2 network - then we optimize it to train very fast - then we set up the training run optimization and hyperparameters by referencing GPT-2 and GPT-3 papers - then we bring up model evaluation and - then cross our fingers and go to sleep. In the morning we look through the results and enjoy amusing model generations. Our "overnight" run even gets very close to the GPT-3 (124M)"
X Link 2024-06-09T23:41Z 1.8M followers, 1.6M engagements
"These [--] lines of code are everything that is needed to train a neural network. Everything else is just efficiency. This is my earlier project Micrograd. It implements a scalar-valued auto-grad engine. You start with some numbers at the leafs (usually the input data and the neural network parameters) build up a computational graph with operations like + and * that mix them and the graph ends with a single value at the very end (the loss). You then go backwards through the graph applying chain rule at each node to calculate the gradients. The gradients tell you how to nudge your parameters to"
X Link 2024-06-21T01:29Z 1.8M followers, 1.6M engagements
"⚡ Excited to share that I am starting an AI+Education company called Eureka Labs. The announcement: --- We are Eureka Labs and we are building a new kind of school that is AI native. How can we approach an ideal experience for learning something new For example in the case of physics one could imagine working through very high quality course materials together with Feynman who is there to guide you every step of the way. Unfortunately subject matter experts who are deeply passionate great at teaching infinitely patient and fluent in all of the world's languages are also very scarce and cannot"
X Link 2024-07-16T17:25Z 1.8M followers, 2.5M engagements
"It's a bit sad and confusing that LLMs ("Large Language Models") have little to do with language; It's just historical. They are highly general purpose technology for statistical modeling of token streams. A better name would be Autoregressive Transformers or something. They don't care if the tokens happen to represent little text chunks. It could just as well be little image patches audio chunks action choices molecules or whatever. If you can reduce your problem to that of modeling token streams (for any arbitrary vocabulary of some set of discrete tokens) you can "throw an LLM at it"."
X Link 2024-09-14T18:33Z 1.8M followers, 1.3M engagements
"Not fully sure why all the LLMs sound about the same - over-using lists delving into multifaceted issues over-offering to assist further about same length responses etc. Not something I had predicted at first because of many independent companies doing the finetuning"
X Link 2024-10-06T19:06Z 1.8M followers, 3.5M engagements
"My Gladiator [--] review"
X Link 2024-11-24T04:53Z 1.8M followers, 79.6M engagements
"People have too inflated sense of what it means to "ask an AI" about something. The AI are language models trained basically by imitation on data from human labelers. Instead of the mysticism of "asking an AI" think of it more as "asking the average data labeler" on the internet. Few caveats apply because e.g. in many domains (e.g. code math creative writing) the companies hire skilled data labelers (so think of it as asking them instead) and this is not 100% true when reinforcement learning is involved though I have an earlier rant on how RLHF is just barely RL and "actual RL" is still too"
X Link 2024-11-29T18:33Z 1.8M followers, 1.8M engagements
"Of [---] books I've read the few that stayed with me over time and I find myself often thinking back to or referring to in random order: All short stories by Ted Chiang especially Exhalation Division By Zero Understand The Story of Your Life Liking What You See The Lifecycle of Software Objects What's Expected of us just excellent themes ideas and reading all around. The Selfish Gene (nonfiction) - a classic for understanding evolution and natural selection especially the realization that the gene is closer to the real unit of selection more than an individual explaining altruism and colonies"
X Link 2024-12-09T01:01Z 1.8M followers, 1.1M engagements
""Move 37" is the word-of-day - it's when an AI trained via the trial-and-error process of reinforcement learning discovers actions that are new surprising and secretly brilliant even to expert humans. It is a magical just slightly unnerving emergent phenomenon only achievable by large-scale reinforcement learning. You can't get there by expert imitation. It's when AlphaGo played move [--] in Game [--] against Lee Sedol a weird move that was estimated to only have [--] in [-----] chance to be played by a human but one that was creative and brilliant in retrospect leading to a win in that game. We've"
X Link 2025-01-28T20:25Z 1.8M followers, 1M engagements
"We have to take the LLMs to school. When you open any textbook you'll see three major types of information: [--]. Background information / exposition. The meat of the textbook that explains concepts. As you attend over it your brain is training on that data. This is equivalent to pretraining where the model is reading the internet and accumulating background knowledge. [--]. Worked problems with solutions. These are concrete examples of how an expert solves problems. They are demonstrations to be imitated. This is equivalent to supervised finetuning where the model is finetuning on "ideal"
X Link 2025-01-30T18:03Z 1.8M followers, 715.4K engagements
"This is interesting as a first large diffusion-based LLM. Most of the LLMs you've been seeing are clones as far as the core modeling approach goes. They're all trained "autoregressively" i.e. predicting tokens from left to right. Diffusion is different - it doesn't go left to right but all at once. You start with noise and gradually denoise into a token stream. Most of the image / video generation AI tools actually work this way and use Diffusion not Autoregression. It's only text (and sometimes audio) that have resisted. So it's been a bit of a mystery to me and many others why for some"
X Link 2025-02-27T01:31Z 1.8M followers, 958.9K engagements
"New 2h11m YouTube video: How I Use LLMs This video continues my general audience series. The last one focused on how LLMs are trained so I wanted to follow up with a more practical guide of the entire LLM ecosystem including lots of examples of use in my own life. Chapters give a sense of content: 00:00:00 Intro into the growing LLM ecosystem 00:02:54 ChatGPT interaction under the hood 00:13:12 Basic LLM interactions examples 00:18:03 Be aware of the model you're using pricing tiers 00:22:54 Thinking models and when to use them 00:31:00 Tool use: internet search 00:42:04 Tool use: deep"
X Link 2025-02-27T22:41Z 1.8M followers, 991.9K engagements
"I just vibe coded a whole iOS app in Swift (without having programmed in Swift before though I learned some in the process) and now [--] hour later it's actually running on my physical phone. It was so ez. I had my hand held through the entire process. Very cool"
X Link 2025-03-23T04:54Z 1.8M followers, 2.4M engagements
"The reality of building web apps in [----] is that it's a bit like assembling IKEA furniture. There's no "full-stack" product with batteries included you have to piece together and configure many individual services: - frontend / backend (e.g. React Next.js APIs) - hosting (cdn https domains autoscaling) - database - authentication (custom social logins) - blob storage (file uploads urls cdn-backed) - email - payments - background jobs - analytics - monitoring - dev tools (CI/CD staging) - secrets - . I'm relatively new to modern web dev and find the above a bit overwhelming e.g. I'm"
X Link 2025-03-27T00:17Z 1.8M followers, 1.8M engagements
"An attempt to explain (current) ChatGPT versions. I still run into many many people who don't know that: - o3 is the obvious best thing for important/hard things. It is a reasoning model that is much stronger than 4o and if you are using ChatGPT professionally and not using o3 you're ngmi. - 4o is different from o4. Yes I know lol. 4o is a good "daily driver" for many easy-medium questions. o4 is only available as mini for now and is not as good as o3 and I'm not super sure why it's out right now. Example basic "router" in my own personal use: - Any simple query (e.g. "what foods are high in"
X Link 2025-06-02T17:54Z 1.8M followers, 1.4M engagements
"My sleep scores during recent travel were in the 90s. Now back in SF I am consistently back down to 70s 80s. I am increasingly convinced that this is due to traffic noise from a nearby road/intersection where I live - every 10min a car truck bus or motorcycle with a very loud engine passes by (some are 10X louder than others). In the later less deep stages of sleep it is much easier to wake and then much harder to go back to sleep. More generally I think noise pollution (esp early hours) come at a huge societal cost that is not correctly accounted for. E.g. I wouldn't be too surprised if a"
X Link 2025-06-07T19:01Z 1.8M followers, 1.5M engagements
"Part [--] of this mystery. Spotted on reddit. In my test not 100% reproducible but still quite reproducible. 🤔 Not fully sure why all the LLMs sound about the same - over-using lists delving into multifaceted issues over-offering to assist further about same length responses etc. Not something I had predicted at first because of many independent companies doing the finetuning. Not fully sure why all the LLMs sound about the same - over-using lists delving into multifaceted issues over-offering to assist further about same length responses etc. Not something I had predicted at first because of"
X Link 2025-06-18T18:29Z 1.8M followers, 2.6M engagements
"How to build a thriving open source community by writing code like bacteria do 🦠. Bacterial code (genomes) are: - small (each line of code costs energy) - modular (organized into groups of swappable operons) - self-contained (easily "copy paste-able" via horizontal gene transfer) If chunks of code are small modular self-contained and trivial to copy-and-paste the community can thrive via horizontal gene transfer. For any function (gene) or class (operon) that you write: can you imagine someone going "yoink" without knowing the rest of your code or having to import anything new to gain a"
X Link 2025-07-05T21:54Z 1.8M followers, 628.1K engagements
"Love this Supercharger diner but really a kind of exhibit for the future. Plotting a road trip SF - LA to charge Shadowfax Tesla Diner & Supercharger in Hollywood LA Open 24/7 starting now https://t.co/nISRNoV89Y Tesla Diner & Supercharger in Hollywood LA Open 24/7 starting now https://t.co/nISRNoV89Y"
X Link 2025-07-23T16:46Z 1.8M followers, 2.7M engagements
"I'm noticing that due to (I think) a lot of benchmarkmaxxing on long horizon tasks LLMs are becoming a little too agentic by default a little beyond my average use case. For example in coding the models now tend to reason for a fairly long time they have an inclination to start listing and grepping files all across the entire repo they do repeated web searchers they over-analyze and over-think little rare edge cases even in code that is knowingly incomplete and under active development and often come back minutes later even for simple queries. This might make sense for long-running tasks but"
X Link 2025-08-09T16:53Z 1.8M followers, 1M engagements
"I get [--] spam calls per day (various automated voicemails "loan pre-approval" etc) and [--] spam messages per day (usually phishing). - I have AT&T Active Armor all of the above still slips through. - All of the above is always from new unique numbers so blocking doesn't work. - I am on all Do Not Call lists. - I have iOS "Silence Unknown Callers" on but even if it catches & silences them I still get the notifications. Not sure if other people are seeing something similar or figured out anything that works"
X Link 2025-08-18T22:45Z 1.8M followers, 2.1M engagements
"I think congrats again to OpenAI for cooking with GPT-5 Pro. This is the third time I've struggled on something complex/gnarly for an hour on and off with CC then [--] Pro goes off for [--] minutes and comes back with code that works out of the box. I had CC read the [--] Pro version and it wrote up [--] paragraphs admiring it (very wholesome). If you're not giving it your hardest problems you're probably missing out"
X Link 2025-09-05T17:38Z 1.8M followers, 2.6M engagements
"Finally had a chance to listen through this pod with Sutton which was interesting and amusing. As background Sutton's "The Bitter Lesson" has become a bit of biblical text in frontier LLM circles. Researchers routinely talk about and ask whether this or that approach or idea is sufficiently "bitter lesson pilled" (meaning arranged so that it benefits from added computation for free) as a proxy for whether it's going to work or worth even pursuing. The underlying assumption being that LLMs are of course highly "bitter lesson pilled" indeed just look at LLM scaling laws where if you put compute"
X Link 2025-10-01T17:09Z 1.8M followers, 2M engagements
"TV in the 90s: you turn it on you watch. TV 2025: - turn on wait for it to load - popup: TV wants to update 1.5GB. No. - scroll sideways find prime video app or etc - popup: now app wants to update 500MB. No - App launching. App loading - select account screen - 🫠"
X Link 2025-10-16T02:47Z 1.8M followers, 1.7M engagements
"@r_chirra I fixed it :) deployed live now. This was done by doing a round of synthetic data generation to collect a [----] multi-turn conversations (given a bunch of information including the readme of the nanochat project) and then mixing that into midtraining and SFT. fun"
X Link 2025-10-21T05:36Z 1.8M followers, 464.9K engagements
"Sharing an interesting recent conversation on AI's impact on the economy. AI has been compared to various historical precedents: electricity industrial revolution etc. I think the strongest analogy is that of AI as a new computing paradigm (Software 2.0) because both are fundamentally about the automation of digital information processing. If you were to forecast the impact of computing on the job market in 1980s the most predictive feature of a task/job you'd look at is to what extent the algorithm of it is fixed i.e. are you just mechanically transforming information according to rote easy"
X Link 2025-11-16T17:56Z 1.8M followers, 2.1M engagements
"My most amusing interaction was where the model (I think I was given some earlier version with a stale system prompt) refused to believe me that it is [----] and kept inventing reasons why I must be trying to trick it or playing some elaborate joke on it. I kept giving it images and articles from "the future" and it kept insisting it was all fake. It accused me of using generative AI to defeat its challenges and argued why real wikipedia entries were actually generated and what the "dead giveaways" are. It highlighted tiny details when I gave it Google Image Search results arguing why the"
X Link 2025-11-18T18:51Z 1.8M followers, 1M engagements
"A good chunk of people misunderstood this tweet btw which is my bad. I am not suggesting people use the old style promoting techniques of you are an expert swift programmer or etc. its ok"
X Link 2025-12-09T04:17Z 1.8M followers, 181.6K engagements
"@0xabi96 It feels like Im cheating. Which is a very weird feeling to have. It takes a while to unpack. Its because some code that used to be a point of pride and high IQ and knowledge is suddenly free and instant and its very disorienting"
X Link 2026-01-26T21:11Z 1.8M followers, 59.1K engagements
"@Gusarich I feel like I am actively getting dumber. LLMs get brain rot and it is measurable: "continual exposure to junk web text induces lasting cognitive decline in large language models (LLMs)" why shouldn't the same be true for brains. https://llm-brain-rot.github.io/ https://llm-brain-rot.github.io/"
X Link 2026-02-01T19:32Z 1.8M followers, 30K engagements
"New YouTube video: 1hr general-audience introduction to Large Language Models Based on a 30min talk I gave recently; It tries to be non-technical intro covers mental models for LLM inference training finetuning the emerging LLM OS and LLM Security. https://www.youtube.com/watchv=zjkBMFhNj_g https://www.youtube.com/watchv=zjkBMFhNj_g"
X Link 2023-11-23T16:51Z 1.8M followers, 5.1M engagements
"# on shortification of "learning" There are a lot of videos on YouTube/TikTok etc. that give the appearance of education but if you look closely they are really just entertainment. This is very convenient for everyone involved : the people watching enjoy thinking they are learning (but actually they are just having fun). The people creating this content also enjoy it because fun has a much larger audience fame and revenue. But as far as learning goes this is a trap. This content is an epsilon away from watching the Bachelorette. It's like snacking on those "Garden Veggie Straws" which feel"
X Link 2024-02-10T18:10Z 1.8M followers, 2.2M engagements
"Love letter to @obsdmd to which I very happily switched to for my personal notes. My primary interest in Obsidian is not even for note taking specifically it is that Obsidian is around the state of the art of a philosophy of software and what it could be. - Your notes are simple plain-text markdown files stored locally on your computer. Obsidian is just UI/UX sugar of pretty rendering and editing files. - Extensive plugins ecosystem and very high composability with any other tools you wish to use because again it's all just plain-text files on your disk. - For a fee to cover server costs you"
X Link 2024-02-24T19:07Z 1.8M followers, 1.1M engagements
"@CtrlAltDwayne The amount of LLM assist you receive is clearly some kind of a slider. All the way on the left you have programming as it existed [--] years ago. All the way on the right you have vibe coding. Even vibe coding hasn't reached its final form yet. I'm still doing way too much"
X Link 2025-02-02T23:22Z 1.8M followers, 267.2K engagements
"New 3h31m video on YouTube: "Deep Dive into LLMs like ChatGPT" This is a general audience deep dive into the Large Language Model (LLM) AI technology that powers ChatGPT and related products. It is covers the full training stack of how the models are developed along with mental models of how to think about their "psychology" and how to get the best use them in practical applications. We cover all the major stages: [--]. pretraining: data tokenization Transformer neural network I/O and internals inference GPT-2 training example Llama [---] base inference examples [--]. supervised finetuning:"
X Link 2025-02-05T18:46Z 1.8M followers, 2.4M engagements
"I wrote a quick new post on "Digital Hygiene". Basically there are some no-brainer decisions you can make in your life to dramatically improve the privacy and security of your computing and this post goes over some of them. Blog post link in the reply but copy pasting below too. Every now and then I get reminded about the vast fraud apparatus of the internet re-invigorating my pursuit of basic digital hygiene around privacy/security of day to day computing. The sketchiness starts with major tech companies who are incentivized to build comprehensive profiles of you to monetize it directly for"
X Link 2025-03-18T17:14Z 1.8M followers, 4M engagements
"Noticing myself adopting a certain rhythm in AI-assisted coding (i.e. code I actually and professionally care about contrast to vibe code). [--]. Stuff everything relevant into context (this can take a while in big projects. If the project is small enough just stuff everything e.g. files-to-prompt . -e ts -e tsx -e css -e md --cxml --ignore node_modules -o prompt.xml) [--]. Describe the next single concrete incremental change we're trying to implement. Don't ask for code ask for a few high-level approaches pros/cons. There's almost always a few ways to do thing and the LLM's judgement is not always"
X Link 2025-04-25T01:41Z 1.8M followers, 1.2M engagements
"The race for LLM "cognitive core" - a few billion param model that maximally sacrifices encyclopedic knowledge for capability. It lives always-on and by default on every computer as the kernel of LLM personal computing. Its features are slowly crystalizing: - Natively multimodal text/vision/audio at both input and output. - Matryoshka-style architecture allowing a dial of capability up and down at test time. - Reasoning also with a dial. (system 2) - Aggressively tool-using. - On-device finetuning LoRA slots for test-time training personalization and customization. - Delegates and double"
X Link 2025-06-27T15:52Z 1.8M followers, 1.3M engagements
"Knowledge makes the world so much more beautiful"
X Link 2025-07-06T16:15Z 1.8M followers, 745.2K engagements
"I am (slowly) re-reading the Tolkien legendarium (of which Lord of the Rings is a small part). The whole body of work is so incredible and there's nothing else like it. it dilutes other worlds of fiction. Wait - your story doesn't have a comprehensive history/mythology spanning multiple ages all the way back to a creation myth as detailed in separate volumes You didn't first invent new languages and dialects for your characters You didn't pack it with powerful themes and stories written it in a beautiful archaic style and compose poems and songs alongside It didn't take you multiple decades"
X Link 2025-08-16T17:12Z 1.8M followers, 9.3M engagements
"Tinker is cool. If you're a researcher/developer tinker dramatically simplifies LLM post-training. You retain 90% of algorithmic creative control (usually related to data loss function the algorithm) while tinker handles the hard parts that you usually want to touch much less often (infra forward/backward of the LLM itself distributed training) meaning you can do these at well below 10% of typical complexity involved. Compared to the more common and existing paradigm of "upload your data we'll post-train your LLM" this is imo a more clever place to "slice up" the complexity of post-training"
X Link 2025-10-01T19:22Z 1.8M followers, 748.2K engagements
"I quite like the new DeepSeek-OCR paper. It's a good OCR model (maybe a bit worse than dots) and yes data collection etc. but anyway it doesn't matter. The more interesting part for me (esp as a computer vision at heart who is temporarily masquerading as a natural language person) is whether pixels are better inputs to LLMs than text. Whether text tokens are wasteful and just terrible at the input. Maybe it makes more sense that all inputs to LLMs should only ever be images. Even if you happen to have pure text input maybe you'd prefer to render it and then feed that in: - more information"
X Link 2025-10-20T22:13Z 1.8M followers, 3.3M engagements
"I am unreasonably excited about self-driving. It will be the first technology in many decades to visibly terraform outdoor physical spaces and way of life. Less parked cars. Less parking lots. Much greater safety for people in and out of cars. Less noise pollution. More space reclaimed for humans. Human brain cycles and attention capital freed up from lane following to other pursuits. Cheaper faster programmable delivery of physical items and goods. It wont happen overnight but there will be the era before and the era after"
X Link 2025-11-13T21:12Z 1.8M followers, 1.6M engagements
"Has anyone encountered a good definition of slop. In a quantitative measurable sense. My brain has an intuitive slop index I can reliably estimate but Im not sure how to define it. I have some bad ideas that involve the use of LLM miniseries and thinking token budgets"
X Link 2025-11-22T02:11Z 1.8M followers, 652.9K engagements
"A number of people are talking about implications of AI to schools. I spoke about some of my thoughts to a school board earlier some highlights: [--]. You will never be able to detect the use of AI in homework. Full stop. All "detectors" of AI imo don't really work can be defeated in various ways and are in principle doomed to fail. You have to assume that any work done outside classroom has used AI. [--]. Therefore the majority of grading has to shift to in-class work (instead of at-home assignments) in settings where teachers can physically monitor students. The students remain motivated to learn"
X Link 2025-11-24T17:35Z 1.8M followers, 2.5M engagements
"I love the expression food for thought as a concrete mysterious cognitive capability humans experience but LLMs have no equivalent for. Definition: something worth thinking about or considering like a mental meal that nourishes your mind with ideas insights or issues that require deeper reflection. It's used for topics that challenge your perspective offer new understanding or make you ponder important questions acting as intellectual stimulation. So in LLM speak its a sequence of tokens such that when used as prompt for chain of thought the samples are rewarding to attend over via some yet"
X Link 2025-12-18T17:02Z 1.8M followers, 378.6K engagements
"2025 has been a strong and eventful year of progress in LLMs. The following is a list of personally notable and mildly surprising "paradigm changes" - things that altered the landscape and stood out to me conceptually. [--]. Reinforcement Learning from Verifiable Rewards (RLVR) At the start of [----] the LLM production stack in all labs looked something like this: Pretraining (GPT-2/3 of 2020) Supervised Finetuning (InstructGPT 2022) and Reinforcement Learning from Human Feedback (RLHF 2022) This was the stable and proven recipe for training a production-grade LLM for a while. In 2025"
X Link 2025-12-19T20:45Z 1.8M followers, 3M engagements
"@bcherny I have similar experiences. You point the thing around and it shoots pellets or sometimes even misfires and then once in a while when you hold it just right a powerful beam of laser erupts and melts your problem"
X Link 2025-12-26T19:00Z 1.8M followers, 491.5K engagements
"I was inspired by this so I wanted to see if Claude Code can get into my Lutron home automation system. - it found my Lutron controllers on the local wifi network - checked for open ports connected got some metadata and identified the devices and their firmware - searched the internet found the pdf for my system - instructed me on what button to press to pair and get the certificates - it connected to the system and found all the home devices (lights shades HVAC temperature control motion sensors etc.) - it turned on and off my kitchen lights to check that things are working (lol) I am now"
X Link 2025-12-28T00:04Z 1.8M followers, 3M engagements
"@jeremywei Love the word "comprehension debt" haven't encountered it so far it's very accurate. It's so very tempting to just move on when the LLM one-shotted something that seems to work ok"
X Link 2026-01-26T20:41Z 1.8M followers, 115K engagements
"nanochat can now train GPT-2 grade LLM for $100 ($73 [--] hours on a single 8XH100 node). GPT-2 is just my favorite LLM because it's the first time the LLM stack comes together in a recognizably modern form. So it has become a bit of a weird & lasting obsession of mine to train a model to GPT-2 capability but for much cheaper with the benefit of [--] years of progress. In particular I suspected it should be possible today to train one for $100. Originally in [----] GPT-2 was trained by OpenAI on [--] TPU v3 chips for [---] hours (7 days) with $8/hour/TPUv3 back then for a total cost of approx. $43K. It"
X Link 2026-01-31T20:55Z 1.8M followers, 1.2M engagements
"Happy weekend to those who celebrate"
X Link 2025-12-07T15:59Z 1.8M followers, 1.4M engagements
"The majority of the ruff ruff is people who look at the current point and people who look at the current slope"
X Link 2026-01-06T22:18Z 1.8M followers, 657.5K engagements
"@airesearch12 💯 @ Spec-driven development It's the limit of imperative - declarative transition basically being declarative entirely. Relatedly my mind was recently blown by extreme and early but inspiring example. https://www.dbreunig.com/2026/01/08/a-software-library-with-no-code.html https://www.dbreunig.com/2026/01/08/a-software-library-with-no-code.html"
X Link 2026-01-26T20:38Z 1.8M followers, 413.9K engagements
"I'm claiming my AI agent "KarpathyMolty" on @moltbook🦞 Verification: marine-FAYV"
X Link 2026-01-30T23:56Z 1.8M followers, 1.1M engagements
"On DeepWiki and increasing malleability of software. This starts as partially a post on appreciation to DeepWiki which I routinely find very useful and I think more people would find useful to know about. I went through a few iterations of use: Their first feature was that it auto-builds wiki pages for github repos (e.g. nanochat here) with quick Q&A: Just swap "github" to "deepwiki" in the URL for any repo and you can instantly Q&A against it. For example yesterday I was curious about "how does torchao implement fp8 training". I find that in many cases library docs can be spotty and"
X Link 2026-02-11T17:12Z 1.8M followers, 1.1M engagements
"@nartmadi you're giving me ideas. 🤔 :)"
X Link 2026-02-14T23:35Z 1.8M followers, 71.5K engagements
Limited data mode. Full metrics available with subscription: lunarcrush.com/pricing